Associate-Cloud-Engineer Google Cloud Certified - Associate Cloud Engineer Questions and Answers

In general, Google recommends that each instance that needs to call a Google API should run as a service account with the minimum permissions necessary for that instance to do its job. In practice, this means you should configure service accounts for your instances with the following process: Create a new service account rather than using the Compute Engine default service account. Grant IAM roles to that service account for only the resources that it needs. Configure the instance to run as that service account. Grant the instance the https://www.googleapis.com/auth/cloud-platform scope to allow full access to all Google Cloud APIs, so that the IAM permissions of the instance are completely determined by the IAM roles of the service account. Avoid granting more access than necessary and regularly check your service account permissions to make sure they are up-to-date.https://cloud.google.com/compute/docs/access/create-enable-service-accounts-for-instances#best_practices

As mentioned in Schema and data model, you should be careful when choosing a primary key to not accidentally create hotspots in your database. One cause of hotspots is having a column whose value monotonically increases as the first key part, because this results in all inserts occurring at the end of your key space. This pattern is undesirable because Cloud Spanner divides data among servers by key ranges, which means all your inserts will be directed at a single server that will end up doing all the work.https://cloud.google.com/spanner/docs/schema-design#primary-key-prevent-hotspots

https://cloud.google.com/deployment-manager/docs/configuration/create-basic-configuration

Folders are nodes in the Cloud Platform Resource Hierarchy. A folder can contain projects, other folders, or a combination of both. Organizations can use folders to group projects under the organization node in a hierarchy. For example, your organization might contain multiple departments, each with its own set of Google Cloud resources. Folders allow you to group these resources on a per-department basis. Folders are used to group resources that share common IAM policies. While a folder can contain multiple folders or resources, a given folder or resource can have exactly one parent.https://cloud.google.com/resource-manager/docs/creating-managing-folders

https://cloud.google.com/compute/docs/instance-groups

https://cloud.google.com/load-balancing/docs/network/transition-to-backend-services#console

In order to enable auto-healing, you need to group the instances into a managed instance group. Managed instance groups (MIGs) maintain the high availability of your applications by proactively keeping your virtual machine (VM) instances available. An auto-healing policy on the MIG relies on an application-based health check to verify that an application is responding as expected. If the auto-healer determines that an application isnt responding, the managed instance group automatically recreates that instance.

It is important to use separate health checks for load balancing and for auto-healing. Health checks for load balancing can and should be more aggressive because these health checks determine whether an instance receives user traffic. You want to catch non-responsive instances quickly, so you can redirect traffic if necessary. In contrast, health checking for auto-healing causes Compute Engine to proactively replace failing instances, so this health check should be more conservative than a load balancing health check.

Types of audit logs Cloud Audit Logs provides the following audit logs for each Cloud project, folder, and organization: Admin Activity audit logs Data Access audit logs System Event audit logs Policy Denied audit logs *Data Access audit logs contain API calls that read the configuration or metadata of resources, as well as user-driven API calls that create, modify, or read user-provided resource data.https://cloud.google.com/logging/docs/audit#types

https://cloud.google.com/logging/docs/audit#data-access Cloud Storage: When Cloud Storage usage logs are enabled, Cloud Storage writes usage data to the Cloud Storagebucket, which generates Data Access audit logs for the bucket. The generated Data Access audit log has its caller identity redacted.

The requirement is for private communication between a Cloud Run service and a GKE API, following best practices.

Option A exposes the GKE API to the public internet, which violates the "privately reach" requirement. Relying on dynamic IP allowlisting with Cloud Armor is complex and less secure than private networking.

Options B and C configure overly permissive firewall rules (allowing all egress or ingress) and do not establish the necessary private network path between Cloud Run (which normally runs outside your VPC) and the GKE cluster within your VPC.

Option D describes the standard Google-recommended pattern for this scenario:

Internal Application Load Balancer (ILB): Expose the GKE service (API) using an ILB. This gives the service a private IP address accessible only within the VPC network (or connected networks).

Cloud DNS: Create a private DNS zone and record pointing a fully qualified domain name (FQDN) to the ILB's private IP address. This allows services to reach the API via a stable name instead of an IP.

Serverless VPC Access Connector: This connector creates a bridge allowing serverless services like Cloud Run to send traffic into your VPC network.

Cloud Run Configuration: Configure the Cloud Run service to use the VPC Access connector. The application code can then call the GKE API using its private FQDN registered in Cloud DNS.

This setup ensures traffic flows entirely over private networks (within the VPC via the ILB and through the VPC Access connector), meeting the private communication requirement securely and reliably.

https://cloud.google.com/logging/docs/audit

Data Access audit logs Data Access audit logs contain API calls that read the configuration or metadata of resources, as well as user-driven API calls that create, modify, or read user-provided resource data.

https://cloud.google.com/logging/docs/audit#data-access

The best option is to attach a single service account to the compute instances and add minimal rights to the service account. Then, allow the service account to impersonate a Cloud Identity user with elevated permissions to create, update, or delete resources. This way, the service account can use short-lived access tokens to authenticate to Google Cloud APIs without needing to manage service account keys. This option follows the principle of least privilege and reduces the risk of credential leakage and misuse.

Option A is not recommended because it requires human intervention, which can slow down the CI/CD pipeline and introduce human errors. Option C is not secure because it grants all required IAM permissions to a single service account, which can increase the impact of a compromised key. Option D is not cost-effective because it requires creating and managing multiple service accounts and keys, as well as using a secret manager service.

1: https://cloud.google.com/iam/docs/impersonating-service-accounts

2: https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys

3: https://cloud.google.com/iam/docs/understanding-service-accounts

they require cloud storage for archival and the want to automate the process to upload new medical image to cloud storage, hence we go for gsutil to copy on-prem images to cloud storage and automate the process via cron job. whereas Pub/Sub listens to the changes in the Cloud Storage bucket and triggers the pub/sub topic, which is not required.

Comprehensive and Detailed In Depth Explanation:

The problem states that a subnet is nearing its IP address capacity, and the requirement is to expand it without downtime and with minimal operational cost, following Google-recommended practices.

A. Creating a second VPC with the same subnet IP range and peering: While VPC Network Peering allows communication between VPCs, having overlapping IP ranges is generally not recommended and can lead to routing complexities if not managed carefully. It also adds operational overhead of managing two VPCs. This is not the most straightforward or cost-effective solution for simply expanding IP capacity within the same logical network.

B. Deleting and recreating the subnet: Deleting a subnet that contains active VM instances will cause downtime for those instances, violating a key requirement.

C. Using the Google Cloud CLI tool to expand the primary IP range of your subnet: Google Cloud allows you to expand the primary IP range of an existing subnet after it's created, as long as there are no conflicting subnets in the VPC. This operation does not require deleting the subnet or restarting the existing VMs within it, thus avoiding downtime. It's a direct and cost-effective way to increase the available IP address space within the existing subnet. This is a Google-recommended practice for handling subnet capacity issues.

D. Permitting additional traffic with firewall rules: Firewall rules control network traffic based on IP ranges, protocols, and ports. They do not increase the number of available private IP addresses within the subnet. This option does not address the core issue of IP address exhaustion.

Therefore, expanding the primary IP range of the existing subnet using the Google Cloud CLI is the recommended solution that meets all the requirements: addressing IP capacity, minimizing operational costs, and avoiding downtime.

Google Cloud Documentation References:

Expanding Subnet IP Ranges: https://cloud.google.com/vpc/docs/expand-subnet - This documentation explicitly describes how to expand the IP range of an existing subnet without downtime. It outlines the prerequisites and steps involved using the gcloud CLI or the Google Cloud Console.

VPC Network Overview: https://cloud.google.com/vpc/docs/vpc - Provides context on subnet IP ranges and their management.

===========

"The Cloud Spanner to Cloud Storage Text template is a batch pipeline that reads in data from a Cloud Spanner table, optionally transforms the data via a JavaScript User Defined Function (UDF) that you provide, and writes it to Cloud Storage as CSV text files."

https://cloud.google.com/dataflow/docs/guides/templates/provided-batch#cloudspannertogcstext

"The Dataflow connector for Cloud Spanner lets you read data from and write data to Cloud Spanner in a Dataflow pipeline"

https://cloud.google.com/spanner/docs/dataflow-connector

https://cloud.google.com/bigquery/external-data-sources

You can generate Windows passwords using either the Google Cloud Console or the gcloud command-line tool. This option uses the right syntax to reset the windows password.

gcloud compute reset-windows-password windows-instance

Ref: https://cloud.google.com/compute/docs/instances/windows/creating-passwords-for-windows-instances#gcloud

Let's evaluate each option based on the requirement of sub-millisecond latency for globally stored IoT data:

A. Spanner with Caching: While Spanner offers strong consistency and global scalability, the base latency might not consistently be sub-millisecond for all read/write operations globally. Introducing caching adds complexity and doesn't guarantee sub-millisecond latency for all initial reads or cache misses.

B. Bigtable: Bigtable is a highly scalable NoSQL database service designed for low-latency, high-throughput workloads. It excels at storing and retrieving large volumes of time-series data, which is typical for IoT sensor data. Its architecture is optimized for single-key lookups and scans, providing consistent sub-millisecond latency, making it a strong candidate for this use case.

C. BigQuery: BigQuery is a fully managed, serverless data warehouse designed for analytical queries on large datasets. While it's excellent for analyzing IoT data in batch, it's not optimized for the low-latency, high-throughput ingestion and retrieval required for real-time IoT applications with sub-millisecond latency needs.

D. Cloud Storage with Cloud CDN: Cloud Storage is object storage and is not designed for low-latency transactional workloads. Cloud CDN is a content delivery network that caches contentcloser to users for faster delivery, but it's not suitable for the primary storage of rapidly incoming IoT sensor data requiring sub-millisecond write latency.

Google Cloud Documentation References:

Cloud Bigtable Overview: https://cloud.google.com/bigtable/docs/overview - This document highlights Bigtable's suitability for low-latency and high-throughput applications, including IoT. It mentions its ability to handle massive amounts of data with consistent performance.

Spanner Overview: https://cloud.google.com/spanner/docs/overview - While Spanner offers low latency, Bigtable is generally preferred for extremely high-throughput, low-latency use cases like raw sensor data ingestion due to its optimized architecture for such workloads.

BigQuery Overview: https://cloud.google.com/bigquery/docs/introduction - This emphasizes BigQuery's analytical capabilities rather than low-latency operational workloads.

Cloud Storage Overview: https://cloud.google.com/storage/docs/overview - This describes Cloud Storage as object storage, not ideal for sub-millisecond latency reads and writes required for real-time IoT data.

===========

Let's analyze each option based on the requirements: PostgreSQL compatibility, significant performance improvement, minimal schema/code changes, handling large data volumes, Google-recommended practices, and cost minimization:

A. Migrate your data to AlloyDB: AlloyDB for PostgreSQL is a fully managed, PostgreSQL-compatible database service that offers significant performance improvements over standard PostgreSQL due to its architectural optimizations. It is designed to handle large data volumes and minimizes the need for schema and application code changes as it's wire-compatible with PostgreSQL. This aligns well with the requirements for performance improvement, minimal changes, large data, and being a Google-recommended option for PostgreSQL workloads.

B. Migrate your data to Spanner: Spanner is a globally distributed, horizontally scalable database with strong consistency. While it offers excellent scalability and performance, it's not directly PostgreSQL-compatible. Migrating to Spanner would likely require significant schema and application code changes due to differences in data modeling and SQL dialect.

C. Migrate your data to Firebase: Firebase is a suite of mobile and web development tools, with its primary database offering being Firestore (a NoSQL document database) and Realtime Database. These are not PostgreSQL-compatible and would require substantial changes to the data model and application code.

D. Migrate your data to Bigtable: Bigtable is a highly scalable NoSQL wide-column store. It's not compatible with PostgreSQL and requires a completely different data model and application logic.

Therefore, AlloyDB is the most suitable option as it provides PostgreSQL compatibility for minimal migration effort, significant performance improvements, scalability for large data volumes, and is a recommended Google Cloud database service for PostgreSQL workloads.

Google Cloud Documentation References:

AlloyDB for PostgreSQL Overview: https://cloud.google.com/alloydb/docs/overview - This document highlights AlloyDB's PostgreSQL compatibility, performance benefits, scalability, and suitability for migrating existing PostgreSQL workloads.

Spanner Overview: https://cloud.google.com/spanner/docs/overview - This emphasizes Spanner's unique features and differences from traditional relational databases like PostgreSQL.

Firebase Documentation: https://firebase.google.com/docs - This outlines the features of Firebase, including Firestore and Realtime Database, highlighting their NoSQL nature and incompatibility with PostgreSQL.

Cloud Bigtable Overview: https://cloud.google.com/bigtable/docs/overview - This describes Bigtable as a NoSQL database, emphasizing its differences from relational databases like PostgreSQL.

===========

To set a default cluster for gcloud commands, run the following command: gcloud config set container/cluster CLUSTER_NAMEhttps://cloud.google.com/kubernetes-engine/docs/how-to/managing-clusters?hl=en

https://cloud.google.com/vpc/docs/vpc#manually_created_subnet_ip_ranges

https://blog.doit-intl.com/the-truth-behind-google-cloud-egress-traffic-6e8f57b5c2f8

(A) Object Lifecycle Management

Delete

The Delete action deletes an object when the object meets all conditions specified in the lifecycle rule.

Exception: In buckets with Object Versioning enabled, deleting the live version of an object causes it to become a noncurrent version, while deleting a noncurrent version deletes that version permanently.

https://cloud.google.com/storage/docs/lifecycle#delete

(B) Signed URLs

This page provides an overview of signed URLs, which you use to give time-limited resource access to anyone in possession of the URL, regardless of whether they have a Google account

https://cloud.google.com/storage/docs/access-control/signed-urls

https://cloud.google.com/compute/docs/instances/managing-instance-access

This is the most optimal solution. Rather than recreating all nodes, you create a new node pool with GPU enabled. You then modify the pod specification to target particular GPU types by adding node selector to your workloads Pod specification. YOu still have a single cluster so you pay Kubernetes cluster management fee for just one cluster thus minimizing the cost.

Ref: https://cloud.google.com/kubernetes-engine/docs/how-to/gpus

Ref: https://cloud.google.com/kubernetes-engine/pricing

Example:

apiVersion: v1

kind: Pod

metadata:

name:my-gpu-pod

spec:

containers:

name:my-gpu-container

image: nvidia/cuda:10.0-runtime-ubuntu18.04

command: [/bin/bash]

resources:

limits:

nvidia.com/gpu: 2

nodeSelector:

cloud.google.com/gke-accelerator: nvidia-tesla-k80# or nvidia-tesla-p100 or nvidia-tesla-p4 or nvidia-tesla-v100 or nvidia-tesla-t4

Keywords, Financial data (large data) used globally, data stored and queried using relational structure (SQL), clients should get exact identical copies(Strong Consistency), Multiple region, low latency to end user, select storage option to minimize latency.

Configure the ACLs on each image in Cloud Storage to give read-only access to the default Compute Engine service account. is not right.

As mentioned above, Container Registry ignores permissions set on individual objects within the storage bucket so this isnt going to work.

Ref: https://cloud.google.com/container-registry/docs/access-control

https://cloud.google.com/sdk/gcloud

https://cloud.google.com/sdk/docs/configurations#multiple_configurations

Keyword, Version, traffic splitting, App Engine supports traffic splitting for versions before releasing.

IAP controls access to your App Engine apps and Compute Engine VMs running on Google Cloud. It leverages user identity and the context of a request to determine if a user should be allowed access. IAP is a building block toward BeyondCorp, an enterprise security model that enables employees to work from untrusted networks without using a VPN.

By default, IAP uses Google identities and IAM. By leveraging Identity Platform instead, you can authenticate users with a wide range of external identity providers, such as:

Email/password

OAuth (Google, Facebook, Twitter, GitHub, Microsoft, etc.)

SAML

OIDC

Phone number

Custom

Anonymous

This is useful if your application is already using an external authentication system, and migrating your users to Google accounts is impractical.

https://cloud.google.com/iap/docs/using-tcp-forwarding#grant-permission

Google HTTP(S) Load Balancing has native support for the WebSocket protocol when you use HTTP or HTTPS, not HTTP/2, as the protocol to the backend.

Ref: https://cloud.google.com/load-balancing/docs/https#websocket_proxy_support

So the next possible step is to Meet with the cloud enablement team to discuss load balancer options.

We dont need to convert WebSocket code to use HTTP streaming or Redesign the application, as WebSocket support is offered by Google HTTP(S) Load Balancing. Reviewing the encryption requirements is a good idea but it has nothing to do with WebSockets.

Keywords: - continually -> Standard - mission-critical analytics -> dual-regional

Cloud Debugger helps inspect the state of an application, at any code location, without stopping or slowing down the running app //https://cloud.google.com/debugger/docs

Quickstart: using the Google Cloud Console

This page shows you how to perform basic tasks in Pub/Sub using the Google Cloud Console.

Note: If you are new to Pub/Sub, we recommend that you start with the interactive tutorial.

Before you begin

Set up a Cloud Console project.

Set up a project

Click to:

Create or select a project.

Enable the Pub/Sub API for that project.

You can view and manage these resources at any time in the Cloud Console.

Install and initialize the Cloud SDK.

Note: You can run the gcloud tool in the Cloud Console without installing the Cloud SDK. To run the gcloud tool in the Cloud Console, use Cloud Shell .

https://cloud.google.com/pubsub/docs/quickstart-console

The goal is to create a serverless, scalable, and asynchronous transaction processing system with minimal management overhead.

Serverless Requirement:Options involving installing Kafka on VMs (A, B) or using VM instances for processing (B, C) introduce management overhead associated with VMs (patching, scaling configuration, OS management) and Kafka cluster management, violating the serverless and minimal management criteria.

Asynchronous Requirement:Both Kafka and Pub/Sub can handle asynchronous messaging. However, Pub/Sub is Google Cloud's fully managed, serverless messaging service, inherently minimizing management overhead compared to self-managed Kafka on VMs.

Scalability and Processing:Cloud Run is a fully managed, serverless platform that automatically scales based on traffic, suitable for processing transactions without managing underlying infrastructure. VM instances require manual scaling configuration or managed instance groups, adding overhead.

Combining Pub/Sub for asynchronous message ingestion (fully managed, serverless) and Cloud Run for processing (fully managed, serverless, scalable) directly meets all requirements: serverless, scalable, asynchronous, and minimal management overhead. Option D is the only one that uses fully serverless components for both ingestion and processing.

https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-autoscaler#adding_a_node_pool_with_autoscaling

https://gtseres.medium.com/using-service-accounts-across-projects-in-gcp-cf9473fef8f0

You create the service account in proj-sa and take note of the service account email, then you go to proj-vm in IAM > ADD and add the service account's email as new member and give it the Compute Storage Admin role.

https://cloud.google.com/compute/docs/access/iam#compute.storageAdmin

The most direct and recommended way to get a granular breakdown of your Google Cloud costs in BigQuery is to enable Cloud Billing data export to BigQuery when you create or manage your Cloud Billing account. This automatically sets up a daily export of your billing data to a BigQuery dataset you specify.

Option A: Enabling the BigQuery API and managing IAM roles are necessary for interacting with BigQuery, but they don't automatically populate it with Cloud Billing data. Selecting a data location is also important for BigQuery datasets but is a separate step from enabling billing export.

Option B: The BigQuery Data Transfer Service is used for transferring data from various sources into BigQuery, but for Cloud Billing data, the direct export feature is the standard and simpler method.

Option D: Enabling Cloud Billing and linking an account makes billing data available in the Cloud Billing console, but it doesn't automatically export it to BigQuery for detailed analysis. You need to explicitly configure the BigQuery export.

Reference to Google Cloud Certified - Associate Cloud Engineer Documents:

The process of setting up Cloud Billing export to BigQuery is clearly documented in the Google Cloud Billing documentation, which is a fundamental area for the Associate Cloud Engineer certification. Understanding how to access and analyze billing data is crucial for cost management.

You can deploy to a different project by using –project flag.

By default, the service is deployed the current project configured via:

$ gcloud config set core/project PROJECT

To override this value for a single deployment, use the –project flag:

$ gcloud app deploy ~/my_app/app.yaml –project=PROJECT

Ref:https://cloud.google.com/sdk/gcloud/reference/app/deploy

Granting more permissions than necessary violates the principle of least privilege, a fundamental security best practice. While option A grants the necessary permissions (as subsets exist in two predefined roles), it might also grant more permissions than the Operations team strictly requires for their tasks on GKE and Cloud SQL. Option D is too broad; 'Admin' roles grant extensive permissions that likely exceed the specific needs.

Google Cloud's best practices strongly recommend adhering to the principle of least privilege. Creating a custom role allows you to precisely define the set of permissions the Operations team needs for their specific tasks on the GKE cluster and the Cloud SQL instance, without granting any unnecessary permissions. This minimizes the potential blast radius in case of accidental or malicious actions.

Google Cloud Documentation References:

IAM best practices: https://cloud.google.com/iam/docs/best-practices - This document explicitly recommends granting the minimum necessary permissions.

Creating and managing custom roles: https://cloud.google.com/iam/docs/creating-managing-custom-roles - This explains how to create roles tailored to specific job functions.

Understanding roles: https://cloud.google.com/iam/docs/understanding-roles - This outlines the concepts of predefined and custom roles and their use cases.

===========

"Billing Account Viewer access would usually be granted to finance teams, it provides access to spend information, but does not confer the right to link or unlink projects or otherwise manage the properties of the billing account."https://cloud.google.com/billing/docs/how-to/billing-access

A custom image belongs only to your project. To create an instance with a custom image, you must first have a custom image.

https://cloud.google.com/compute/docs/instance-groups/rolling-out-updates-to-managed-instance-groups#max_unavailable

The gcloud projects get-iam-policy command displays the IAM policy for a project, which includes the roles and members assigned to those roles. The Project Owner role grants full access to all resources and actions in the project. By using this command, you can review who has been granted this role and make any necessary changes. References:

1: Associate Cloud Engineer Certification Exam Guide | Learn - Google Cloud

2: gcloud projects get-iam-policy | Cloud SDK Documentation | Google Cloud

3: Understanding roles | Cloud IAM Documentation | Google Cloud

https://cloud.google.com/config-connector/docs/how-to/secrets#gcloud

M1 machine series Medium in-memory databases such as SAP HANA Tasks that require intensive use of memory with higher memory-to-vCPU ratios than the general-purpose high-memory machine types. In-memory databases and in-memory analytics, business warehousing (BW) workloads, genomics analysis, SQL analysis services. Microsoft SQL Server and similar databases.

https://cloud.google.com/compute/docs/machine-types

https://cloud.google.com/compute/docs/machine-types#:~:text=databases%20such%20as-,SAP%20HANA,-In%2Dmemory%20databases

https://www.sap.com/india/products/hana.html#:~:text=is%20SAP%20HANA-,in%2Dmemory,-database%3F

https://cloud.google.com/storage/docs/parallel-composite-uploads

https://cloud.google.com/storage/docs/uploads-downloads#parallel-composite-uploads

This command correctly specifies the duration that the signed url should be valid for by using the -d flag. The default is 1 hour so omitting the -d flag would have also resulted in the same outcome. Times may be specified with no suffix (default hours), or with s = seconds, m = minutes, h = hours, d = days. The max duration allowed is 7d.

Ref: https://cloud.google.com/storage/docs/gsutil/commands/signurl

Cloud Functions is Google Cloud’s event-driven serverless compute platform that lets you run your code locally or in the cloud without having to provision servers. Cloud Functions scales upor down, so you pay only for compute resources you use. Cloud Functions have excellent integration with Cloud Pub/Sub, lets you scale down to zero and is recommended by Google as the ideal serverless platform to use when dependent on Cloud Pub/Sub.

"If you’re building a simple API (a small set of functions to be accessed via HTTP or Cloud Pub/Sub), we recommend using Cloud Functions."

Ref: https://cloud.google.com/serverless-options

Google Cloud Storage Triggers

Cloud Functions can respond to change notifications emerging from Google Cloud Storage. These notifications can be configured to trigger in response to various events inside a bucket—object creation, deletion, archiving and metadata updates.

Note: Cloud Functions can only be triggered by Cloud Storage buckets in the same Google Cloud Platform project.

Event types

Cloud Storage events used by Cloud Functions are based on Cloud Pub/Sub Notifications for Google Cloud Storage and can be configured in a similar way.

Supported trigger type values are:

google.storage.object.finalize

google.storage.object.delete

google.storage.object.archive

google.storage.object.metadataUpdate

Object Finalize

Trigger type value: google.storage.object.finalize

This event is sent when a new object is created (or an existing object is overwritten, and a new generation of that object is created) in the bucket.

https://cloud.google.com/functions/docs/calling/storage#event_types

When applied to a dataset, this role provides the ability to read the dataset's metadata and list tables in the dataset. When applied to a project, this role also provides the ability to run jobs, including queries, within the project. A member with this role can enumerate their own jobs, cancel their own jobs, and enumerate datasets within a project. Additionally, allows the creation of new datasets within the project; the creator is granted the BigQuery Data Owner role (roles/bigquery.dataOwner) on these new datasets.

https://cloud.google.com/bigquery/docs/access-control

Coldline Storage is the perfect service to store audit logs from all the projects and is very cost-efficient as well. Coldline Storage is a very low-cost, highly durable storage service for storing infrequently accessed data.

An external HTTP(S) load balancer is a Google-recommended solution for distributing web traffic across multiple regions and zones, and providing high availability, scalability, and security for web applications. It supports both IPv4 and IPv6 addresses, and can handle SSL/TLS termination and encryption. It also integrates with Cloud CDN, Cloud Armor, and Cloud Identity-Aware Proxy for enhanced performance and protection. A managed instance group (MIG) can be used as a backend service for the HTTP(S) load balancer, and can automatically scale the number of VM instances based on the CPU load. A Cloud Storage bucket can also be used as a backend service for the HTTP(S) load balancer, and can serve static content such as images, videos, or HTML files. A URL map can be used to route requests to different backend services based on the path or host of the request. For example, a URL map can send requests for /static/* to the Cloud Storage bucket, and requests for /dynamic/* to the MIG. A managed SSL certificate can be used to secure the connection between the clients and the load balancer, and can be automatically provisioned and renewed by Google.

A is incorrect because an internal HTTP(S) load balancer is only visible within a VPC network, and not to the public internet. It is used for internal applications that need to communicate with other internal services. Identity-Aware Proxy is a service that provides secure access to web applications without using a VPN. It is not a load balancer, and it does not distribute user traffic.

B is incorrect because installing HTTPS certificates on the instance is not necessary, as the HTTP(S) load balancer can handle SSL/TLS termination and encryption. It is also more complex and less secure to manage the certificates on the instance level, as they need to be updated and synchronized across multiple instances.

D is incorrect because an external network load balancer is a TCP/UDP load balancer that operates at the network layer. It is not suitable for web applications that use HTTP(S) protocols, as it does not support SSL/TLS termination and encryption, URL maps, or Cloud Storage backends. It is also less efficient and scalable to forward the requests to the Cloud Storage from the web servers, as it adds an extra hop and latency.

HTTP(S) Load Balancing documentation

Setting up HTTP(S) Load Balancing with Cloud Storage

Creating and using SSL certificates

Choosing a load balancer

Comprehensive and Detailed Explanation From Exact Extract:

The goal is to create a landing zone facilitating private IP communication across production projects and apply organization-wide firewall rules, following best practices and minimizing operational costs.

Network Structure:Individual VPCs with Peering (A, B): While VPC Peering allows private connectivity, managing a full mesh or complex peering topology across many projects becomes operationally complex and can hit peering limits. It's not the recommended pattern for centralized connectivity in a landing zone.

Shared VPC (C, D): This is the Google-recommended practice for scenarios where resources from multiple projects need to communicate privately within a common VPC network. A central host project owns the network, and service projects use it. This simplifies network administration and connectivity.

Firewall Rules:Organization Policies (A, C): These enforce organizational constraints (e.g., disable external IPs, restrict locations) but do not define specific network firewall rules (like allowing TCP ports).

Hierarchical Firewall Policies (B, D): These allow defining firewall rules at the Organization or Folder level, which are inherited by resources in descendant projects/folders. This is the mechanism to apply consistent firewall rules (like allowing specific TCP ports) across all VMs inthe organization (or a specific folder) efficiently, without managing rules in each individual VPC or project.

Combining Shared VPC for the network structure (best practice for cross-project private communication and central management) with Hierarchical Firewall Policies (for applying organization-wide firewall rules) meets all requirements efficiently and follows Google recommendations.  

Best practices for persistent disk snapshots

You can create persistent disk snapshots at any time, but you can create snapshots more quickly and with greater reliability if you use the following best practices.

Creating frequent snapshots efficiently

Use snapshots to manage your data efficiently.

Create a snapshot of your data on a regular schedule to minimize data loss due to unexpected failure.

Improve performance by eliminating excessive snapshot downloads and by creating an image and reusing it.

Set your snapshot schedule to off-peak hours to reduce snapshot time.

Snapshot frequency limits

Creating snapshots from persistent disks

You can snapshot your disks at most once every 10 minutes. If you want to issue a burst of requests to snapshot your disks, you can issue at most 6 requests in 60 minutes.

If the limit is exceeded, the operation fails and returns the following error:

https://cloud.google.com/compute/docs/disks/snapshot-best-practices

Let's analyze each option:

A. Using gsutil: While gsutil can transfer data to Cloud Storage, for 150 TB of infrequently accessed data, direct transfer over the network might be slow and consume significant bandwidth, potentially impacting other network operations. It also lacks built-in mechanisms for filtering files based on age.

B. Using Cloud Interconnect and Filestore: Cloud Interconnect provides a dedicated connection, but Filestore is a fully managed NFS service primarily designed for high-performance file sharing for applications running in Google Cloud. Migrating 150 TB of infrequently accessed data to Filestore would be cost-inefficient compared to Cloud Storage and doesn't directly address the requirement of moving older than one year files.

C. Using Transfer Appliance: Transfer Appliance is suitable for very large datasets (petabytes) or when network connectivity is poor or unreliable. While it addresses bandwidth concerns, it involves a physical appliance and might be an overkill for 150 TB of data, especially if network connectivity is reasonable.

D. Using Storage Transfer Service: Storage Transfer Service is specifically designed for moving large amounts of data between online storage systems, including on-premises file systems and Cloud Storage. It offers features like filtering by file age, scheduling transfers, and bandwidth control, directly addressing all the requirements of the question: migrating infrequently accessed files older than one year to Cloud Storage, minimizing costs (by using appropriate Cloud Storage classes for infrequent access), and controlling bandwidth usage.

Google Cloud Documentation References:

Storage Transfer Service Overview: https://cloud.google.com/storage-transfer-service/docs/overview - This page details the capabilities and use cases of Storage Transfer Service, including transferring from on-premises.

Storage Transfer Service for on-premises data: https://cloud.google.com/storage-transfer-service/docs/on-prem-overview - This specifically covers transferring data from on-premises file systems.

Cloud Storage Classes: https://cloud.google.com/storage/docs/storage-classes - Understanding the different storage classes (Standard, Nearline, Coldline, Archive) is crucial for cost optimization of infrequently accessed data. Storage Transfer Service can be configured to move data to a cost-effective class like Nearline or Coldline.

===========

https://cloud.google.com/bigquery/docs/access-control#custom_roles

Custom roles enable you to enforce the principle of least privilege, ensuring that the user and service accounts in your organization have only the permissions essential to performing their intended functions.

Given that the new instance wants to access the application on the existing compute engine instance, these applications seem to be related so they should be within the same VPC. It is possible to have them in different VPCs and peer the VPCs but this is a lot of additional work and we can simplify this by choosing the option below (which is the answer)

1. Create a subnet in the same VPC, in europe-west1.

2. Create the new instance in the new subnet and use the first instance subnets private address as the endpoint. is the right answer.

We can create another subnet in the same VPC and this subnet is located in europe-west1. We can then spin up a new instance in this subnet. We also have to set up a firewall rule to allow communication between the two subnets. All instances in the two subnets with the same VPC can communicate through the internal IP Address

Ref: https://cloud.google.com/vpc

https://support.google.com/cloudidentity/answer/6262987?hl=en &ref_topic=7558767

Before you run the gcloud compute instances list command, you need to do two things: authenticate with your user account and set the default project for gcloud CLI.

To authenticate with your user account, you need to run gcloud auth login, enter your login credentials in the dialog window, and paste the received login token to gcloud CLI. This will authorize the gcloud CLI to access Google Cloud resources on your behalf1.

To set the default project for gcloud CLI, you need to run gcloud config set project $my_project, where $my_project is the ID of the project that contains the instances you want to list. This will saveyou from having to specify the project flag for every gcloud command2.

Option B is not recommended, because using a service account key increases the risk of credential leakage and misuse. It is also not necessary, because you can use your user account to authenticate to the gcloud CLI3. Option C is not correct, because there is no such thing as a Cloud Identity user account key. Cloud Identity is a service that provides identity and access management for Google Cloud users andgroups4. Option D is not required, because the gcloud compute instances list command does not depend on the default zone. You can list instances from all zones or filter by a specific zone using the --filter flag.

1: https://cloud.google.com/sdk/docs/authorizing

2: https://cloud.google.com/sdk/gcloud/reference/config/set

3: https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys

4: https://cloud.google.com/identity/docs/overview

https://cloud.google.com/sdk/gcloud/reference/compute/instances/list

The resource that you need to get access is in the other project.

roles/bigquery.dataViewer BigQuery Data Viewer

When applied to a table or view, this role provides permissions to:

Read data and metadata from the table or view.

This role cannot be applied to individual models or routines.

When applied to a dataset, this role provides permissions to:

Read the dataset's metadata and list tables in the dataset.

Read data and metadata from the dataset's tables.

When applied at the project or organization level, this role can also enumerate all datasets in the project. Additional roles, however, are necessary to allow the running of jobs.

https://cloud.google.com/bigquery/external-data-sources

An external data source is a data source that you can query directly from BigQuery, even though the data is not stored in BigQuery storage.

BigQuery supports the following external data sources:

Amazon S3

Azure Storage

Cloud Bigtable

Cloud Spanner

Cloud SQL

Cloud Storage

Drive

https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints This list constraint defines the set of domains that email addresses added to Essential Contacts can have. By default, email addresses with any domain can be added to Essential Contacts. The allowed/denied list must specify one or more domains of the form @example.com. If this constraint is active and configured with allowed values, only email addresses with a suffix matching one of the entries from the list of allowed domains can be added in Essential Contacts. This constraint has no effect on updating or removing existing contacts. constraints/essentialcontacts.allowedContactDomains

Pub/Sub is a scalable and reliable messaging service that can handle large volumes of data from different sources at different rates. It allows you to decouple the producers and consumers of the data, and provides a durable and persistent storage for the messages until they are delivered. Cloud Functions is a serverless platform that can execute code in response to events, such as messages published to a Pub/Sub topic. It can scale automatically based on the load, and you only pay for the resources you use. By using Pub/Sub and Cloud Functions, you can optimize the storage and processing of the votes, as you can handle the variable rates of incoming votes, process them in real time or near real time, and avoid managing servers or VMs. References:

Pub/Sub documentation

Cloud Functions documentation

Choosing a messaging service for Google Cloud

https://cloud.google.com/iam/docs/best-practices-for-using-and-managing-service-accounts

If an application uses third-party or custom identities and needs to access a resource, such as a BigQuery dataset or a Cloud Storage bucket, it must perform a transition between principals. Because Google Cloud APIs don't recognize third-party or custom identities, the application can't propagate the end-user's identity to BigQuery or Cloud Storage. Instead, the application has to perform the access by using a different Google identity.