Associate-Cloud-Engineer Google Cloud Certified - Associate Cloud Engineer Questions and Answers

Keyword need to look for

- "High Throughput",

- "Consistent",

- "Property based data insert/fetch like ngine status, distance traveled, fuel level, and more." which can be designed in column,

- "Large Scale Customer Base + Each Customer has multiple sensor which send event in seconds" This will go for pera bytes situation,

- Export data based on the time of the event.

- Atomic

o BigTable will fit all requirement.

o DataStore is not fully Atomic

o CloudStorage is not a option where we can export data based on time of event. We need another solution to do that

o FireStore can be used with MobileSDK.

App engine is a regional service, which means the infrastructure that runs your app(s) is located in a specific region and is managed by Google to be redundantly available across all the zones within that region. Once an app engine deployment is created in a region, it cant be changed. The only way is to create a new project and create an App Engine instance in europe-west3, send all user traffic to this instance and delete the app engine instance in us-central.

Ref: https://cloud.google.com/appengine/docs/locations

An external HTTP(S) load balancer is a Google-recommended solution for distributing web traffic across multiple regions and zones, and providing high availability, scalability, and security for web applications. It supports both IPv4 and IPv6 addresses, and can handle SSL/TLS termination and encryption. It also integrates with Cloud CDN, Cloud Armor, and Cloud Identity-Aware Proxy for enhanced performance and protection. A managed instance group (MIG) can be used as a backend service for the HTTP(S) load balancer, and can automatically scale the number of VM instances based on the CPU load. A Cloud Storage bucket can also be used as a backend service for the HTTP(S) load balancer, and can serve static content such as images, videos, or HTML files. A URL map can be used to route requests to different backend services based on the path or host of the request. For example, a URL map can send requests for /static/* to the Cloud Storage bucket, and requests for /dynamic/* to the MIG. A managed SSL certificate can be used to secure the connection between the clients and the load balancer, and can be automatically provisioned and renewed by Google.

A is incorrect because an internal HTTP(S) load balancer is only visible within a VPC network, and not to the public internet. It is used for internal applications that need to communicate with other internal services. Identity-Aware Proxy is a service that provides secure access to web applications without using a VPN. It is not a load balancer, and it does not distribute user traffic.

B is incorrect because installing HTTPS certificates on the instance is not necessary, as the HTTP(S) load balancer can handle SSL/TLS termination and encryption. It is also more complex and less secure to manage the certificates on the instance level, as they need to be updated and synchronized across multiple instances.

D is incorrect because an external network load balancer is a TCP/UDP load balancer that operates at the network layer. It is not suitable for web applications that use HTTP(S) protocols, as it does not support SSL/TLS termination and encryption, URL maps, or Cloud Storage backends. It is also less efficient and scalable to forward the requests to the Cloud Storage from the web servers, as it adds an extra hop and latency.

HTTP(S) Load Balancing documentation

Setting up HTTP(S) Load Balancing with Cloud Storage

Creating and using SSL certificates

Choosing a load balancer

From this source:https://cloud.google.com/billing/docs/how-to/custom-roles#permission_association_and_inheritance

"For example, associating a project with a billing account requires the billing.resourceAssociations.create permission on the billing account and also the resourcemanager.projects.createBillingAssignment permission on the project. This is because project permissions are required for actions where project owners control access, while billing account permissions are required for actions where billing account administrators control access. When both should be involved, both permissions are necessary."

https://developers.google.com/cloud-search/docs/guides/audit-logging-manual

As part of your workload, there might be certain VM instances that are critical to running your application or services, such as an instance running a SQL server, a server used as a license manager, and so on. These VM instances might need to stay running indefinitely so you need a way to protect these VMs from being deleted. By setting the deletionProtection flag, a VM instance can be protected from accidental deletion. If a user attempts to delete a VM instance for which you have set the deletionProtection flag, the request fails. Only a user that has been granted a role with compute.instances.create permission can reset the flag to allow the resource to be deleted.

Ref: https://cloud.google.com/compute/docs/instances/preventing-accidental-vm-deletion

Cloud Storage allows world-wide storage and retrieval of any amount of data at any time. We dont need to set up auto-scaling ourselves. Cloud Storage autoscaling is managed by GCP. Cloud Storage is an object store so it is suitable for storing photos. Cloud Storage allows world-wide storage and retrieval so cater well to our worldwide audience. Cloud storage provides us lifecycle rules that can be configured to automatically delete objects older than 30 days. This also fits our requirements. Finally, Google Cloud Storage offers several storage classes such as Nearline Storage ($0.01 per GB per Month) Coldline Storage ($0.007 per GB per Month) and Archive Storage ($0.004 per GB per month) which are significantly cheaper than any of the options above.

Ref: https://cloud.google.com/storage/docs

Ref: https://cloud.google.com/storage/pricing

Preventing Accidental VM Deletion This document describes how to protect specific VM instances from deletion by setting the deletionProtection property on an Instance resource. To learn more about VM instances, read the Instances documentation. As part of your workload, there might be certain VM instances that are critical to running your application or services, such as an instance running a SQL server, a server used as a license manager, and so on. These VM instances might need to stay running indefinitely so you need a way to protect these VMs from being deleted. By setting the deletionProtection flag, a VM instance can be protected from accidental deletion. If a user attempts to delete a VM instance for which you have set the deletionProtection flag, the request fails. Only a user that has been granted a role with compute.instances.create permission can reset the flag to allow the resource to be deleted.https://cloud.google.com/compute/docs/instances/preventing-accidental-vm-deletion

We need to apply the GCP Best practices. roles/browser Browser Read access to browse the hierarchy for a project, including the folder, organization, and IAM policy. This role doesn't include permission to view resources in the project.https://cloud.google.com/iam/docs/understanding-roles

https://cloud.google.com/iam/docs/understanding-service-accounts#using_service_accounts_with_compute_engine

https://cloud.google.com/storage/docs/access-control/iam-roles

Create an instance template for the instances so VMs have same specs. Set the "˜Automatic Restart' to on to VM automatically restarts upon crash. Set the "˜On-host maintenance' to Migrate VM instance. This will take care of VM during maintenance window. It will migrate VM instance making it highly available Add the instance template to an instance group so instances can be managed.

• onHostMaintenance: Determines the behavior when a maintenance event occurs that might cause your instance to reboot.

• [Default] MIGRATE, which causes Compute Engine to live migrate an instance when there is a maintenance event.

• TERMINATE, which stops an instance instead of migrating it.

• automaticRestart: Determines the behavior when an instance crashes or is stopped by the system.

• [Default] true, so Compute Engine restarts an instance if the instance crashes or is stopped.

• false, so Compute Engine does not restart an instance if the instance crashes or is stopped.

Enabling automatic restart ensures that compute engine instances are automatically restarted when they crash. And Enabling Migrate VM Instance enables live migrates i.e. compute instances are migrated during system maintenance and remain running during the migration.

Automatic Restart  If your instance is set to terminate when there is a maintenance event, or if your instance crashes because of an underlying hardware issue, you can set up Compute Engine to automatically restart the instance by setting the automaticRestart field to true. This setting does not apply if the instance is taken offline through a user action, such as calling sudo shutdown, or during a zone outage.

Ref: https://cloud.google.com/compute/docs/instances/setting-instance-scheduling-options#autorestart

Enabling the Migrate VM Instance option migrates your instance away from an infrastructure maintenance event, and your instance remains running during the migration.Your instance might experience a short period of decreased performance, although generally, most instances should not notice any difference. This is ideal for instances that require constant uptime and can tolerate a short period of decreased performance.

Ref: https://cloud.google.com/compute/docs/instances/setting-instance-scheduling-options#live_migrate

This command correctly deletes the deployment. Pods are managed by kubernetes workloads (deployments). When a pod is deleted, the deployment detects the pod is unavailable and brings up another pod to maintain the replica count. The only way to delete the workload is by deleting the deployment itself using the kubectl delete deployment command.

$ kubectldeletedeployment nginx

deployment.apps nginx deleted

Ref: https://kubernetes.io/docs/reference/kubectl/cheatsheet/#deleting-resources

The best option is to instruct the external consultant to generate an SSH key pair, and request the public key from the consultant. Then, add the public key to the instance yourself, and have the consultant access the instance through SSH with their private key. This way, you can grant the consultant access to the instance without requiring a Google account or exposing the instance’s public IP address. This option also follows the best practice of using user-managed SSH keys instead of service account keys for SSH access1.

Option A is not feasible because the external consultant does not have a Google account, and therefore cannot use Identity-Aware Proxy (IAP) to access the instance. IAP requires the user to authenticate with a Google account and have the appropriate IAM permissions to access the instance2. Option B is not secure because it exposes the instance’s public IP address, which can increase the risk of unauthorized access or attacks. Option D is not correct because it reverses the roles of the public and private keys. The public key should be added to the instance, and the private key should be kept by the consultant. Sharing the private key with anyone else can compromise the security of the SSH connection3.

1: https://cloud.google.com/compute/docs/instances/adding-removing-ssh-keys

2: https://cloud.google.com/iap/docs/using-tcp-forwarding

3: https://cloud.google.com/compute/docs/instances/connecting-advanced#sshbetweeninstances

https://cloud.google.com/run/docs/tutorials/pubsub#integrating-pubsub

1. Create a service account. 2. Give the Cloud Run Invoker role to that service account for your Cloud Run application. 3. Create a Cloud Pub/Sub subscription that uses that service account and uses your Cloud Run application as the push endpoint.

Instance groups are collections of virtual machine (VM) instances that you can manage as a single entity. Instance groups can help you simplify the management of multiple instances, reduce operational costs, and improve the availability and performance of your applications. Instance groups support autoscaling, which automatically adds or removes instances from the group based on increases or decreases in load. Autoscaling helps your applications gracefully handle increases in traffic and reduces cost when the need for resources is lower. You can set the autoscaling policy based on CPU utilization, load balancing capacity, Cloud Monitoring metrics, or a queue-based workload. In this case, since the video encoding software is CPU-intensive, setting the autoscaling based on CPU utilization is the best option to ensure high availability and optimal performance. References:

Instance groups

Autoscaling groups of instances

Cloud SQL Auth Proxy: This proxy ensures secure connections to your Cloud SQL database by automatically handling encryption (SSL/TLS) and IAM-based authentication. It simplifies the management of secure connections without needing to manage SSL/TLS certificates manually. IAM Database Authentication: This allows you to use IAM credentials to authenticate to the database, providing a unified and secure authentication mechanism that integrates seamlessly with Google Cloud IAM.

https://cloud.google.com/vpc/docs/vpc#manually_created_subnet_ip_ranges

`gcloud services list --available` returns not only the enabled services in the project but also services that CAN be enabled.

https://cloud.google.com/sdk/gcloud/reference/services/list#--available

Run the following command to list the enabled APIs and services in your current project:

gcloud services list

whereas, Run the following command to list the APIs and services available to you in your current project:

gcloud services list –available

https://cloud.google.com/sdk/gcloud/reference/services/list#--available

--available

Return the services available to the project to enable. This list will include any services that the project has already enabled.

To list the services the current project has enabled for consumption, run:

gcloud services list --enabled

To list the services the current project can enable for consumption, run:

gcloud services list –available

Understanding IAM custom roles

Key Point: Custom roles enable you to enforce the principle of least privilege, ensuring that the user and service accounts in your organization have only the permissions essential to performing their intended functions.

Basic concepts

Custom roles are user-defined, and allow you to bundle one or more supported permissions to meet your specific needs. Custom roles are not maintained by Google; when new permissions, features, or services are added to Google Cloud, your custom roles will not be updated automatically.

When you create a custom role, you must choose an organization or project to create it in. You can then grant the custom role on the organization or project, as well as any resources within that organization or project.

https://cloud.google.com/iam/docs/understanding-custom-roles#basic_concepts

https://kubernetes.io/docs/tasks/debug-application-cluster/debug-application/#debugging-pods

Installing Jenkins

In this section, you use Cloud Marketplace to provision a Jenkins instance. You customize this instance to use the agent image you created in the previous section.

Go to the Cloud Marketplace solution for Jenkins.

Click Launch on Compute Engine.

Change the Machine Type field to 4 vCPUs 15 GB Memory, n1-standard-4.

Machine type selection for Jenkins deployment.

Click Deploy and wait for your Jenkins instance to finish being provisioned. When it is finished, you will see:

Jenkins has been deployed.

https://cloud.google.com/solutions/using-jenkins-for-distributed-builds-on-compute-engine#installing_jenkins

To minimize latency for global users and support growth across multiple geographical regions, you must deploy your resources in those regions and use a global load balancing service.

Action B (Multi-Region Deployment): Deploying VMs in multiple regions closest to your users is necessary to reduce latency (by shortening the physical distance data must travel) and support regional growth.

Action C (Global External Application Load Balancer): The external Application Load Balancer (Global) is the Google-recommended Layer 7 solution for applications that require global reach. It provides a single anycast IP address and directs users to the nearest healthy backend, thus minimizing latency globally.

https://cloud.google.com/bigquery/docs/access-control#custom_roles

Custom roles enable you to enforce the principle of least privilege, ensuring that the user and service accounts in your organization have only the permissions essential to performing their intended functions.

https://cloud.google.com/spanner/docs/iam#spanner.databaseUser

Using the gcloud tool, execute the gcloud iam roles describe roles/spanner.databaseUser command on Cloud Shell. Attach the users to a newly created Google group and add the group to the role.

The organization policy explicitly prevents the use of service account keys, so options A and B, which involve requesting exceptions to create them, are not in line with the policy and Google's recommended practices for secure authentication.

https://cloud.google.com/spanner/docs/getting-started/set-up

The Vertical Pod Autoscaler (VPA) in Kubernetes automatically adjusts the CPU and memory requests and limits of the containers within a pod based on historical and real-time resource usage. In this scenario, where a single-replica stateful application needs more CPU during peak times, VPA can dynamically increase the CPU allocated to the pod when needed and potentially decrease it during off-peak periods to optimize resource utilization and cost efficiency.

Option A: Cluster autoscaling adds or removes nodes in your GKE cluster based on the resource requests of your pods. While it can help with overall cluster capacity, it oesn't directly address the need for more CPU for a specific pod.

Option C: Horizontal Pod Autoscaler (HPA) scales the number of pod replicas based on observed CPU utilization or other select metrics. Since the application can only have one replica, HPA is not suitable.

Option D: Node auto-provisioning is similar to cluster autoscaling, automatically creating and deleting node pools based on workload demands. It doesn't directly manage the resources of individual pods.

Reference to Google Cloud Certified - Associate Cloud Engineer Documents:

The functionality and use cases of the Vertical Pod Autoscaler (VPA) are detailed in the Google Kubernetes Engine documentation, specifically within the resource management and autoscaling sections. Understanding how VPA can dynamically adjust pod resources is relevant to the Associate Cloud Engineer certification.

https://cloud.google.com/kubernetes-engine/docs/concepts/daemonset

https://cloud.google.com/kubernetes-engine/docs/concepts/daemonset#usage_patterns

DaemonSets attempt to adhere to a one-Pod-per-node model, either across the entire cluster or a subset of nodes. As you add nodes to a node pool, DaemonSets automatically add Pods to the new nodes as needed.

In GKE, DaemonSets manage groups of replicated Pods and adhere to a one-Pod-per-node model, either across the entire cluster or a subset of nodes. As you add nodes to a node pool, DaemonSets automatically add Pods to the new nodes as needed. So, this is a perfect fit for our monitoring pod.

Ref: https://cloud.google.com/kubernetes-engine/docs/concepts/daemonset

DaemonSets are useful for deploying ongoing background tasks that you need to run on all or certain nodes, and which do not require user intervention. Examples of such tasks include storage daemons like ceph, log collection daemons like fluentd, and node monitoring daemons like collectd. For example, you could have DaemonSets for each type of daemon run on all of your nodes. Alternatively, you could run multiple DaemonSets for a single type of daemon, but have them use different configurations for different hardware types and resource needs.

https://cloud.google.com/run/docs/tutorials/gcloud

https://cloud.google.com/resource-manager/docs/creating-managing-projects

https://cloud.google.com/iam/docs/understanding-roles#primitive_roles

You can shut down projects using the Cloud Console. When you shut down a project, this immediately happens: All billing and traffic serving stops, You lose access to the project, The owners of the project will be notified and can stop the deletion within 30 days, The project will be scheduled to be deleted after 30 days. However, some resources may be deleted much earlier.

OS Login is a Google-recommended practice for managing access to Linux VMs in Compute Engine. It centralizes user account management by linking the Linux user accounts on the VMs to Google Cloud identities. You then use IAM roles to grant users the necessary permissions to access the VMs (e.g., roles/compute.osLogin or roles/compute.osAdminLogin). This simplifies management as you control access through IAM policies rather than managing individual SSH keys on each VM, thus minimizing operational costs.

Option B: While enabling OS Login is a good first step, writing custom startup scripts to manage user permissions adds complexity and operational overhead, contradicting the goal of simplification and minimizing costs.

Option C: Requiring developers to manage their own SSH keys and making the owner root is a significant security risk and not a recommended practice. It also doesn't centralize management.

Option D: This approach also involves managing individual SSH keys and custom scripts, which increases operational overhead and doesn't leverage the centralized management benefits of OS Login.

Reference to Google Cloud Certified - Associate Cloud Engineer Documents:

OS Login and its benefits for simplified and secure Linux VM access management are detailed in the Compute Engine documentation, which is a key area for the Associate Cloud Engineer certification. The integration with IAM for permission control is a central aspect of this service.

IAP controls access to your App Engine apps and Compute Engine VMs running on Google Cloud. It leverages user identity and the context of a request to determine if a user should be allowed access. IAP is a building block toward BeyondCorp, an enterprise security model that enables employees to work from untrusted networks without using a VPN.

By default, IAP uses Google identities and IAM. By leveraging Identity Platform instead, you can authenticate users with a wide range of external identity providers, such as:

Email/password

OAuth (Google, Facebook, Twitter, GitHub, Microsoft, etc.)

SAML

OIDC

Phone number

Custom

Anonymous

This is useful if your application is already using an external authentication system, and migrating your users to Google accounts is impractical.

https://cloud.google.com/iap/docs/using-tcp-forwarding#grant-permission

The resource that you need to get access is in the other project.

roles/bigquery.dataViewer BigQuery Data Viewer

When applied to a table or view, this role provides permissions to:

Read data and metadata from the table or view.

This role cannot be applied to individual models or routines.

When applied to a dataset, this role provides permissions to:

Read the dataset's metadata and list tables in the dataset.

Read data and metadata from the dataset's tables.

When applied at the project or organization level, this role can also enumerate all datasets in the project. Additional roles, however, are necessary to allow the running of jobs.

https://gtseres.medium.com/using-service-accounts-across-projects-in-gcp-cf9473fef8f0

You create the service account in proj-sa and take note of the service account email, then you go to proj-vm in IAM > ADD and add the service account's email as new member and give it the Compute Storage Admin role.

https://cloud.google.com/compute/docs/access/iam#compute.storageAdmin

To set a default cluster for gcloud commands, run the following command: gcloud config set container/cluster CLUSTER_NAMEhttps://cloud.google.com/kubernetes-engine/docs/how-to/managing-clusters?hl=en

The requirement is for a stable set of egress IP addresses from a GKE cluster for allowlisting by a third party, following best practices.

Option A is not recommended: Using a single node lacks scalability and high availability. Relying on a single node's static IP creates a single point of failure and doesn't align with GKE's design principles. Disabling autoscaling hinders elasticity.

Option C is complex and unreliable: Public nodes typically have ephemeral external IPs (unless manually configured per node, which is difficult to manage with autoscaling). Dynamically tracking and emailing IPs daily is operationally burdensome and prone to race conditions where the allowlist might lag behind IP changes.

Option D uses Cloud NAT but with dynamic IPs. Dynamic IPs change over time, making them unsuitable for stable firewall allowlists.

Option B is the Google-recommended practice: Configuring the GKE cluster with private nodes enhances security as nodes don't have direct external IPs. Cloud NAT provides managed network address translation for these private nodes to access the internet. By configuring Cloud NAT with a static allocation of external IP addresses, all egress traffic from the private GKE nodes will appear to originate from this stable, predictable set of IPs. This set can be given to the vendor for allowlisting without worrying about node IP changes due to scaling or maintenance.

This approach decouples the application's egress IP from the individual nodes, providing stability and adhering to the principle of least privilege (private nodes).

https://cloud.google.com/logging/docs/audit

Data Access audit logs Data Access audit logs contain API calls that read the configuration or metadata of resources, as well as user-driven API calls that create, modify, or read user-provided resource data.

https://cloud.google.com/logging/docs/audit#data-access

https://cloud.google.com/sql/docs/postgres

When encountering a quotaExceeded error in BigQuery, you should follow these steps to diagnose and mitigate the issue:

Understand the Error:

The error message indicates that a quota was exceeded (either a short-term rate limit or a longer-term limit).

The response payload contains information about which quota was reached.

Quotas can fall into two categories:

rateLimitExceeded: Short-term limits. Retry the operation after a few seconds using exponential backoff.

quotaExceeded: Longer-term limits. Wait 10 minutes or longer before retrying the operation.

Search Errors in Cloud Audit Logs (Option A):

Cloud Audit Logs provide detailed information about API requests and responses.

By searching the logs, you can identify the specific API call that triggered the quotaExceeded error.

This helps you understand which resource or operation exceeded the quota.

View Errors in Cloud Monitoring (Option C):

Cloud Monitoring (formerly known as Stackdriver) provides insights into your Google Cloud resources.

Check the monitoring dashboard for any alerts related to BigQuery quotas.

You can set up custom monitoring rules to track specific quotas and receive notifications.

Other Options:

B. Configure Cloud Trace: Cloud Trace is used for performance analysis and latency tracking. It’s not directly related to quota issues.

D. Use Information Schema Views: Information schema views provide metadata about your datasets and tables but won’t help diagnose quota errors.

E. Use BigQuery Bl Engine: There is no such tool called “BigQuery Bl Engine.” This option is invalid.

Remember that some quotas replenish incrementally over a 24-hour period, so you don’t always need to wait a full 24 hours after reaching the limit. Ifyou consistently hit longer-term quotas, consider workload optimization or requesting a quota increase

https://cloud.google.com/config-connector/docs/how-to/secrets#gcloud

https://cloud.google.com/sdk/gcloud

https://cloud.google.com/sdk/docs/configurations#multiple_configurations

https://cloud.google.com/sdk/gcloud/reference/projects/create

Quickstart: Creating a New Instance Using the Command Line

Before you begin

1. In the Cloud Console, on the project selector page, select or create a Cloud project.

2. Make sure that billing is enabled for your Google Cloud project. Learn how to confirm billing is enabled for your project.

To use the gcloud command-line tool for this quickstart, you must first install and initialize the Cloud SDK:

1. Download and install the Cloud SDK using the instructions given on Installing Google Cloud SDK.

2. Initialize the SDK using the instructions given on Initializing Cloud SDK.

To use gcloud in Cloud Shell for this quickstart, first activate Cloud Shell using the instructions given on Starting Cloud Shell.

https://cloud.google.com/ai-platform/deep-learning-vm/docs/quickstart-cli#before-you-begin

Authorizing with a service account

gcloud auth activate-service-account authorizes access using a service account. As with gcloud init and gcloud auth login, this command saves the service account credentials to the local system on successful completion and sets the specified account as the active account in your Cloud SDK configuration.

https://cloud.google.com/sdk/docs/authorizing#authorizing_with_a_service_account

Change your default zone and region in the metadata server Note: This only applies to the default configuration. You can change the default zone and region in your metadata server by making a request to the metadata server. For example: gcloud compute project-info add-metadata \ --metadata google-compute-default-region=europe-west1,google-compute-default-zone=europe-west1-b The gcloud command-line tool only picks up on new default zone and region changes after you rerun the gcloud init command. After updating your default metadata, run gcloud init to reinitialize your default configuration.https://cloud.google.com/compute/docs/gcloud-compute#change_your_default_zone_and_region_in_the_metadata_server

Shared VPC allows an organization to connect resources from multiple projects to a common Virtual Private Cloud (VPC) network, so that they can communicate with each other securely and efficiently using internal IPs from that network. When you use Shared VPC, you designate a project as a host project and attach one or more other service projects to it. The VPC networks in the host project are called Shared VPC networks. Eligible resources from service projects can use subnets in the Shared VPC network

https://cloud.google.com/vpc/docs/shared-vpc

"For example, an existing instance in a service project cannot be reconfigured to use a Shared VPC network, but a new instance can be created to use available subnets in a Shared VPC network."

The key to understand the requirement is : "The objects are written once and accessed frequently for 30 days"

Standard Storage

Standard Storage is best for data that is frequently accessed ("hot" data) and/or stored for only brief periods of time.

Archive Storage

Archive Storage is the lowest-cost, highly durable storage service for data archiving, online backup, and disaster recovery. Unlike the "coldest" storage services offered by other Cloud providers, your data is available within milliseconds, not hours or days. Archive Storage is the best choice for data that you plan to access less than once a year.

https://cloud.google.com/storage/docs/storage-classes#standard

(A) Object Lifecycle Management

Delete

The Delete action deletes an object when the object meets all conditions specified in the lifecycle rule.

Exception: In buckets with Object Versioning enabled, deleting the live version of an object causes it to become a noncurrent version, while deleting a noncurrent version deletes that version permanently.

https://cloud.google.com/storage/docs/lifecycle#delete

(B) Signed URLs

This page provides an overview of signed URLs, which you use to give time-limited resource access to anyone in possession of the URL, regardless of whether they have a Google account

https://cloud.google.com/storage/docs/access-control/signed-urls

https://cloud.google.com/vpc/docs/vpc-peering

As to mexblood1's point, CPU utilization is a recommended proxy for traffic when it comes to Cloud Spanner. See: Alerts for high CPU utilization The following table specifies our recommendations for maximum CPU usage for both single-region and multi-region instances. These numbers are to ensure that your instance has enough compute capacity to continue to serve your traffic in the event of the loss of an entire zone (for single-region instances) or an entire region (for multi-region instances). -https://cloud.google.com/spanner/docs/cpu-utilization

as the instances (normal or preemptible) would be terminated and relaunched if the health check fails either due to application not configured properly or the instances firewall do not allow health check to happen.

GCP provides health check systems that connect to virtual machine (VM) instances on a configurable, periodic basis. Each connection attempt is called a probe. GCP records the success or failure of each probe.

Health checks and load balancers work together. Based on a configurable number of sequential successful or failed probes, GCP computes an overall health state for each VM in the load balancer. VMs that respond successfully for the configured number of times are considered healthy. VMs that fail to respond successfully for a separate number of times are unhealthy.

GCP uses the overall health state of each VM to determine its eligibility for receiving new requests. In addition to being able to configure probe frequency and health state thresholds, you can configure the criteria that define a successful probe.

Comprehensive and Detailed In Depth Explanation:

The goal is to proactively identify stuck Dataflow jobs with minimal management effort. Let's analyze each option:

A. Error Reporting for slowdowns: Error Reporting primarily focuses on capturing and aggregating exceptions and errors (stack traces). While a stuck job might eventually throw an error, it might also just become unresponsive without generating explicit errors. Relying solely on Error Reporting might not provide timely detection of stuck jobs. Identifying stack traces that indicate slowdowns can also be complex and require significant manual configuration and analysis.

B. Personalized Service Health dashboard: The Personalized Service Health dashboard provides information about Google Cloud service incidents that might be affecting your resources. While it can alert you to broader Dataflow service outages, it won't specifically identify individual stuck jobs due to application-level errors or logic within your Dataflow pipeline.

C. Pub/Sub messages for delays, backup job, and Cloud Tasks alerts: This approach involves significant custom implementation and management. You would need to instrument your Dataflow jobs to detect delays, send messages to Pub/Sub, manage a backup job, and configure Cloud Tasks for alerting. This adds considerable operational overhead and complexity.

D. Cloud Monitoring alerts on data freshness metric: Dataflow provides built-in metrics, including "data freshness" (or similar metrics like "system lag" or "processing time"), which indicate how far behind the pipeline is in processing data. If a job gets stuck, the data freshness will deteriorate beyond an acceptable threshold. Cloud Monitoring allows you to easily set up alerts based on these built-in metrics. This requires minimal custom coding and leverages the platform's existing monitoring capabilities, aligning with the "minimal management effort" requirement.

Therefore, setting up Cloud Monitoring alerts on relevant Dataflow metrics like data freshness is the most efficient and recommended way to detect stuck Dataflow jobs with minimal management overhead.

Google Cloud Documentation References:

Monitoring Dataflow Pipelines: https://cloud.google.com/dataflow/docs/guides/monitoring-your-pipeline - This document details the various metrics available for monitoring Dataflow jobs in Cloud Monitoring, including metrics related to processing time, system lag, and data freshness.

Creating Alerts in Cloud Monitoring: https://cloud.google.com/monitoring/alerts/create - Explains how to set up alerts based on metrics collected by Cloud Monitoring.

Dataflow Metrics: https://cloud.google.com/dataflow/docs/reference/monitoring-metrics - Provides a comprehensive list of Dataflow metrics that can be used for monitoring and alerting.

https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-autoscaler#adding_a_node_pool_with_autoscaling

https://cloud.google.com/storage/docs/object-versioning

https://cloud.google.com/appengine/docs/standard/python/splitting-traffic#gcloud

https://cloud.google.com/compute/docs/instance-groups

https://cloud.google.com/load-balancing/docs/network/transition-to-backend-services#console

In order to enable auto-healing, you need to group the instances into a managed instance group. Managed instance groups (MIGs) maintain the high availability of your applications by proactively keeping your virtual machine (VM) instances available. An auto-healing policy on the MIG relies on an application-based health check to verify that an application is responding as expected. If the auto-healer determines that an application isnt responding, the managed instance group automatically recreates that instance.

It is important to use separate health checks for load balancing and for auto-healing. Health checks for load balancing can and should be more aggressive because these health checks determine whether an instance receives user traffic. You want to catch non-responsive instances quickly, so you can redirect traffic if necessary. In contrast, health checking for auto-healing causes Compute Engine to proactively replace failing instances, so this health check should be more conservative than a load balancing health check.

gcloud compute ssh ensures that the user’s public SSH key is present in the project’s metadata. If the user does not have a public SSH key, one is generated using ssh-keygen and added to the project’s metadata. This is similar to the other option where we copy the key explicitly to the project’s metadata but here it is done automatically for us. There are also security benefits with this approach. When we use gcloud compute ssh to connect to Linux instances, we are adding a layer of security by storing your host keys as guest attributes. Storing SSH host keys as guest attributes improve the security of your connections by helping to protect against vulnerabilities such as man-in-the-middle (MITM) attacks. On the initial boot of a VM instance, if guest attributes are enabled, Compute Engine stores your generated host keys as guest attributes.

Compute Engine then uses these host keys that were stored during the initial boot to verify all subsequent connections to the VM instance.

Ref: https://cloud.google.com/compute/docs/instances/connecting-to-instance

Ref: https://cloud.google.com/sdk/gcloud/reference/compute/ssh

Pub/Sub is a scalable and reliable messaging service that can handle large volumes of data from different sources at different rates. It allows you to decouple the producers and consumers of the data, and provides a durable and persistent storage for the messages until they are delivered. Cloud Functions is a serverless platform that can execute code in response to events, such as messages published to a Pub/Sub topic. It can scale automatically based on the load, and you only pay for the resources you use. By using Pub/Sub and Cloud Functions, you can optimize the storage and processing of the votes, as you can handle the variable rates of incoming votes, process them in real time or near real time, and avoid managing servers or VMs. References:

Pub/Sub documentation

Cloud Functions documentation

Choosing a messaging service for Google Cloud

Best practices for persistent disk snapshots

You can create persistent disk snapshots at any time, but you can create snapshots more quickly and with greater reliability if you use the following best practices.

Creating frequent snapshots efficiently

Use snapshots to manage your data efficiently.

Create a snapshot of your data on a regular schedule to minimize data loss due to unexpected failure.

Improve performance by eliminating excessive snapshot downloads and by creating an image and reusing it.

Set your snapshot schedule to off-peak hours to reduce snapshot time.

Snapshot frequency limits

Creating snapshots from persistent disks

You can snapshot your disks at most once every 10 minutes. If you want to issue a burst of requests to snapshot your disks, you can issue at most 6 requests in 60 minutes.

If the limit is exceeded, the operation fails and returns the following error:

https://cloud.google.com/compute/docs/disks/snapshot-best-practices

Granting more permissions than necessary violates the principle of least privilege, a fundamental security best practice. While option A grants the necessary permissions (as subsets exist in two predefined roles), it might also grant more permissions than the Operations team strictly requires for their tasks on GKE and Cloud SQL. Option D is too broad; 'Admin' roles grant extensive permissions that likely exceed the specific needs.

Google Cloud's best practices strongly recommend adhering to the principle of least privilege. Creating a custom role allows you to precisely define the set of permissions the Operations team needs for their specific tasks on the GKE cluster and the Cloud SQL instance, without granting any unnecessary permissions. This minimizes the potential blast radius in case of accidental or malicious actions.

Google Cloud Documentation References:

IAM best practices: https://cloud.google.com/iam/docs/best-practices - This document explicitly recommends granting the minimum necessary permissions.

Creating and managing custom roles: https://cloud.google.com/iam/docs/creating-managing-custom-roles - This explains how to create roles tailored to specific job functions.

Understanding roles: https://cloud.google.com/iam/docs/understanding-roles - This outlines the concepts of predefined and custom roles and their use cases.

===========

The requirements specify a relational database that provides global scale and is a managed service.

Cloud Spanner is Google Cloud's globally distributed, strong-consistency database service. It provides limitless horizontal scaling while maintaining the structure of a relational database, making it the best fit for an application with a rapidly growing, global user base. As a fully managed service, it inherently minimizes management and administration efforts.

Cloud SQL (Option A) is a regional managed relational database and does not offer the same unlimited, global horizontal scaling capability as Spanner.

You can use the Google Cloud Pricing Calculator to total the estimated monthly costs for each GCP product. You dont incur any charges for doing so.

Ref: https://cloud.google.com/products/calculator

Keyword, Version, traffic splitting, App Engine supports traffic splitting for versions before releasing.

You write a lifecycle management rule in XML and push it to the bucket with gsutil lifecycle set config-xml-file. is not right.

gsutil lifecycle set enables you to set the lifecycle configuration on one or more buckets based on the configuration file provided. However, XML is not a valid supported type for the configuration file.

Ref: https://cloud.google.com/storage/docs/gsutil/commands/lifecycle

Write a script that runs gsutil ls -lr gs://myapp-gcp-ace-logs/ to find and remove items older than 90 days. Repeat this process every morning. is not right.

This manual approach is error-prone, time-consuming and expensive. GCP Cloud Storage provides lifecycle management rules that let you achieve this with minimal effort.

Write a script that runs gsutil ls -l gs://myapp-gcp-ace-logs/ to find and remove items older than 90 days. Schedule the script with cron. is not right.

This manual approach is error-prone, time-consuming and expensive. GCP Cloud Storage provides lifecycle management rules that let you achieve this with minimal effort.

Write a lifecycle management rule in JSON and push it to the bucket with gsutil lifecycle set config-json-file. is the right answer.

You can assign a lifecycle management configuration to a bucket. The configuration contains a set of rules which apply to current and future objects in the bucket. When an object meets the criteria of one of the rules, Cloud Storage automatically performs a specified action on the object. One of the supported actions is to Delete objects. You can set up a lifecycle management to delete objects older than 90 days. gsutil lifecycle set enables you to set the lifecycle configuration on the bucket based on the configuration file. JSON is the only supported type for the configuration file. The config-json-file specified on the command line should be a path to a local file containing the lifecycle configuration JSON document.

Ref: https://cloud.google.com/storage/docs/gsutil/commands/lifecycle

Ref: https://cloud.google.com/storage/docs/lifecycle

auto-provisioning = Attaches and deletes node pools to cluster based on the requirements. Hence creating a GPU node pool, and auto-scaling would be betterhttps://cloud.google.com/kubernetes-engine/docs/how-to/node-auto-provisioning

Comprehensive and Detailed In Depth Explanation:

The problem states that a subnet is nearing its IP address capacity, and the requirement is to expand it without downtime and with minimal operational cost, following Google-recommended practices.

A. Creating a second VPC with the same subnet IP range and peering: While VPC Network Peering allows communication between VPCs, having overlapping IP ranges is generally not recommended and can lead to routing complexities if not managed carefully. It also adds operational overhead of managing two VPCs. This is not the most straightforward or cost-effective solution for simply expanding IP capacity within the same logical network.

B. Deleting and recreating the subnet: Deleting a subnet that contains active VM instances will cause downtime for those instances, violating a key requirement.

C. Using the Google Cloud CLI tool to expand the primary IP range of your subnet: Google Cloud allows you to expand the primary IP range of an existing subnet after it's created, as long as there are no conflicting subnets in the VPC. This operation does not require deleting the subnet or restarting the existing VMs within it, thus avoiding downtime. It's a direct and cost-effective way to increase the available IP address space within the existing subnet. This is a Google-recommended practice for handling subnet capacity issues.

D. Permitting additional traffic with firewall rules: Firewall rules control network traffic based on IP ranges, protocols, and ports. They do not increase the number of available private IP addresses within the subnet. This option does not address the core issue of IP address exhaustion.

Therefore, expanding the primary IP range of the existing subnet using the Google Cloud CLI is the recommended solution that meets all the requirements: addressing IP capacity, minimizing operational costs, and avoiding downtime.

Google Cloud Documentation References:

Expanding Subnet IP Ranges: https://cloud.google.com/vpc/docs/expand-subnet - This documentation explicitly describes how to expand the IP range of an existing subnet without downtime. It outlines the prerequisites and steps involved using the gcloud CLI or the Google Cloud Console.

VPC Network Overview: https://cloud.google.com/vpc/docs/vpc - Provides context on subnet IP ranges and their management.

===========

Showing Deployment Manager templates to your team will allow you to define the changes you want to implement in your cloud infrastructure. You can use Cloud Source Repositories to store Deployment Manager templates and collaborate with your team. Cloud Source Repositories are fully-featured, scalable, and private Git repositories you can use to store, manage and track changes to your code.

https://cloud.google.com/source-repositories/docs/features