Professional-Data-Engineer Exam Dumps - Google Professional Data Engineer Exam

Go to page:

<< First
Prev
1
2
3
4
5
6
7
8
9
Next
Last >>

Question # 33

You are designing storage for two relational tables that are part of a 10-TB database on Google Cloud. You want to support transactions that scale horizontally. You also want to optimize data for range queries on nonkey columns. What should you do?

Use Cloud SQL for storage. Add secondary indexes to support query patterns.

Use Cloud SQL for storage. Use Cloud Dataflow to transform data to support query patterns.

Use Cloud Spanner for storage. Add secondary indexes to support query patterns.

Use Cloud Spanner for storage. Use Cloud Dataflow to transform data to support query patterns.

Full Access

Question # 34

Your infrastructure includes a set of YouTube channels. You have been tasked with creating a process for sending the YouTube channel data to Google Cloud for analysis. You want to design a solution that allows your world-wide marketing teams to perform ANSI SQL and other types of analysis on up-to-date YouTube channels log data. How should you set up the log data transfer into Google Cloud?

Use Storage Transfer Service to transfer the offsite backup files to a Cloud Storage Multi-Regional storage bucket as a final destination.

Use Storage Transfer Service to transfer the offsite backup files to a Cloud Storage Regional bucket as a final destination.

Use BigQuery Data Transfer Service to transfer the offsite backup files to a Cloud Storage Multi-Regional storage bucket as a final destination.

Use BigQuery Data Transfer Service to transfer the offsite backup files to a Cloud Storage Regionalstorage bucket as a final destination.

Full Access

Question # 35

An external customer provides you with a daily dump of data from their database. The data flows into Google Cloud Storage GCS as comma-separated values (CSV) files. You want to analyze this data in Google BigQuery, but the data could have rows that are formatted incorrectly or corrupted. How should you build this pipeline?

Use federated data sources, and check data in the SQL query.

Enable BigQuery monitoring in Google Stackdriver and create an alert.

Import the data into BigQuery using the gcloud CLI and set max_bad_records to 0.

Run a Google Cloud Dataflow batch pipeline to import the data into BigQuery, and push errors to another dead-letter table for analysis.

Full Access

Question # 36

You have uploaded 5 years of log data to Cloud Storage A user reported that some data points in the log data are outside of their expected ranges, which indicates errors You need to address this issue and be able to run the process again in the future while keeping the original data for compliance reasons. What should you do?

Import the data from Cloud Storage into BigQuery Create a new BigQuery table, and skip the rows with errors.

Create a Compute Engine instance and create a new copy of the data in Cloud Storage Skip the rows with errors

Create a Cloud Dataflow workflow that reads the data from Cloud Storage, checks for values outside the expected range, sets the value to an appropriate default, and writes the updated records to a new dataset inCloud Storage

Full Access

Question # 37

You migrated your on-premises Apache Hadoop Distributed File System (HDFS) data lake to Cloud Storage. The data scientist team needs to process the data by using Apache Spark and SQL. Security policies need to be enforced at the column level. You need a cost-effective solution that can scale into a data mesh. What should you do?

1. Load the data to BigQuery tables.2. Create a taxonomy of policy tags in Data Catalog.3. Add policy tags to columns.4. Process with the Spark-BigQuery connector or BigQuery SQL.

1. Deploy a long-living Dataproc cluster with Apache Hive and Ranger enabled.2. Configure Ranger for column level security.3. Process with Dataproc Spark or Hive SQL.

1. Apply an Identity and Access Management (IAM) policy at the file level in Cloud Storage.2. Define a BigQuery external table for SQL processing.3. Use Dataproc Spark to process the Cloud Storage files.

1. Define a BigLake table.2. Create a taxonomy of policy tags in Data Catalog.3. Add policy tags to columns.4. Process with the Spark-BigQuery connector or BigQuery SQL.

Full Access

Answer:

Explanation:

The key requirements are:

Data on Cloud Storage (migrated from HDFS).

Processing with Spark and SQL.

Column-level security.

Cost-effective and scalable for a data mesh.

Let's analyze the options:

Option A (Load to BigQuery tables, policy tags, Spark-BQ connector/BQ SQL):

Pros: BigQuery native tables offer excellent performance. Policy tags provide robust column-level security managed centrally in Data Catalog. The Spark-BigQuery connector allows Spark to read from/write to BigQuery. BigQuery SQL is powerful. Scales well.

Cons: "Loading" the data into BigQuery means moving it from Cloud Storage into BigQuery's managed storage. This incurs storage costs in BigQuery and an ETL step. While effective, it might not be the most "cost-effective" if the goal is to query data in place on Cloud Storage, especially for very large datasets.

Option B (Long-living Dataproc, Hive, Ranger):

Pros: Provides a Hadoop-like environment with Spark, Hive, and Ranger for column-level security.

Cons: "Long-living Dataproc cluster" is generally not the most cost-effective, as you pay for the cluster even when idle. Managing Hive and Ranger adds operational overhead. While scalable, it requires more infrastructure management than serverless options.

Option C (IAM at file level, BQ external table, Dataproc Spark):

Pros: Using Cloud Storage is cost-effective for storage. BigQuery external tables allow SQL access.

Cons: IAM at the file level in Cloud Storage does not provide column-level security. This option fails to meet a critical requirement.

Option D (Define a BigLake table, policy tags, Spark-BQ connector/BQ SQL):

Pros:BigLake Tables: These tables allow you to query data in open formats (like Parquet, ORC) on Cloud Storage as if it were a native BigQuery table, but without ingesting the data into BigQuery's managed storage. This is highly cost-effective for storage.

Column-Level Security with Policy Tags: BigLake tables integrate with Data Catalog policy tags to enforce fine-grained column-level security on the data residing in Cloud Storage. This is a centralized and robust security model.

Spark and SQL Access: Data scientists can use BigQuery SQL directly on BigLake tables. The Spark-BigQuery connector can also be used to access BigLake tables, enabling Spark processing.

Cost-Effective & Scalable Data Mesh: This approach leverages the cost-effectiveness of Cloud Storage, the serverless querying power and security features of BigQuery/Data Catalog, and provides a clear path to building a data mesh by allowing different domains to manage their data in Cloud Storage while exposing it securely through BigLake.

Cons: Performance for BigLake tables might be slightly different than BigQuery native storage for some workloads, but it's designed for high performance on open formats.

Why D is superior for this scenario:

BigLake tables (Option D) directly address the need to keep data in Cloud Storage (cost-effective for a data lake) while providing strong, centrally managed column-level security via policy tags and enabling both SQL (BigQuery) and Spark (via Spark-BigQuery connector) access. This is more aligned with modern data lakehouse and data mesh architectures than loading everything into native BigQuery storage (Option A) if the data is already in open formats on Cloud Storage, or managing a full Hadoop stack on Dataproc (Option B).

[Reference:, , Google Cloud Documentation: BigLake > Overview. "BigLake lets you unify your data warehouses and data lakes. BigLake tables provide fine-grained access control for tables based on data in Cloud Storage, while preserving access through other Google Cloud services like BigQuery, GoogleSQL, Spark, Trino, and TensorFlow.", Google Cloud Documentation: BigLake > Introduction to BigLake tables. "BigLake tables bring BigQuery features to your data in Cloud Storage. You can query external data with fine-grained security (including row-level and column-level security) without needing to move or duplicate data.", Google Cloud Documentation: Data Catalog > Overview of policy tags. "You can use policy tags to enforce column-level access control for BigQuery tables, including BigLake tables.", Google Cloud Blog: "Announcing BigLake â€“ Unifying data lakes and warehouses" (and similar articles) highlight how BigLake enables querying data in place on Cloud Storage with BigQuery's governance features., , , , , ]

Question # 38

You issue a new batch job to Dataflow. The job starts successfully, processes a few elements, and then suddenly fails and shuts down. You navigate to the Dataflow monitoring interface where you find errors related to a particular DoFn in your pipeline. What is the most likely cause of the errors?

Exceptions in worker code

Job validation

Graph or pipeline construction

Insufficient permissions

Full Access

Question # 39

You have important legal hold documents in a Cloud Storage bucket. You need to ensure that these documents are not deleted or modified. What should you do?

Set a retention policy. Lock the retention policy.

Set a retention policy. Set the default storage class to Archive for long-term digital preservation.

Enable the Object Versioning feature. Add a lifecycle rule.

Enable the Object Versioning feature. Create a copy in a bucket in a different region.

Full Access

Question # 40

You are designing storage for 20 TB of text files as part of deploying a data pipeline on Google Cloud. Your input data is in CSV format. You want to minimize the cost of querying aggregate values for multiple users who will query the data in Cloud Storage with multiple engines. Which storage service and schema design should you use?

Use Cloud Bigtable for storage. Install the HBase shell on a Compute Engine instance to query the Cloud Bigtable data.

Use Cloud Bigtable for storage. Link as permanent tables in BigQuery for query.

Use Cloud Storage for storage. Link as permanent tables in BigQuery for query.

Use Cloud Storage for storage. Link as temporary tables in BigQuery for query.

Full Access

Go to page: