A data engineer has been using a Databricks SQL dashboard to monitor the cleanliness of the input data to an ELT job. The ELT job has its Databricks SQL query that returns the number of input records containing unexpected NULL values. The data engineer wants their entire team to be notified via a messaging webhook whenever this value reaches 100.
Which of the following approaches can the data engineer use to notify their entire team via a messaging webhook whenever the number of NULL values reaches 100?

1. A. They can set up an Alert with a custom template.
2. B. They can set up an Alert with a new email alert destination.
3. C. They can set up an Alert with a new webhook alert destination.
4. D. They can set up an Alert with one-time notifications.
5. E. They can set up an Alert without notifications.

Correct Answer: C
To achieve this, the data engineer can set up an Alert in the Databricks workspace that triggers when the query results exceed the threshold of 100 NULL values. They can create a new webhook alert destination in the Alert's configuration settings and provide the necessary messaging webhook URL to receive notifications. When the Alert is triggered, it will send a message to the configured webhook URL, which will then notify the entire team of the issue.

A dataset has been defined using Delta Live Tables and includes an expectations clause:
CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION FAIL UPDATE
What is the expected behavior when a batch of data containing data that violates these constraints is processed?

1. A. Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
2. B. Records that violate the expectation cause the job to fail.
3. C. Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table.
4. D. Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
5. E. Records that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.

Correct Answer: B
https://docs.databricks.com/en/delta-live-tables/expectations.html Action
Result
warn (default)
Invalid records are written to the target; failure is reported as a metric for the dataset. drop
Invalid records are dropped before data is written to the target; failure is reported as a metrics for the dataset.
fail
Invalid records prevent the update from succeeding. Manual intervention is required before re-processing.

A data analyst has developed a query that runs against Delta table. They want help from the data engineering team to implement a series of tests to ensure the data returned by the query is clean. However, the data engineering team uses Python for its tests rather than SQL.
Which of the following operations could the data engineering team use to run the query and operate with the results in PySpark?

1. A. SELECT * FROM sales
2. B. spark.delta.table
3. C. spark.sql
4. D. There is no way to share data between PySpark and SQL.
5. E. spark.table

Correct Answer: C
from pyspark.sql import SparkSession spark = SparkSession.builder.getOrCreate()
df = spark.sql("SELECT * FROM sales") print(df.count())

Which of the following data lakehouse features results in improved data quality over a traditional data lake?

1. A. A data lakehouse provides storage solutions for structured and unstructured data.
2. B. A data lakehouse supports ACID-compliant transactions.
3. C. A data lakehouse allows the use of SQL queries to examine data.
4. D. A data lakehouse stores data in open formats.
5. E. A data lakehouse enables machine learning and artificial Intelligence workloads.

Correct Answer: B
One of the key features of a data lakehouse that results in improved data quality over a traditional data lake is its support for ACID (Atomicity, Consistency, Isolation, Durability) transactions. ACID transactions provide data integrity and consistency guarantees, ensuring that operations on the data are reliable and that data is not left in an inconsistent state due to failures or concurrent access. In a traditional data lake, such transactional guarantees are often lacking, making it challenging to maintain data quality,
especially in scenarios involving multiple data writes, updates, or complex transformations. A data lakehouse, by offering ACID compliance, helps maintain data quality by providing strong consistency and reliability, which is crucial for data pipelines and analytics.

A data engineer needs access to a table new_table, but they do not have the correct permissions. They can ask the table owner for permission, but they do not know who the table owner is.
Which of the following approaches can be used to identify the owner of new_table?

1. A. Review the Permissions tab in the table's page in Data Explorer
2. B. All of these options can be used to identify the owner of the table
3. C. Review the Owner field in the table's page in Data Explorer
4. D. Review the Owner field in the table's page in the cloud storage solution
5. E. There is no way to identify the owner of the table

Correct Answer: C