Databricks-Machine-Learning-Professional Databricks Certified Machine Learning Professional Questions and Answers

To log single values like the number of trees in a random forest, you can use the mlflow.log_param function. This function allows you to log a parameter (e.g. model hyperparameter) under the current run. Parameters can be of any type, and can be logged using the following syntax:

Python

mlflow.log_param("num_trees", 100)

AI-generated code. Review and use carefully. More info on FAQ.

MLflow also offers a convenient way to log multiple parameters by indicating all of them using a dictionary1. The other options are incorrect because:

• Option A: mlflow.log_artifact is used to log output files in any format, not single values2.
• Option B: mlflow.log_model is used to log an MLflow Model along with its artifacts, not single values3.
• Option C: mlflow.log_metric is used to log numeric metrics, not single values4.
• Option E: There is a way to store values like this using the mlflow.log_param function1. References: mlflow.log_param, mlflow.log_artifact, mlflow.log_model, mlflow.log_metric

To enable MLflow Autologging for all supported machine learning libraries in a notebook, you can use the mlflow.autolog() function. This function enables autologging for each library by invoking the corresponding autologging function, such as mlflow.sklearn.autolog() or mlflow.spark.autolog(). This way, you do not need to call each autologging function separately. The mlflow.autolog() function also takes care of the compatibility and configuration issues that may arise from different versions of the Databricks Runtime for Machine Learning or workspace-wide settings in the Admin Console. The other options are incorrect because:

• Option A: mlflow.sklearn.autolog() will only enable autologging for scikit-learn, not for other libraries.
• Option B: mlflow.spark.autolog() will only enable autologging for Spark MLlib, not for other libraries.
• Option C: spark.conf.set(“autologging”, True) is not a valid way to enable autologging, as there is no such configuration option in Spark.
• Option D: It is possible to automatically log MLflow runs using the autologging functions provided by MLflow. References: Automatic Logging with MLflow Tracking, MLflow quickstart (Python)

Real-time deployment is a paradigm that can centrally compute predictions for a single record with exceedingly fast results. Real-time deployment involves exposing a model as a REST API endpoint that can receive requests and return predictions in milliseconds. This paradigm is suitable for scenarios where low latency and high throughput are required, such as fraud detection, recommendation systems, or chatbots. The other deployment paradigms are not as fast or as centralized as real-time deployment. Streaming deployment involves processing data streams in micro-batches, which can introduce some delay. Batch deployment involves processing large batches of data at scheduled intervals, which can take hours or days. Edge or on-device deployment involves deploying models to edge devices, such as mobile phones or IoT sensors, which can reduce the network overhead but also limit the computational power and model size. https://docs.databricks.com/en/machine-learning/ml-tutorials.html

Z-Ordering is an optimization technique that can speed up the query by colocating similar records while considering values in multiple columns. Z-Ordering is a way of organizing data in storage based on the values of one or more columns. Z-Ordering maps multidimensional data to one dimension while preserving locality of the data points. This means that rows with similar values for the specified columns are stored close together in the same set of files. This improves the performance of queries that filter on those columns, as they can skip over irrelevant files or data blocks. Z-Ordering also enhances data skipping and caching, as it reduces the number of distinct values per file for the chosen columns1. The other options are incorrect because:

• Option B: Bin-packing is an optimization technique that compacts small files into larger ones, but does not colocate similar records based on multiple columns. Bin-packing can improve the performance of queries by reducing the number of files that need to be read, but it does not affect the data layout within the files2.
• Option C: Writing as a Parquet file is not an optimization technique, but a file format choice. Parquet is a columnar storage format that supports efficient compression and encoding schemes. Parquet can improve the performance of queries by reducing the storage footprint and the amount of data transferred, but it does not colocate similar records based on multiple columns3.
• Option D: Data skipping is an optimization technique that skips over files or data blocks that do not match the query predicates, but does not colocate similar records based on multiple columns. Data skipping can improve the performance of queries by avoiding unnecessary data scans, but it depends on the data layout and the metadata collected for each file4.
• Option E: Tuning the file size is an optimization technique that adjusts the size of the data files to a target value, but does not colocate similar records based on multiple columns. Tuning the file size can improve the performance of queries by balancing the trade-off between parallelism and overhead, but it does not affectthe data layout within the files5. References: Z-Ordering (multi-dimensional clustering), Compaction (bin-packing), Parquet, Data skipping, Tuning file sizes

MLflow provides a utility function mlflow.models.signature.infer_signature that can infer the input schema and the output schema of a model from a sample dataset. The input schema and the output schema are represented as mlflow.types.schema.Schema objects, which define the names, data types, and shapes of the model inputs and outputs. The infer_signature function can be used for any model that implements a predict method that takes a pandas DataFrame or a numpy array as input and returns a pandas DataFrame, a pandas Series, a numpy array, or a list as output. The infer_signature function can also be used for models that implement a transform method that takes a pandas DataFrame as input and returns a pandas DataFrame as output. The infer_signature function can be used before logging the model with MLflow, as it does not require the model to be in MLflow format or registered with MLflow Model Registry. References:

• MLflow Models Documentation, Model Signature and Input Example, p. 6-7
• mlflow.types.schema Documentation, p. 1-2
• Machine Learning Engineering with MLflow, Managing Model Signatures and Schemas, p. 1

A model signature is a specification of the input and output schema of a model, which defines the names, data types, and shapes of the model inputs and outputs. Logging a model signature with an MLflow model has several benefits, such as:

• The signature can be used to document what type of data is expected as input and output by the model, which can help users and developers understand how to use the model correctly.
• The signature can be used to enforce schema validation when serving models via REST API, which can help prevent errors and ensure data quality. For example, Azure Machine Learning enforces compliance with the model signature for MLflow models that contain a signature1.
• The signature can be used to enable automatic conversion of input data to the appropriate format for the model, such as pandas DataFrame, numpy array, or PyTorch tensor. This can simplify the data preprocessing and inference steps and reduce the code complexity.

References:

• MLflow Models Documentation, Model Signature and Input Example, p. 6-7
• mlflow.models.signature Documentation, p. 1-2
• Guidelines for deploying MLflow models - Azure Machine Learning, p. 1

Real-time deployment is a type of model deployment where predictions are computed on-demand at query time. This can be useful when the feature values are not available until the time of delivery, and results are needed exceedingly fast for one record at a time. Real-time deployment can leverage Databricks Model Serving, which allows users to host MLflow models as REST endpoints on Databricks1. The other options are incorrect because they either do not meet the requirements or are not supported by Databricks Machine Learning. Edge/on-device deployment is a type of model deployment where predictions are computed on the edge device, such as a smartphone or a sensor. This can be useful when there is limited or no network connectivity, or when privacy and security are important. However, edge deployment is not supported by Databricks Machine Learning2. Streaming deployment is a type of model deployment where predictions are computed on streaming data, such as sensor readings or web logs. This can be useful when the data volume is high and the results need to be updated continuously. However, streaming deployment is not supported by Databricks Model Serving1. Batch deployment is a type of model deployment where predictions are computed in advance and stored for later use. This can be useful when the feature values are known ahead of time and the predictions do not need to be delivered quickly. However, batch deployment does not meet the requirements of this scenario, as the feature values are not available until the time of delivery, and results are needed exceedingly fast for one record at a time. References: MLflow Model Serving, Model deployment patterns

Categorical feature drift is a change in the distribution of the input data over time for categorical features, which can affect the performance and accuracy of the model. Monitoring categorical feature drift is important to ensure that the model is still valid and reliable for the current data. One simple statistic to monitor for categorical feature drift is the combination of mode, number of unique values, and percentage of missing values for each categorical feature. These statistics can provide a quick overview of the changes in the data distribution, such as the most frequent category, the diversity of categories, and the quality of data. If these statistics deviate significantly from the baseline values, it may indicate a categorical feature drift. However, these statistics may not capture all the nuances of the data distribution, such as the relative frequencies of different categories, the similarity of categories, etc. Therefore, other methods, such as statistical tests or information-theoretic measures, may be needed to complement the simple statistics and provide a more comprehensive analysis of the categorical feature drift123 References:

• Monitoring Feature Drift - Databricks
• Drift Metrics: How to Select the Right Metric to Analyze Drift
• Detect data drift on datasets (preview) - Azure Machine Learning

Concept drift in machine learning and data mining refers to the change in the relationships between input and output data in the underlying problem over time. In other domains, this change maybe called “covariate shift,” “dataset shift,” or “nonstationarity.” Concept drift can affect the performance and accuracy of predictive models that assume a static relationship between input and output variables. Concept drift can be caused by various factors, such as changes in user behavior, environmental conditions, market trends, etc. Concept drift can be detected and handled by various methods, such as periodic retraining, online learning, ensemble methods, etc References:

• Concept drift - Wikipedia
• A Gentle Introduction to Concept Drift in Machine Learning
• Model Drift & Machine Learning: Concept Drift, Feature Drift, Etc.
• Data Drift vs. Concept Drift: What Is the Difference?

The code block that the team is using is correct and does not need any changes to accomplish the task. The update_registered_model method of the MlflowClient class can be used to update the metadata of a registered model, such as its name or description. The method takes the following parameters:

• name: The name of the registered model to update.
• description: The new description for the registered model.
• new_name: The new name for the registered model.

The method returns a RegisteredModel object that represents the updated registered model1

The other options are incorrect because:

• A. Replacing update_registered_model with update_model_version would not update the metadata of the registered model, but rather the metadata of a specific model version. The update_model_version method can be used to update the stage, description, or name of a model version. The method takes the following parameters:

The method returns a ModelVersion object that represents the updated model version2

• C. Replacing description with artifact would not update the description of the registered model, but rather raise an error, as artifact is not a valid parameter for the update_registered_model method. The artifact parameter is used for the log_model or save_model methods, which can be used to log or save a model with a specific flavor and artifact path34
• D. Replacing client.update_registered_model with mlflow would not update the registered model, but rather raise an error, as mlflow is not a valid method, but rather a module that contains various submodules and functions. The mlflow module does not have an update_registered_model function, but rather the MlflowClient class has an update_registered_model method5
• E. Adding a Python model as an argument to update_registered_model would not update the registered model, but rather raise an error, as a Python model is not a valid argument for the update_registered_model method. The update_registered_model method does not take a model as an argument, but rather the name and description of the registered model to update. To add a Python model to the Model Registry, the create_model_version method of the MlflowClient class can be used, which takes the following parameters:

The method returns a ModelVersion object that represents the created model version6

References:

• mlflow.tracking.client.MlflowClient.update_registered_model — MLflow 2.9.1 documentation
• mlflow.tracking.client.MlflowClient.update_model_version — MLflow 2.9.1 documentation
• mlflow..log_model — MLflow 2.9.1 documentation
• mlflow..save_model — MLflow 2.9.1 documentation
• mlflow — MLflow 2.9.1 documentation
• mlflow.tracking.client.MlflowClient.create_model_version — MLflow 2.9.1 documentation

Drift is the change over time in the statistical properties of the data that was used to train a machine learning model. This can cause the model to become less accurate or perform differently than it was designed to1. Drift can be detected by monitoring the statistics of the input and output data over time and comparing them with the baseline statistics from the training data2. Depending on the type and severity of the drift, different responses may be appropriate. Some possible responses are:

• Retraining and deploying a model on more recent data: This can help the model adapt to the changes in the data and improve its performance. However, this may require frequent retraining and deployment cycles, which can be costly and time-consuming. Also, retraining may not be sufficient if the drift is caused by a change in the underlying concept or relationship between the input and output variables3.
• Rebuilding the machine learning application with a new label variable: This can help the model capture the new concept or relationship that has emerged in the data. However, this may require a significant redesign of the application and the data pipeline, as well as collecting and labeling new data. Also, rebuilding may not be feasible if the concept or relationship is constantly changing or unknown3.
• Sunsetting the machine learning application: This can help avoid the risks and costs of maintaining a model that is no longer reliable or useful. However, this may mean losing the benefits and value of the application and the data. Also, sunsetting may not be an option if the application is critical or mandatory for the business or the users3.

Therefore, all of these responses are probable, depending on the situation and the trade-offs involved. References:

• Databricks Machine Learning Professional Exam Guide, Section 4: Solution and Data Monitoring, p. 5
• Databricks Machine Learning Documentation, Monitoring ML Models, Data Drift Detection, p. 2-3
• A Gentle Introduction to Concept Drift in Machine Learning, Types of Concept Drift, p. 3-4
• Understanding Data Drift and Model Drift: Drift Detection in Python, Types of Drift, p. 2-3

To move a model version from the Staging stage to the Production stage using the MLflow Client, you can use the client.transition_model_version_stage method. This method takes the name of the registered model, the version of the model, and the target stage as arguments. It transitions the model version to the specified stage and returns the updated model version object. The syntax of the method is as follows:

Python

client.transition_model_version_stage (name, version, stage)

AI-generated code. Review and use carefully. More info on FAQ.

The other options are incorrect because:

• Option A: client.transition_model_version_stage with the stage argument set to “Staging” will not move the model version from the Staging stage to the Production stage, but rather keep it in the same stage.
• Option B: client.transition_model_stage is not a valid method name, as it is missing an underscore. The correct method name is client.transition_model_version_stage.
• Option D: client.transition_model_stage is not a valid method name, as it is missing an underscore. The correct method name is client.transition_model_version_stage. Also, the method does not take from and to arguments, but only a stage argument.
• Option E: client.transition_model_version_stage does not take from and to arguments, but only a stage argument. The from argument is redundant, as the method will automatically transition the model version from its current stage tothe target stage. References: MLflow Model Registry, MlflowClient.transition_model_version_stage

The client.delete_registered_model method of the MlflowClient class can be used to delete a model from the MLflow Model Registry. The method takes the following parameter:

• name: The name of the registered model to delete.

The method deletes the registered model and all its versions from the Model Registry. The method does not return any value1

The other options are incorrect because:

• A. The client.transition_model_version_stage method can be used to transition a model version to a different stage, such as None, Staging, Production, or Archived. The method does not delete the model or the model version from the Model Registry2
• B. The client.delete_model_version method can be used to delete a specific model version from the Model Registry. The method does not delete the registered model or all its versions from the Model Registry3
• C. The client.update_registered_model method can be used to update the metadata of a registered model, such as its name or description. The method does not delete the model or its versions from the Model Registry4
• D. There is no method called client.delete_model. To delete a model from the Model Registry, the client.delete_registered_model method should be used.

References:

• mlflow.tracking.client.MlflowClient.delete_registered_model — MLflow 2.9.1 documentation
• mlflow.tracking.client.MlflowClient.transition_model_version_stage — MLflow 2.9.1 documentation
• mlflow.tracking.client.MlflowClient.delete_model_version — MLflow 2.9.1 documentation
• mlflow.tracking.client.MlflowClient.update_registered_model — MLflow 2.9.1 documentation

Streaming with Spark as a model deployment strategy means applying a machine learning model to data streams that are processed incrementally and continuously by Spark Structured Streaming. Spark Structured Streaming is a scalable and fault-tolerant stream processing engine that enables complex analytics on live data streams using the Dataset/DataFrame API1. Spark Structured Streaming supports various sources and sinks for streaming data, such as Kafka, Kinesis, TCP sockets, Delta tables, etc2. Spark Structured Streaming also supports various types of operations on streaming data, such as aggregations, windowing, joins, and stateful transformations3. To deploy a machine learning model on streaming data, you can use the MLflow model registry to managethe model lifecycle and versioning4. You can also use the MLflow model serving feature to serve the model as a REST API endpoint that can be invoked by Spark Structured Streaming5. Alternatively, you can use the UDF (user-defined function) feature to apply the model to streaming data within Spark Structured Streaming6.

The inference of incrementally processed records as soon as trigger is hit describes the streaming with Spark as a model deployment strategy. A trigger defines when the results of a streaming query should be written to the output sink. A trigger can be based on a processing time interval, a data size limit, or a continuous mode that writes the results as soon as they are available. The trigger ensures that the streaming query is executed incrementally and continuously, and the model inference is applied to the latest available data. The other options are incorrect because:

• Option A: The inference of batch processed records as soon as a trigger is hit does not describe streaming with Spark, but rather batch processing with Spark. Batch processing means applying a machine learning model to a finite set of data that is processed as a single job. Batch processing does not require a trigger, as the results are written to the output sink when the job is completed.
• Option B: The inference of all types of records in real-time does not describe streaming with Spark, but rather a generic definition of real-time processing. Real-time processing means applying a machine learning model to data streams that are processed as soon as they arrive, with minimal latency. Real-time processing does not necessarily use Spark Structured Streaming, as there are other frameworks and tools that can support it, such as Apache Flink, Apache Storm, etc.
• Option C: The inference of batch processed records as soon as a Spark job is run does not describe streaming with Spark, but rather batch processing with Spark. Batch processing means applying a machine learning model to a finite set of data that is processed as a single job. Batch processing does not depend on a Spark job, as the model inference can be done outside of Spark, such as using a REST API endpoint, a command-line tool, etc.
• Option E: The inference of incrementally processed records as soon as a Spark job is run does not describe streaming with Spark, but rather a contradiction. Incrementally processed records imply streaming processing, while a Spark job implies batch processing. Streaming processing and batch processing are different paradigms of data processing, and cannot be mixed in this way. References: Structured Streaming Programming Guide, Input Sources and Output Sinks, Operations on streaming DataFrames/Datasets, MLflow Model Registry, MLflow Model Serving, Apply machine learning models, [Triggers], [Trigger Types], [Batch Processing], [Real-time Processing], [Real-time Data Processing Frameworks], [Deploy machine learning models], [Batch vs Streaming Processing]

To create nested runs in MLflow, you need to specify the nested=True argument in the parent run, and omit it in the child runs. This will set a special tag (mlflow.utils.mlflow_tags.MLFLOW_PARENT_RUN_ID) in the child runs that indicates the parent run ID. This way, the UI will show the child runs under the parent run in a collapsible hierarchy. The indentation of the code blocks is not relevant, as long as the child runs are created within the context of the parent run. The run name parameter is also not relevant, as it does not affect the nesting of the runs. References: Understanding Parent and Child Runs in MLflow, Nested runs using MLflowClient, creating-child-runs-in-mlflow