Databricks-Generative-AI-Engineer-Associate Free Practice Test

A Generative AI Engineer has created a RAG application which can help employees retrieve answers from an internal knowledge base, such as Confluence pages or Google Drive. The prototype application is now working with some positive feedback from internal company testers. Now the Generative Al Engineer wants to formally evaluate the system??s performance and understand where to focus their efforts to further improve the system.
How should the Generative AI Engineer evaluate the system?

1. A. Use cosine similarity score to comprehensively evaluate the quality of the final generated answers.
2. B. Curate a dataset that can test the retrieval and generation components of the system separatel
3. C. Use MLflow??s built in evaluation metrics to perform the evaluation on the retrieval and generation components.
4. D. Benchmark multiple LLMs with the same data and pick the best LLM for the job.
5. E. Use an LLM-as-a-judge to evaluate the quality of the final answers generated.

Correct Answer: B

A Generative Al Engineer is tasked with developing a RAG application that will help a small internal group of experts at their company answer specific questions, augmented by an internal knowledge base. They want the best possible quality in the answers, and neither latency nor throughput is a huge concern given that the user group is small and they??re willing to wait for the best answer. The topics are sensitive in nature and the data is highly confidential and so, due to regulatory requirements, none of the information is allowed to be transmitted to third parties.
Which model meets all the Generative Al Engineer??s needs in this situation?

1. A. Dolly 1.5B
2. B. OpenAI GPT-4
3. C. BGE-large
4. D. Llama2-70B

Correct Answer: C

A Generative AI Engineer has a provisioned throughput model serving endpoint as part of a RAG application and would like to monitor the serving endpoint??s incoming requests and outgoing responses. The current approach is to include a micro-service in between the endpoint and the user interface to write logs to a remote server.
Which Databricks feature should they use instead which will perform the same task?

1. A. Vector Search
2. B. Lakeview
3. C. DBSQL
4. D. Inference Tables

Correct Answer: D

Generative AI Engineer at an electronics company just deployed a RAG application for customers to ask questions about products that the company carries. However, they received feedback that the RAG response often returns information about an irrelevant product.
What can the engineer do to improve the relevance of the RAG??s response?

1. A. Assess the quality of the retrieved context
2. B. Implement caching for frequently asked questions
3. C. Use a different LLM to improve the generated response
4. D. Use a different semantic similarity search algorithm

Correct Answer: A

A Generative AI Engineer has been asked to build an LLM-based question-answering application. The application should take into account new documents that are frequently published. The engineer wants to build this application with the least cost and least development effort and have it operate at the lowest cost possible.
Which combination of chaining components and configuration meets these requirements?

1. A. For the application a prompt, a retriever, and an LLM are require
2. B. The retriever output is inserted into the prompt which is given to the LLM to generate answers.
3. C. The LLM needs to be frequently with the new documents in order to provide most up-to- date answers.
4. D. For the question-answering application, prompt engineering and an LLM are required to generate answers.
5. E. For the application a prompt, an agent and a fine-tuned LLM are require
6. F. The agent is used by the LLM to retrieve relevant content that is inserted into the prompt which is given to the LLM to generate answers.

Correct Answer: A