- (Topic 2)
What is the main purpose of accountability structures under the Govern function of the NIST Al Risk Management Framework?

1. A. To empower and train appropriate cross-functional teams.
2. B. To establish diverse, equitable and inclusive processes.
3. C. To determine responsibility for allocating budgetary resources.
4. D. To enable and encourage participation by external stakeholders.

Correct Answer: A
The NIST AI Risk Management Framework’s Govern function emphasizes the importance of establishing accountability structures that empower and train cross-functional teams. This is crucial because cross-functional teams bring diverse perspectives and expertise, which are essential for effective AI governance and risk management. Training these teams ensures that they are well-equipped to handle their responsibilities and can make informed decisions that align with the organization’s AI principles and ethical standards. Reference: NIST AI Risk Management Framework documentation, Govern function section.

- (Topic 1)
CASE STUDY
Please use the following answer the next question:
Good Values Corporation (GVC) is a U.S. educational services provider that employs teachers to create and deliver enrichment courses for high school students. GVC has learned that many of its teacher employees are using generative Al to create the enrichment courses, and that many of the students are using generative Al to complete their assignments.
In particular, GVC has learned that the teachers they employ used open source large language models (“LLM”) to develop an online tool that customizes study questions for individual students. GVC has also discovered that an art teacher has expressly incorporated the use of generative Al into the curriculum to enable students to use prompts to create digital art.
GVC has started to investigate these practices and develop a process to monitor any use of generative Al, including by teachers and students, going forward.
All of the following may be copyright risks from teachers using generative Al to create course content EXCEPT?

1. A. Content created by an LLM may be protectable under U.
2. B. intellectual property law.
3. C. Generative Al is generally trained using intellectual property owned by third parties.
4. D. Students must expressly consent to this use of generative Al.
5. E. Generative Al often creates content without attribution.

Correct Answer: C
All of the options listed may pose copyright risks when teachers use generative AI to create course content, except for students must expressly consent to this use of generative AI. While obtaining student consent is essential for ethical and privacy reasons, it does not directly relate to copyright risks associated with the creation and use of AI-generated content.
Reference: The AIGP Body of Knowledge discusses the importance of addressing intellectual property (IP) risks when using AI-generated content. Copyright risks are typically associated with the use of third-party data and the lack of attribution, rather than the consent of users.

- (Topic 2)
CASE STUDY
Please use the following answer the next question:
A mid-size US healthcare network has decided to develop an Al solution to detect a type of cancer that is most likely arise in adults. Specifically, the healthcare network intends to create a recognition algorithm that will perform an initial review of all imaging and then route records a radiologist for secondary review pursuant Agreed-upon criteria (e.g., a confidence score below a threshold).
To date, the healthcare network has taken the following steps: defined its Al ethical principles: conducted discovery to identify the intended uses and success criteria for the system: established an Al governance committee; assembled a broad, crossfunctional team with clear roles and responsibilities; and created policies and procedures to document standards, workflows, timelines and risk thresholds during the project.
The healthcare network intends to retain a cloud provider to host the solution and a
consulting firm to help develop the algorithm using the healthcare network's existing data and de-identified data that is licensed from a large US clinical research partner.
Which of the following steps can best mitigate the possibility of discrimination prior to training and testing the Al solution?

1. A. Procure more data from clinical research partners.
2. B. Engage a third party to perform an audit.
3. C. Perform an impact assessment.
4. D. Create a bias bounty program.

Correct Answer: C
Performing an impact assessment is the best step to mitigate the possibility of discrimination before training and testing the AI solution. An impact assessment, such as a Data Protection Impact Assessment (DPIA) or Algorithmic Impact Assessment (AIA), helps identify potential biases and discriminatory outcomes that could arise from the AI system. This process involves evaluating the data and the algorithm for fairness, accountability, and transparency. It ensures that any biases in the data are detected and addressed, thus preventing discriminatory practices and promoting ethical AI deployment. Reference: AIGP Body of Knowledge on Ethical AI and Impact Assessments.

- (Topic 1)
What is the primary purpose of an Al impact assessment?

1. A. To define and evaluate the legal risks associated with developing an Al system.
2. B. Anticipate and manage the potential risks and harms of an Al system.
3. C. To define and document the roles and responsibilities of Al stakeholders.
4. D. To identify and measure the benefits of an Al system.

Correct Answer: B
The primary purpose of an AI impact assessment is to anticipate and manage the potential risks and harms of an AI system. This includes identifying the possible negative outcomes and implementing measures to mitigate these risks. This process helps ensure that AI systems are developed and deployed in a manner that is ethically and socially responsible, addressing concerns such as bias, fairness, transparency, and accountability. The assessment often involves a thorough evaluation of the AI system's design, data inputs, outputs, and the potential impact on various stakeholders. This approach is crucial for maintaining public trust and adherence to regulatory requirements.

- (Topic 2)
You are part of your organization’s ML engineering team and notice that the accuracy of a model that was recently deployed into production is deteriorating.
What is the best first step address this?

1. A. Replace the model with a previous version.
2. B. Conduct champion/challenger testing.
3. C. Perform an audit of the model.
4. D. Run red-teaming exercises.

Correct Answer: B
When the accuracy of a model deteriorates, the best first step is to conduct champion/challenger testing. This involves deploying a new model (challenger) alongside the current model (champion) to compare their performance. This method helps identify if the new model can perform better under current conditions without immediately discarding the existing model. It provides a controlled environment to test improvements and understand the reasons behind the deterioration. This approach is preferable to directly replacing the model, performing audits, or running red-teaming exercises, which may be subsequent steps based on the findings from the champion/challenger testing.
Reference: AIGP BODY OF KNOWLEDGE, sections on model performance management and testing strategies.