CT-AI Exam Dumps - Certified Tester AI Testing Exam

Back-to-back testing isused to compare two different versions of an ML model, which is precisely what is needed in this scenario.

The model initiallymisclassified dogs as wolvesdue to feature similarities.

Thetest team retrains the modelwith additional images of dogs and wolves.

The best way to verify whether this additional trainingimproved classification accuracyis to compare theoriginal model's output with the newly trained model's output using the same test dataset​.

A (Metamorphic Testing):Metamorphic testing is useful forgenerating new test casesbased on existing ones but does not directly compare different model versions​.

B (Adversarial Testing):Adversarial testing is used to check how robust a model is againstmaliciously perturbed inputs, not to verify training effectiveness​.

C (Pairwise Testing):Pairwise testing is a combinatorial technique for reducing the number of test casesby focusing on key variable interactions, not for validating model improvements​.

ISTQB CT-AI Syllabus (Section 9.3: Back-to-Back Testing)

"Back-to-back testing is used when an updated ML model needs to be compared against a previous version to confirm that it performs better or as expected"​.

"The results of the newly trained model are compared with those of the prior version to ensure that changes did not negatively impact performance"​.

Why Other Options Are Incorrect:Supporting References from ISTQB Certified Tester AI Testing Study Guide:Conclusion:To verify that the model's performance improved after retraining,back-to-back testing is the most appropriate methodas it compares both model versions. Hence, thecorrect answer is D.

Explainability in AI-based systems refers to the ease with which users can determine how the system reaches a particular result. It is a crucial aspect when demonstrating AI decision-making, as it ensures that decisions made by AI models are transparent, interpretable, and understandable by stakeholders.

Before deploying an AI-based system, a developer must validate how decisions are made in a test environment. This process falls under the characteristic of explainability because it involves clarifying how an AI model arrives at its conclusions, which helps build trust in the system and meet regulatory and ethical requirements.

ISTQB CT-AI Syllabus (Section 2.7: Transparency, Interpretability, and Explainability)

"Explainability is considered to be the ease with which users can determine how the AI-based system comes up with a particular result"​.

"Most users are presented with AI-based systems as ‘black boxes’ and have little awareness of how these systems arrive at their results. This ignorance may even apply to the data scientists who built the systems. Occasionally, users may not even be aware they are interacting with an AI-based system"​.

ISTQB CT-AI Syllabus (Section 8.6: Testing the Transparency, Interpretability, and Explainability of AI-based Systems)

"Testing the explainability of AI-based systems involves verifying whether users can understand and validate AI-generated decisions. This ensures that AI systems remain accountable and do not make incomprehensible or biased decisions"​.

Contrast with Other Options:

Autonomy (B): Autonomy relates to an AI system's ability to operate independently without human oversight​. While decision-making is a key function of autonomy, the focus here is on demonstrating the reasoning behind decisions, which falls under explainability rather than autonomy.

Self-learning (C): Self-learning systems adapt based on previous data and experiences, which is different from making decisions understandable to humans​.

Non-determinism (D): AI-based systems are often probabilistic and non-deterministic, meaning they do not always produce the same output for the same input. This can make testing and validation more challenging, but it does not relate to explaining the decision-making process​.

Supporting References from ISTQB Certified Tester AI Testing Study Guide:Conclusion:Since the question explicitly asks about the characteristic under which decision-making falls when being demonstrated before deployment,explainability is the correct choicebecause it ensures that AI decisions are transparent, understandable, and accountable to stakeholders​.

Recognizing digits from a scan of handwritten numbers using machine learning is an example of classification. Here’s a breakdown:

Classification: This type of machine learning involves categorizing input data into predefined classes. In this scenario, the input data (handwritten digits) are classified into one of the 10 digit classes (0-9).

Why Not Other Options:

Reinforcement Learning: This involves learning by interacting with an environment to achieve a goal, which does not fit the problem of recognizing digits.

Regression: This is used for predicting continuous values, not discrete categories like digit recognition.

Clustering: This involves grouping similar data points together without predefined classes, which is not the case here.

References:The explanation is based on the definitions of different machine learning types as outlined in the ISTQB CT-AI syllabus, specifically under supervised learning and classification​​.

ï‚· Factors Affecting ML Functional Performance:The data pipeline, quality of the labeling, and biased data are all factors that significantly affect the performance of machine learning models. The number of classes, while relevant for the model structure, is not a direct factor affecting the performance metrics such as accuracy or bias.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Sections on Data Quality and its Effect on the ML Model and ML Functional Performance Metrics​​​​.

ï‚· Coverage for Neuron Activation Values:Sign change coverage is used to ensure that test cases cause each neuron to achieve both positive and negative activation values. This type of coverage ensures that the neurons are thoroughly tested under different activation states.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Section 6.2 Coverage Measures for Neural Networks, which details different types of coverage measures, including sign change coverage​​.

Threshold coverageis a specific type of coverage measure used in neural network testing. It ensures that each neuron in the network achieves an activation value greater than a specified threshold. This is particularly relevant to the scenario described, where testers verify that neurons activate only when the future value of the commodity exceeds the spot price by at least0.75%.

Threshold-based activation:The test case in the question isexplicitly verifying whether neurons activate only when a certain threshold (0.75%) is exceeded.This aligns perfectly with the definition ofthreshold coverage.

Common in Neural Network Testing:Threshold coverage is used to measurewhether each neuron in a neural network reaches a specified activation value, ensuring that the neural network behaves as expected when exposed to different test inputs.

Precedent in Research:TheDeepXplore frameworkused a threshold of0.75%to identify incorrect behaviors in neural networks, making this coverage criterion well-documented in AI testing research.

(B) Neuron Coverage❌

Neuron coverageonly checks whether a neuron activates (non-zero value)at some point during testing. It does not consider specific activation thresholds, making it less precise for this scenario.

(C) Sign-Change Coverage❌

This coverage measures whether each neuron exhibitsboth positive and negative activation values, which isnot relevant to the given scenario(where activation only matters when exceeding a specific threshold).

(D) Value-Change Coverage❌

This coverage requires each neuron to producetwo activation values that differ by a chosen threshold, but the question focuses onwhether activation occurs beyond a fixed threshold, not changes in activation values.

Threshold coverage ensures that neurons exceed a given activation threshold"Full threshold coverage requires that each neuron in the neural network achieves an activation value greater than a specified threshold. The researchers who created the DeepXplore framework suggested neuron coverage should be measured based on an activation value exceeding a threshold, changing based on the situation."​

Why is Threshold Coverage Correct?Why Other Options are Incorrect?References from ISTQB Certified Tester AI Testing Study GuideThus,option A is the correct answer, asthreshold coverage ensures the neural network's activation is correctly evaluated based on the required condition (0.75%).

Exploratory Data Analysis (EDA) is an essential technique for examining datasets to uncover patterns, trends, and anomalies, including outliers. In this case, the attacker manipulates the spam filter by injecting emails with red flags and masking them as internal company emails. The primary goal of EDA here is to detect these adversarial modifications.

Detecting Outliers:

EDA techniques such as statistical analysis, clustering, and visualization can reveal patterns in email metadata (e.g., sender details, email content, frequency).

Outlier detection methods like Z-score, IQR (Interquartile Range), or machine learning-based anomaly detection can identify emails that significantly deviate from typical internal communications.

Identifying Distribution Shifts:

By analyzing the frequency and characteristics of emails flagged as spam, testers can detect if the attack has introduced unusual patterns.

If a surge of internal emails is suddenly classified as spam, EDA can help verify whether these classifications are consistent with historical data.

Feature Analysis for Adversarial Patterns:

EDA enables visualization techniques such as scatter plots or histograms to distinguish normal emails from manipulated ones.

Examining email metadata (e.g., changes in headers, unusual wording in email bodies) can reveal adversarial tactics.

Counteracting Adversarial Attacks:

Once anomalies are identified, the spam filter’s detection rules can be improved by retraining the model on corrected datasets.

The adversarial examples can be added to the training data to enhance the robustness of the filter against future attacks.

Exploratory Data Analysis (EDA) is used to detect outliers and adversarial attacks."EDA is where data are examined for patterns, relationships, trends, and outliers. It involves the interactive, hypothesis-driven exploration of data."​

EDA can identify poisoned or manipulated data by detecting anomalies and distribution shifts."Testing to detect data poisoning is possible using EDA, as poisoned data may show up as outliers."​

EDA helps validate ML models and detect potential vulnerabilities."The use of exploratory techniques, primarily driven by data visualization, can help validate the ML algorithm being used, identify changes that result in efficient models, and leverage domain expertise."​

References from ISTQB Certified Tester AI Testing Study GuideThus,option A is the correct answer, as EDA is specifically useful for detecting outliers, which can help identify manipulated spam emails.

Metamorphic Testing (MT)is a testing technique that verifies AI-based systems by generatingfollow-up test casesbased on existing test cases. These follow-up test cases adhere to aMetamorphic Relation (MR), ensuring that if the system is functioning correctly, changes in input should result in predictable changes in output.

Metamorphic testing works by transforming source test cases into follow-up test cases

Here, thesource test caseinvolves testing themedium-speed vehicle’stravel time.

Thefollow-up test casesare derived byextrapolating travel times for fast and slow vehiclesusing predictable relationships based on speed differences.

MR states that modifying input should result in a predictable change in output

Since the speed of the vehicle is a known factor, it is possible to predict the new arrival times and verify whether they follow expected trends.

This is a direct application of metamorphic testing principles

Inroute optimization systems, metamorphic testing often applies transformations tospeed, distance, or conditionsto verify expected outcomes.

(B) Decomposing each route into traffic density and vehicle power❌

While useful for statistical analysis, this approach does not generate follow-up test cases based on a definedmetamorphic relation (MR).

(C) Selecting dissimilar routes and transforming them into a fast or slow route❌

Thisdoes not follow metamorphic testing principles, which require predictable transformations.

(D) Running fast vehicles on long routes and slow vehicles on short routes❌

This methoddoes not maintain a controlled MRand introduces too manyuncontrolled variables.

Metamorphic testing generates follow-up test cases based on a source test case."MT is a technique aimed at generating test cases which are based on a source test case that has passed.One or more follow-up test cases are generated by changing (metamorphizing) the source test case based on a metamorphic relation (MR)."​

MT has been used for testing route optimization AI systems."In the area of AI, MT has been used for testing image recognition, search engines, route optimization and voice recognition, among others."​

Why Option A is Correct?Why Other Options are Incorrect?References from ISTQB Certified Tester AI Testing Study GuideThus,option A is the correct answer, as it aligns with the principles ofmetamorphic testing by modifying input speeds and verifying expected results.