ISTQB AI Testing
Tuotekuvaus
How do you test AI-based systems? What are the challenges for self-learning systems? How can you use AI to improve your testing?
Artificial Intelligence and Machine Learning in particular are used more and more in everyday applications and systems. The ISTQB® AI Testing (CT-AI) certification extends understanding of artificial intelligence and/or deep (machine) learning, most specifically testing AI-based systems and using AI in testing, and gives answers to the above questions.
After the course, you can take the ISTQB certification exam. The multiple choice exam has 40 questions, and in order to pass the exam, you need to score 31 out of 47 possible points. The duration of the exam is one hour; non-native English-speakers are allowed 15 minutes extra time.
Target group:
The Certified Tester AI Testing certification is aimed at anyone involved in testing AI-based systems and/or AI for testing. This includes people in roles such as testers, test analysts, data analysts, test engineers, test consultants, test managers, user acceptance testers, and software developers. This certification is also appropriate for anyone who wants a basic understanding of testing AI-based systems and/or AI for testing, such as project managers, quality managers, software development managers, business analysts, operations team members, IT directors, and management consultants.
Prerequisites:
The participants must hold the ISTQB/ISEB Foundation Certificate in Software Testing. As the course material and the certification exam are in English, the participants are expected to have good command of English language.
Course contents:
Chapter 1: Introduction to AI
Definition of AI and AI Effect
Narrow, General and Super AI
AI-based and Conventional Systems
AI Technologies
AI Development Frameworks
Hardware for AI-Based Systems
AI as a Service (AIaaS)
Pre-Trained Models
Standards, Regulations and AI
Chapter 2: Quality Characteristics for AI-Based Systems
Flexibility and Adaptability
Autonomy
Evolution, Bias and Ethics
Side Effects and Reward Hacking
Transparency, Interpretability and Explainability
Safety and AI
Chapter 3: Machine Learning (ML) – Overview
Forms of ML and ML Workflow
Selecting a Form of ML
Chapter 4: ML – Data
Data Preparation as Part of the ML Workflow
Training, Validation and Test Datasets in the ML Workflow
Dataset Quality Issues
Data Quality and its Effect on the ML Model
Data Labelling for Supervised Learning
Chapter 5: ML Functional Performance Metrics
Confusion Matrix
Additional ML Functional Performance Metrics for Classification, Regression and Clustering
Limitations of ML Functional Performance Metrics
Selecting ML Functional Performance Metrics
Benchmark Suites for ML Performance
Chapter 6: ML – Neural Networks and Testing
Neural Networks
Coverage Measures for Neural Networks
Chapter 7: Testing AI-Based Systems Overview
Specification of AI-Based Systems
Test Levels for AI-Based Systems
Test Data for Testing AI-Based Systems
Testing for Automation Bias in AI-Based Systems
Documenting an AI Component
Testing for Concept Drift
Selecting a Test Approach for an ML System
Chapter 8: Testing AI-Specific Quality Characteristics
Challenges Testing Self-Learning Systems
Testing Autonomous AI-Based Systems
Testing for Algorithmic, Sample and Inappropriate Bias
Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems
Challenges Testing Complex AI-based Systems
Testing the Transparency, Interpretability and Explainability of AI-Based Systems
Test Oracles for AI-Based Systems
Test Objectives and Acceptance Criteria
Chapter 9: Methods and Techniques for the Testing of AI-Based Systems
Adversarial Attacks and Data Poisoning
Pairwise Testing
Back-to-Back Testing
A/B Testing
Metamorphic Testing (MT)
Experience-based testing of AI-based Systems
Selecting Test Techniques for AI-based Systems
Chapter 10: Test Environments for AI-Based Systems
Test Environments for AI-Based Systems
Virtual Test Environments for Testing AI-Based Systems
Chapter 11: Using AI for Testing
AI Technologies for Testing
Using AI to Analyze Reported Defects
Using AI for Test Case Generation
Using AI for the Optimization of Regression Test Suites
Using AI for Defect Prediction
Using AI for Testing User Interfaces