UniversityApply now

Turn Your QA Experience Into AI Evaluation Skills.

Practical AI Testing and Evaluation for QA Professionals

Put your QA skills to work on AI. Get 50 hours of training to evaluate chatbots, RAG applications, and AI agents. Complete a chatbot capstone, build a report that demonstrates your testing judgment, and practice presenting your expertise through mock interviews.

Apply for the January ProgramSee your learning journey

Starts January 9, 2027 · Saturdays at 10:00 AM EST · Live online

Your capstone deliverables16-hour project

Leave with evidence of how you test AI.

  • Evaluation test plan
  • Metrics and test datasets
  • Quality and robustness findings
  • Safety and red-teaming results
  • Final evaluation report

A practical chatbot evaluation you can explain and showcase in interviews.

50 hours
Total training and coursework
9 Saturdays
Live online sessions
1 capstone
Complete chatbot evaluation
EvaliQA
Learn the platform we built
Your QA experience. Your next opportunity.

Built for Your Next Move in AI

You want to bring your testing skills into AI evaluation and build practical evidence of what you can do.

Manual QA

You're a manual QA professional ready to put your testing judgment to work on AI.

Turn your eye for risks, edge cases, and unexpected behavior into a practical approach to evaluating chatbots, RAG applications, and AI agents.

Automation · SDET

You're an automation engineer or SDET ready to evaluate AI behavior.

Expand your toolkit with evaluation datasets, custom metrics, LLM judges, trace analysis, and robustness testing.

QA lead

You lead QA and want a clear approach to assessing AI quality.

Learn to define evaluation scope, select meaningful metrics, calibrate judgments, and explain findings that help your team assess product risks.

New to AI evaluation

You're new to AI evaluation and want guided practice with a tool you can apply.

Build your foundation step by step and learn EvaliQA, the AI testing and evaluation platform we developed, through practical coursework.

Moving into AI QA

You're pursuing an AI QA role and want work you can show hiring teams.

Complete a chatbot evaluation capstone, produce a report you can showcase, and practice presenting your skills through mock interviews with personalized feedback.

Back in the game

You're a QA professional who was laid off and you're ready to get back in the game.

Build on your experience with practical AI evaluation skills, a capstone you can showcase, and interview preparation to help you pursue your next opportunity.

Your journey into AI evaluation

Build skills. Prove your ability.
Take your next career step.

A path from your existing QA experience to practical AI evaluation, career preparation, and an optional project practicum.

  1. 01

    Build practical AI testing and evaluation skills

    Learn to evaluate chatbots, RAG, and AI agents. Choose meaningful metrics, build datasets, analyze results, and assess robustness, bias, and toxicity.

  2. 02

    Demonstrate your skills through a capstone project

    Complete a chatbot evaluation from test planning through reporting. Produce a report that demonstrates your approach, evidence, and testing judgment.

  3. 03

    Strengthen your job search and interview confidence

    Connect your QA background with your AI skills in your résumé and LinkedIn profile. Practice through mock interviews with personalized feedback.

  4. 04Optional add-on · $999

    Build experience through the Eight-Week AI Evaluation Project Practicum

    Extend your learning through practical AI evaluation projects. Apply course methods, deepen your practice, and build examples to discuss with employers.

05Your career goal

Get hired in AI testing and evaluation

Bring your QA experience, evaluation skills, and capstone report into your applications and interviews. Show hiring teams how you identify AI failures, assess quality, and communicate findings. Add practicum project experience if you choose the optional program.

Built by QA professionals. Built for your next step.

The QA Mentor University Advantage

Build the judgment to evaluate AI and the evidence to back it up. Learn practical methods and our own evaluation platform, then put those skills to work on your AI projects.

EvaliQA
Built by QA Mentor

Learn EvaliQA from the team that built it.

We developed EvaliQA, our own AI testing and evaluation platform. Learn to build datasets, configure metrics, run evaluations, and analyze results. Take that practical workflow straight into your own AI projects.

A tool you learn here. Skills you can apply immediately.

How an evaluation runs: a dataset of test cases goes to the chatbot, an LLM judge inspects each answer, a QA engineer reviews it, and a report shows the scores

Know when to trust an evaluation score.

Select metrics, create custom criteria, and calibrate automated judgments against human assessments. Explain why a result deserves confidence.

Follow the evidence inside the AI workflow.

Read traces, interpret evaluation results, and examine non-functional behavior. Connect a failed response with evidence for investigation.

Find the failures easy test cases miss.

Build golden datasets with happy paths, edge cases, negative tests, and robustness scenarios. Make evaluations repeatable and track quality over time.

Adapt your approach to different AI products.

Explore evaluation for chatbots, RAG, agents, harnesses, voice, and images. Understand how each architecture changes your evaluation strategy.

Show employers how you evaluate AI.

Complete a 16-hour chatbot capstone covering planning, datasets, quality, robustness, and red teaming. Present your findings in a final report.

Turn new skills into a stronger interview.

Strengthen your résumé and LinkedIn profile. Explain your approach through mock interviews with personalized feedback and a focused job search strategy.

The January 2027 learning roadmap

Your AI Evaluation Skills,
Week by Week

Build your foundation, practice the evaluation process, and prepare to present your skills to hiring teams.

  • 9 live sessions
  • 50 total hours
  • 16-hour capstone
Start · January 9, 2027Lights out at 10:00 AM EST, live online
  1. Week 1January 9 · 2 hours live

    Introduction to AI Evaluation

    You know what reliable AI evaluation requires.

    January 9, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

    • Understand AI classifications and why AI systems need evaluation.
    • Recognize how probabilistic outputs change traditional testing approaches.
    • Identify evaluation challenges and criteria for reliable results.
  2. Week 2January 16 · 2 hours live

    LLMs, Agents, and AI Architectures

    You understand the AI system you are testing.

    January 16, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

    • Learn how generative models and LLMs are trained and how they work.
    • Understand RAG, single-turn and multi-turn agents, and voice agents.
    • Explore AI harnesses and their role in an AI system.
  3. Week 3January 23 · 2 hours live

    Metrics for AI Evaluation

    You choose metrics and calibrate AI judgments.

    January 23, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

    • Understand deterministic metrics, AI detectors, and LLM-as-a-judge approaches.
    • Choose metrics for RAG, agents, and chatbots, and define custom criteria.
    • Calibrate metrics and align automated judgments with human assessments.
    • Video · 1 hour: Getting started with EvaliQA and DeepEval.
    • Assignment · 1 hour: Prepare metrics for different AI solutions.
  4. Week 4January 30 · 2 hours live

    Test Planning and Dataset Management

    You build a test plan and evaluation datasets.

    January 30, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

    • Define evaluation scope and document your test plan before execution.
    • Build golden datasets, happy paths, edge cases, negative tests, and robustness scenarios.
    • Understand how evaluation-driven development guides iteration.
    • Video · 1 hour: Building a dataset for AI evaluation.
    • Assignment · 2 hours: Prepare happy-path, edge-case, and robustness datasets.
  5. Week 5February 6 · 2 hours live

    Running Evaluations and Analyzing Results

    You run evaluations and investigate failures.

    February 6, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

    • Prepare evaluation inputs and interpret results.
    • Read traces, understand OpenTelemetry, and examine non-functional behavior.
    • Understand evaluation pipelines and prepare an evidence-based report.
    • Videos · 2 hours: EvaliQA and DeepEval execution, trace analysis, and custom metric calibration.
    • Assignment · 3 hours: Run your evaluation, analyze results, and prepare a report.
  6. Week 6February 13 · 2 hours live

    Safety, Toxicity, and Bias

    You test AI for bias, toxicity, and safety.

    February 13, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

    • Understand bias and toxicity and how to evaluate them.
    • Recognize common leaks, vulnerabilities, and attack types.
    • Define a red-teaming strategy and run controlled attacks in an authorized test environment.
    • Videos · 2 hours: Bias and toxicity evaluation with EvaliQA and a practical red-teaming guide.
    • Assignment · 2 hours: Evaluate bias and toxicity and execute your red-teaming strategy.
  7. Week 7February 20 · 2 hours live

    LLM Evaluation

    You compare LLMs using meaningful evidence.

    February 20, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

    • Identify what can be evaluated at the model level and when it is useful.
    • Explore common benchmarks and perform A/B evaluations.
    • Learn to use Hugging Face and evaluation cards.
  8. Week 8February 27 · 2 hours live

    Evaluating AI Harnesses, Voice Agents, and Images

    You adapt evaluation to agents, voice, and images.

    February 27, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

    • Evaluate AI teammates and personal agents.
    • Build golden datasets for harnesses and track quality over time.
    • Explore voice-agent considerations and image-evaluation approaches and metrics.
  9. Week 9March 6 · 3 hours live

    Interview Preparation and Job Search

    You present your AI skills to hiring teams.

    March 6, 2027 · 10:00 AM to 1:00 PM EST · Live online · 3 hours

    • Build a résumé that connects QA experience with AI evaluation skills.
    • Prepare for common interview questions and practice through mock interviews with personalized feedback.
    • Improve LinkedIn visibility, develop your job search strategy, and review optional practicum enrollment.
Finish · March 6, 2027Capstone report done, ready for interviews
Capstone project · 16-hour assignment

Bring the evaluation process together.

Evaluate a chatbot using the provided documentation. Define your test plan and datasets, assess quality and robustness, run red-teaming scenarios, and present your evidence and findings in a final report.

A capstone evaluation report with findings by severity and quality, robustness and safety scores, next to the chatbot under test
Optional add-on · After the course

Eight-Week AI Evaluation Project Practicum

Continue building experience after the course. Apply course methods through practical AI evaluation projects, deepen your skills, and build examples of your work to discuss in interviews.

  • Eight weeks of practical AI evaluation projects
  • Examples of your work to discuss in interviews
  • Available separately from the 50-hour course
Practicum tuition$999Optional program add-onAsk about the practicum
Learn from practitioners

Meet Your Instructors

Learn from people who build AI evaluation tools and lead QA teams in the field.

Portrait of Ruslan Desyatnikov

Ruslan Desyatnikov

Founder & CEO, QA Mentor

28 years in Quality Assurance

Founder and CEO of QA Mentor, a multi-award-winning global software testing company, and inventor of Human Intelligence Software Testing (HIST). Former Global Head of Testing for Worldlink Technology at Citi; built the HSBC Testing Center of Excellence with 150+ testers across seven countries. Testing Leader of the Year, North American Testing Awards 2024.

LinkedIn
Portrait of Alexander Meshkov

Alexander Meshkov

AI Evaluation Expert

14 years in Quality Assurance and AI Evaluation

Head of AI Evaluation at FirstLineSoftware. Creator of eval-ai-library, an open-source framework for AI evaluation. Developer of custom AI evaluation tools for real-world projects.

LinkedIn
Your next step starts in January

Our Next Program

January 2027 Program

Practical AI Testing and Evaluation for QA Professionals

  • Start dateJanuary 9, 2027
  • End dateMarch 6, 2027
  • Course commitment9 live Saturday sessions · 50 total hours
  • FormatLive online instruction, videos, and assignments
  • Capstone16-hour chatbot evaluation project
Early registration tuition · USD
$2,999$3,999Save $1,000

Pay by January 1, 2027 to receive the discount.

Early tuition ends in
--Days--Hours--Min--Sec
Optional add-onEight-Week AI Evaluation Project Practicum
$999
Apply for the January Program

Course Schedule

January to March 2027 · Saturdays

All times EST

19 hours of live instruction. Videos and assignments complete the 50-hour course.

Reserve your seat

Apply for the January 2027 Program

Leave your details and our team will contact you to talk through the program, tuition, and your goals. Early registration tuition is $2,999 when you pay by January 1, 2027.

$2,999 early tuition ends in
--Days--Hours--Min--Sec
  • Starts January 9, 2027, Saturdays at 10:00 AM EST
  • 9 live sessions, 50 total hours
  • 16-hour chatbot capstone and interview preparation

We use these details only to contact you about the program.

By applying you agree to our Privacy Policy.

Your questions, answered

Frequently Asked Questions

Get a clear picture of the training, practical work, schedule, and tuition before choosing your next step.

What makes this program different?

You learn AI evaluation through a QA lens, from test planning and dataset design to metric calibration, trace analysis, safety testing, and reporting.

You also learn EvaliQA, the AI testing and evaluation platform developed by QA Mentor, and complete a chatbot evaluation capstone you can showcase to hiring teams.

Who is this program designed for?

The program is designed for manual testers, automation engineers, SDETs, QA leaders, and QA professionals moving into AI evaluation. It also supports experienced testers returning to the job market after a layoff.

Do I need previous AI evaluation experience?

The course starts with AI evaluation foundations and how LLMs, RAG applications, and agents work. You build your understanding progressively through live sessions, supplementary videos, and practical assignments.

No previous AI evaluation experience is required. Familiarity with QA concepts will help you connect the material to your existing experience.

How long is the program, and when are classes?

The January program runs from January 9 to March 6, 2027, with nine live online sessions on Saturdays starting at 10:00 AM EST.

Live sessions add up to 19 hours. Videos and assignments complete the 50-hour course.

What is the optional eight-week project practicum?

The Eight-Week AI Evaluation Project Practicum is an optional $999 add-on for continued practical learning after the course.

Apply course methods through AI evaluation projects, deepen your skills, and build examples of your work to discuss in interviews.

Will the program help me prepare for interviews?

Yes. The final three-hour live session covers résumé preparation, LinkedIn visibility, job search strategies, common AI evaluation interview questions, and mock interviews with personalized feedback.

Your capstone report also gives you concrete work to explain during interviews.

Is a job guaranteed after completion?

Employment is not guaranteed. The program develops your practical AI evaluation skills and provides career preparation to support your job search.

You can present your QA experience, capstone findings, and evaluation approach to potential employers.

What do I need to participate?

You will need a computer, reliable internet access, and access to the tools used for the practical exercises.

Confirm detailed setup instructions and tool-access arrangements with QA Mentor before the program begins.