About The Role

AI Trainer jobs in Canada focus on improving large language models through RLHF-style evaluations, prompt evaluation, data labeling, and QA evaluation across real AI/ML training pipelines. You will follow detailed annotation guidelines, verify training data quality, and provide structured feedback that improves helpfulness, correctness, and safety.

What You’ll Do

• Perform RLHF evaluations (pairwise ranking, rubric-based scoring) and write clear rationales
• Execute prompt evaluation for instruction-following, factuality, and safety
• Label and validate datasets for NLP and content safety labeling
• Run QA evaluation checks (consistency, agreement, systematic error discovery)
• Document edge cases and build error taxonomies to drive model performance improvement
• Collaborate on rubrics, gold sets, calibration, and regression testing for model updates

Skills And Qualifications ...