AI QA Trainer – LLM Evaluation (Freelance)

About This Role

Shape the future of AI by evaluating and hardening large-scale language models. Tomorrow's enterprise-grade AI platforms depend on rigorous evaluation data. We need AI QA experts to challenge advanced models, document failure modes, and raise the bar for reliability, safety, and real-world performance.

What You'll Do

• Converse with models on real-world scenarios and evaluation prompts to assess performance
• Verify factual accuracy, logical soundness, and consistency across outputs
• Design and execute comprehensive test plans and regression suites
• Build clear rubrics and pass/fail criteria for evaluation
• Capture reproducible error traces with root-cause analysis and improvement suggestions
• Conduct adversarial red-teaming and jailbreak resistance testing
• Audit for bias, fairness, toxicity, and compliance readiness
• Test hallucination detection, prompt-injection resistance, and chain-of-reasoning reliability
• Validate tool-use correctness, retrieval-augmentation fidelity, and end-to-end workflows
• Partner on test automation (Python/SQL) and quality dashboarding

Key Skills & Experience

• Bachelor's, Master's, or PhD in Computer Science, Data Science, Computational Linguistics, Statistics, or related field
• Proven QA experience with ML/AI systems or safety/red-team projects
• Hands-on expertise with LLM evaluation tools (OpenAI Evals, RAG evaluators, W&B)
• Test automation frameworks (e.g., PyTest) and Python/SQL proficiency
• Strong evaluation rubric design and adversarial testing skills
• Experience with prompt engineering, bias/fairness auditing, and grounding verification
• Clear, metacognitive communication and high-signal bug reporting

What You'll Test

Hallucination detection, factual consistency, prompt-injection and jailbreak resistance, bias and fairness, chain-of-reasoning reliability, tool-use correctness, retrieval-augmentation fidelity, multilingual and domain-specific performance, compliance readiness, and precision/recall, faithfulness, toxicity, and latency SLOs.

Work Arrangement

This is a freelance contract role, 100% remote. You'll supply your own secure computer and high-speed internet. Company-sponsored benefits such as health insurance and PTO do not apply.

Salary Disclosure: This editorial market estimate of $90,000–$130,000 USD/year is not employer-provided. The original posting cited $6–$65/hour, with final rates determined by experience, expertise, and location. Our estimate reflects typical market compensation for mid-to-senior AI QA roles and may vary significantly based on your background and geography.