AI Scenario Quality Control Lead

I’m helping Mercor find a top candidate to join their team flexible for the role of AI Scenario Quality Control Lead. You'll guide the next generation of frontier AI by ensuring the highest standards of simulated reality. Compensation: Hidden Location: Remote: USA Mission of Mercor: "We connect human expertise with leading AI labs and enterprises to train frontier models." What makes you a strong candidate: • You are proficient in Quality assurance (QA), Microsoft Excel, JSON, Audit, Airtable. • You have the potential to develop in SQL, Python, Jira, Editing, Data analysis. • English - Conversational Responsibilities and more: We're looking for highly analytical, detail-oriented professionals to serve as Quality Control Leads, focused on evaluating and auditing complex simulated environments and scenarios designed to train frontier AI agents. As an Auditor/Reviewer, you will be the final line of defense, ensuring that the digital worlds (spanning Gmail, Slack, Drive, etc.) and agentic tasks crafted by our design team meet the highest standards of realism, complexity, and technical accuracy. You will apply strict grading rubrics to assess scenario quality and provide vital feedback to the creators. This is a W2 employment position with Cincinnatus LLC, with the opportunity to be placed at a leading AI Lab as part of their extended workforce. You will join a team of domain experts and together, you will guide the next generation of frontier AI tools and agents.What You'll Do Audit & Evaluate: Review richly detailed personas, digital environments, and tasks created by AI Training Scenario Designers to ensure they are logically sound and properly structured. Apply Strict Rubrics: Utilize complex evaluation rubrics to score scenarios objectively, ensuring the tasks effectively challenge an AI agent's ability to reason, filter, and prioritize without being broken or impossible. Quality Assurance: Identify logical inconsistencies, factual errors, missing context, and technical flaws (e.g., invalid JSON structures) within the simulated environments. Provide Feedback: Deliver clear, constructive, and actionable feedback to the scenario designers to help them refine their worlds and improve their overall output. Document Outcomes: Maintain rigorous quality standards by logging your review outcomes, scores, and feedback clearly in Airtable and Crucible. You're a Good Fit If You Have a proven track record in QA, editing, auditing, or reviewing complex written/technical content. Ability to engage for :40 hours/week Excel at learning and rigorously applying comprehensive grading rubrics (prior high scores in rubric-based training or academies is a massive plus). Have an "eagle eye" for inconsistencies, edge cases, and logical gaps that others miss. Are comfortable giving constructive, structured feedback to peers and creators. Can quickly navigate structured tools like JSON editors, Airtable, and AI platforms. Must Have: Undergrad degree, 2+ years professional experience. Nice to Have: Editing / QA background — Senior editor, QA tester, technical reviewer, compliance auditor, or peer-reviewer. Technical / data comfort — JSON familiarity, Jira/Asana power user, data analyst, SQL/Python exposure. You don't need to write the code from scratch, but you must be able to spot when a JSON structure is broken. Process & Systems thinking — Operations managers, technical PMs, or systems designers who understand how complex, multi-stakeholder workflows should logically function in a professional setting.