Evals Specialist - Artifacts

This contractor will support the Frontier Evals team in executing a high-priority evaluation campaign by monitoring the end-to-end data collection process, performing quality control on incoming data, and helping maintain throughput without compromising standards. The role requires fluency in one of Consulting, Finance, Data Science, Software Engineering, or Legal so the contractor can reliably judge whether tasks, deliverables, and rubrics reflect expectations for high-quality work in those fields. Experience in multiple domains is a plus! A passion for professional artifacts quality (docs, spreadsheets, decks, and clear and compelling writing) is a must. Reporting and collaboration Reports to: Frontier Evals Team TPM, working closely with researchers. Scope Of Responsibilities Incoming Data Quality Control (QC) • Review incoming tasks, deliverables, and rubrics to confirm they meet campaign standards. • Identify common failure modes • Provide actionable feedback to contributors and reviewers, and track campaign progress. Data Collection Process Monitoring • Monitor campaign health and throughput, including submission volume, rework rates, bottlenecks, turnaround time, and recurring quality issues. • Surface systematic issues to Frontier Evals leads and propose solutions • Maintain lightweight QC logs, issue trackers, or audit trails to support retrospectives, analysis, and process improvement. Contributor Support and Coordination (as needed) • Answer contributor questions to improve quality • Coordinate with reviewers and subject matter experts on edge cases and escalate ambiguous decisions • Help calibrate contributors on what “good” looks like for tasks and rubrics in the target domain Technical Work (light to moderate) • Use Codex, spreadsheets, light SQL, or simple Python scripts • Assist with basic data hygiene and organization of uploaded materials, naming conventions, and record completeness. Additional Tasks (as Needed) • Draft or revise campaign documentation, examples, QC checklists, and “good vs. bad” examples for the target domain • Participate in targeted QA sweeps Qualifications / Profile • 3+ years of professional experience in the target domain • Passion for docs, spreadsheets, decks, and clear and compelling writing • Strong judgment about what realistic, high-quality work looks like in that field • Excellent attention to detail and comfort applying standards and rubrics consistently • Able to write clear, direct feedback that improves quality quickly • Comfortable working in spreadsheets; familiarity with basic SQL or Python is helpful but not required • Able to work in ambiguous, fast-moving environments and make pragmatic judgment calls Pay Rate: $120-$150/hr DOE