Systems Design Expert | Remote

Work Snapshot Type: Hourly contract Location: Remote Commitment: Full-time or Part-time What You’ll Be Doing • Evaluate AI-generated responses to coding and software engineering queries for accuracy, reasoning, clarity, and completeness • Conduct fact-checking using trusted sources and validate outputs by executing code • Annotate model responses by identifying strengths, gaps, and inaccuracies • Assess code quality, readability, and algorithmic soundness • Ensure responses align with expected conversational behavior and engineering best practices • Apply consistent evaluation standards using defined taxonomies, benchmarks, and guidelines What We’re Looking For • Strong experience in software engineering, data science, or systems design • Strong academic background in computer science or a related technical field • Strong experience in at least two programming languages such as Python, Java, C++, JavaScript, Go, Rust, or similar • Strong problem-solving skills with the ability to handle complex coding challenges • Experience contributing to open-source projects is a plus • Familiarity with using AI tools for coding and understanding their strengths and limitations • High attention to detail with the ability to identify logical flaws and inefficiencies • Strong experience in model evaluation, data annotation, or RLHF is a plus • Ability to explain complex technical concepts clearly How To Apply • Upload resume • Interview • Submit form