Data Analyst III : 25-05999

Primary Skills: Risk Assessment-Expert, Safety Testing-Advanced, Prompt Engineering-Intermediate, Red Teaming-Expert, LLM Safety Risk-High Contract Type: W2 Duration: 3 - 6 Months Location: Palo Alto, CA - 94306 ( - Remote) Pay Range: $70.42 - $80.42 per hour on W2 #LP Job Summary This role involves creating and executing safety testing strategies for LLM-based features, evaluating test results for comprehensive risk assessment, and documenting findings accurately to ensure improvements in system safety and user trust. The successful candidate will work closely with cross-functional teams to identify potential misuses, refine testing processes, and provide actionable insights for enhancing safety protocols in AI systems. Prior experience in a similar role, particularly within Trust & Safety, QA, or adversarial testing, and a strong understanding of AI system risks and mitigation strategies are essential. Key Responsibilities • Create and execute comprehensive safety testing plans for LLM-based features. • Evaluate and analyze testing outcomes for risk assessment and propose mitigation actions. • Document testing strategies, results, and actionable feedback meticulously. • Maintain effective communication with internal stakeholders to align on safety standards and improvements. • Proactively identify and suggest enhancements to the red teaming and testing processes. Must-Have Skills • Advanced understanding of LLM safety risks and mitigation strategies. • Proven ability to apply an adversarial mindset to identify potential system misuses. • Strong documentation and organizational skills to accurately track and report findings. Industry Experience Required • Experience in AI, Trust & Safety, or product QA highly preferred. • Background in Engineering, Computer Science, Data Science, or Information Systems is required. About Akraya Akraya is an award-winning IT staffing firm consistently recognized for our commitment to excellence and a thriving work environment. Most recently, we were recognized Inc's Best Workplaces 2024 and Silicon Valley's Best Places to Work by the San Francisco Business Journal (2024) and Glassdoor's Best Places to Work (2023 & 2022)! Industry Leaders in IT Staffing As staffing solutions providers for Fortune 100 companies, Akraya's industry recognitions solidify our leadership position in the IT staffing space. We don't just connect you with great jobs, we connect you with a workplace that inspires! Join Akraya Today! Let us lead you to your dream career and experience the Akraya difference. Browse our open positions and join our team!