Product Analyst

GC and USC are preferred. EAD we can still consider Title : Product analyst Location - Phoenix, AZ is preferred but remote can be considered Duration - 6+ months Rate - Low in Market Skills - Product Analyst Required Qualifications • 3+ years experience drafting detailed Jira user stories, maintaining product backlogs, and defining acceptance criteria based on business requirements in an Agile product team or equivalent environment. • 2+ years experience in data analysis with the ability to interpret AI performance metrics and translate findings into actionable product recommendations. • 1+ year experience working with LLM-based applications, including understanding of hallucination patterns, output quality issues, grounding techniques, bias detection, and mitigation strategies. • 1+ year hands-on experience executing structured AI agent evaluation plans, tracking iterations, maintaining golden datasets, and performing regression testing. • 1+ year experience supporting the development, evaluation, and continuous improvement of AI-powered products, AI agents, copilots, or generative AI solutions. • Experience translating AI product requirements into prompts, agent workflows, business rules, and measurable success criteria. • Experience defining AI product success metrics and measuring improvements in output quality, user satisfaction, task completion, or productivity. • Experience partnering with engineering, UX, data science, and business stakeholders to prioritize, evaluate, and refine AI product capabilities. Preferred Qualifications • Experience in HR technology, learning technology, content creation platforms, or knowledge management solutions as a Business Analyst, Product Analyst, Product Owner, or equivalent role. • Hands-on experience building and executing specialized AI test cases, evaluating LLM outputs, prompt variations, and agentic workflows. • Hands-on experience with prompt engineering, prompt optimization, and evaluation of AI-generated content quality, accuracy, completeness, and consistency. • Experience designing evaluation rubrics, benchmark datasets, golden datasets, and quality measurement frameworks for AI-generated outputs. • Experience conducting side-by-side model comparisons, experimentation, or A/B testing to improve AI product performance. • Familiarity with Retrieval Augmented Generation (RAG), grounding strategies, hallucination mitigation, and content sourcing concepts. • Familiarity with automated AI evaluation tools, frameworks, and testing methodologies. • Experience with human-in-the-loop workflows and designing review, validation, and approval processes for AI-generated outputs. • Experience working with AI governance, responsible AI practices, bias detection, content safety, and model quality evaluation. • Experience evaluating content quality against defined business, compliance, instructional design, or organizational standards. • Strong written and verbal communication skills, with the ability to document evaluation methodologies, findings, recommendations, and trade-offs for both technical and business audiences.