BA+QA WITh AI- Hybrid- ONLY Locals
⭐ - Featured Role | Apply direct with Data Freelance Hub
This role is a BA+QA with AI position, hybrid work for locals, offering a contract length of "unknown" and a pay rate of "unknown." Requires hands-on experience in Gen AI/ML testing, proficiency in specific tools, and a relevant degree.
🌎 - Country
United States
💱 - Currency
$ USD
💰 - Day rate
Unknown
Unknown
🗓️ - Date discovered
April 29, 2025
🕒 - Project duration
Unknown
🏝️ - Location type
Unknown
📄 - Contract type
Unknown
🔒 - Security clearance
Unknown
📍 - Location detailed
San Jose, CA
🧠 - Skills detailed
#Regression #Model Evaluation #Data Science #ML (Machine Learning) #Scripting #Langchain #AI (Artificial Intelligence) #YAML (YAML Ain't Markup Language) #Security #JavaScript #Agile #REST (Representational State Transfer) #Pytest #Python #JSON (JavaScript Object Notation) #Business Analysis #Automation #POSTMAN #REST API
Role description
Role Summary
We are looking for a detail-oriented engineer with experience in Gen AI / ML application
testing, business analysis, and product validation. You will help shape the quality of next-
gen AI products through systematic testing, prompt validation, and tool-driven evaluation.
Key Responsibilities
• Design and execute test cases for Gen AI / ML features and user workflows
• Use GenAI-specific tools to validate prompt consistency, hallucination detection, etc.
• Collaborate with product managers to convert requirements into test cases and test
data
• Perform exploratory testing, regression, and prompt-based scenario testing
• Write automation scripts to simulate user behavior and backend interactions
• Track and manage issues using QA platforms and agile tools
• Document test plans, test reports, and AI evaluation metrics
Required Skills
• Hands-on testing experience with Gen AI / ML products
• Experience with LLM testing tools like:
• - Promptfoo (prompt testing & evaluation)
• - LangSmith (LangChain tracing & evals)
• - TruLens (feedback tracking for LLMs)
• - Rebuff (security and behavior testing)
• Solid understanding of LLM behavior, hallucinations, prompt design
• Scripting: Python, Shell, or JavaScript
• REST APIs, JSON, YAML
• Familiarity with PyTest, Postman, Selenium, or similar tools
Nice-to-Have Skills
• Experience testing RAG, chatbot, or LLM agent systems
• Familiarity with LangChain, LlamaIndex, or Haystack
• Business analysis experience in AI projects
• Knowledge of AI/ML model evaluation metrics
Education
Bachelor's or Master's in CS, Data Science, AI/ML, or related field
Role Summary
We are looking for a detail-oriented engineer with experience in Gen AI / ML application
testing, business analysis, and product validation. You will help shape the quality of next-
gen AI products through systematic testing, prompt validation, and tool-driven evaluation.
Key Responsibilities
• Design and execute test cases for Gen AI / ML features and user workflows
• Use GenAI-specific tools to validate prompt consistency, hallucination detection, etc.
• Collaborate with product managers to convert requirements into test cases and test
data
• Perform exploratory testing, regression, and prompt-based scenario testing
• Write automation scripts to simulate user behavior and backend interactions
• Track and manage issues using QA platforms and agile tools
• Document test plans, test reports, and AI evaluation metrics
Required Skills
• Hands-on testing experience with Gen AI / ML products
• Experience with LLM testing tools like:
• - Promptfoo (prompt testing & evaluation)
• - LangSmith (LangChain tracing & evals)
• - TruLens (feedback tracking for LLMs)
• - Rebuff (security and behavior testing)
• Solid understanding of LLM behavior, hallucinations, prompt design
• Scripting: Python, Shell, or JavaScript
• REST APIs, JSON, YAML
• Familiarity with PyTest, Postman, Selenium, or similar tools
Nice-to-Have Skills
• Experience testing RAG, chatbot, or LLM agent systems
• Familiarity with LangChain, LlamaIndex, or Haystack
• Business analysis experience in AI projects
• Knowledge of AI/ML model evaluation metrics
Education
Bachelor's or Master's in CS, Data Science, AI/ML, or related field
