Agentic AI Automation
⭐ - Featured Role | Apply direct with Data Freelance Hub
This role is for a Functional Tester specializing in AI Agents, contracted for "X months" at "$X/hour". Key skills include Python, automation frameworks, and performance testing. Experience in AI systems and leadership in QA strategies is required. Remote work location.
🌎 - Country
United States
💱 - Currency
$ USD
-
💰 - Day rate
-
🗓️ - Date discovered
September 11, 2025
🕒 - Project duration
Unknown
-
🏝️ - Location type
Unknown
-
📄 - Contract type
Unknown
-
🔒 - Security clearance
Unknown
-
📍 - Location detailed
Tampa, FL
-
🧠 - Skills detailed
#Pytest #Python #AI (Artificial Intelligence) #GitLab #Datasets #Automation #BERT #Leadership #Scripting #GitHub #API (Application Programming Interface) #Jenkins
Role description
Job Description:
As a Functional Tester specializing in AI Agents, you will be responsible for designing, executing, and maintaining test cases that validate the functionality, reliability, and performance of AI-driven agent systems. You will work closely with development teams, AI specialists, and product owners to ensure that AI agents perform accurately and efficiently in real-world scenarios.
Key Responsibilities:
AILLM-Specific QA Designing test frameworks for LLM outputs (hallucination checks, factuality tests, toxicitybias detection).Automation Frameworks PyTest, Robot Framework, Playwright, Cypress, or custom LLM test harnesses.
Evaluation Metrics BLEU, ROUGE, BERT Score, GPT-based evaluators, human-in-the-loop validation.
Data QA Ensuring high-quality training evaluation datasets, data validation (Great Expectations, Deequ).
Python Automation Scripting test cases, API test automation.
CICD Integration Automated test pipelines in Jenkins GitHub Actions GitLab CI.
Performance Testing Latency, throughput, stress testing of LLM-powered services.
Leadership (Manager level) Define QA strategies for AI systems, lead test automation teams, enforce quality gates.
Job Description:
As a Functional Tester specializing in AI Agents, you will be responsible for designing, executing, and maintaining test cases that validate the functionality, reliability, and performance of AI-driven agent systems. You will work closely with development teams, AI specialists, and product owners to ensure that AI agents perform accurately and efficiently in real-world scenarios.
Key Responsibilities:
AILLM-Specific QA Designing test frameworks for LLM outputs (hallucination checks, factuality tests, toxicitybias detection).Automation Frameworks PyTest, Robot Framework, Playwright, Cypress, or custom LLM test harnesses.
Evaluation Metrics BLEU, ROUGE, BERT Score, GPT-based evaluators, human-in-the-loop validation.
Data QA Ensuring high-quality training evaluation datasets, data validation (Great Expectations, Deequ).
Python Automation Scripting test cases, API test automation.
CICD Integration Automated test pipelines in Jenkins GitHub Actions GitLab CI.
Performance Testing Latency, throughput, stress testing of LLM-powered services.
Leadership (Manager level) Define QA strategies for AI systems, lead test automation teams, enforce quality gates.
