Senior Engineer (Agentic + Data/ML)

Senior Engineer – Agentic Systems & Data/ML its a corp to corp Company Overview Join a core engineering team supporting a leading U.S. video-content provider. This team operates across a large-scale hybrid infrastructure, supporting hundreds of distributed engineering teams across internal data centers and AWS cloud environments (p. 1). Role Summary We are seeking a Senior Engineer to build agentic retrieval workflows, scalable data pipelines, and ML/LLM-driven scoring systems (p. 1). You will focus on improving engineering productivity by designing retrieval agents, optimizing LLM prompting, and integrating backend services that process high-volume code metadata (p. 1). Key Responsibilities Agentic & Scoring Systems: Design retrieval workflows to extract code context and metadata; develop PR-level scoring systems with automated feedback loops (p. 1). Data Engineering: Build robust pipelines to track PRs, commits, and branches; implement retry logic and data-quality checks for large-scale datasets (p. 1). Orchestration & Reliability: Support Dagster migration and orchestration efforts; ensure pipeline scalability and fault tolerance within AWS (p. 2). LLM Optimization: Use frameworks like DSPy to optimize prompting strategies while balancing performance, latency, and cost (p. 2). Backend Integration: Develop APIs to expose code-quality metrics and collaborate with frontend teams (p. 2). Required Qualifications 5–10+ years of experience in software, data, or ML engineering (p. 2). Expertise in Python and modern data/ML tooling (p. 2). Proven experience with production-grade orchestration systems (Dagster preferred, Airflow acceptable) (p. 2). Hands-on experience with AWS-native pipelines (ECS, EventBridge, Lambda, S3, Glue) (p. 3). Practical experience with LLM prompting, retrieval systems, and agentic workflows (p. 3). Strong understanding of Git metadata (PRs, diffs, merges) and data modeling (p. 3). Nice to Have Experience with DSPy or similar prompt optimization frameworks (p. 3). Familiarity with code-quality systems like SonarQube (p. 3). Experience with observability (metrics, logging, tracing) and data-quality alerting (p. 3). Job Type: Contract Pay: $55.00 per hour Expected hours: 40 per week Application Question(s): How many years of experience do you have with Python and production-grade orchestration (e.g., Dagster or Airflow)?" For Agentic Systems: "Briefly describe a production-level agentic workflow or LLM-driven system you have built. What framework (e.g., LangChain, DSPy) did you use?" For Data Engineering: "Describe your experience migrating or managing complex data pipelines in AWS using ECS, S3, or Glue." Do you have experience with data/ML, backend systems, or agentic/LLM-based workflows? Are you currently open to working on a C2C basis? Expertise in Python and modern data/ML tooling Hands-on experience with AWS-native pipelines Practical experience with LLM prompting, retrieval systems, and agentic workflows Education: Master's (Required) Experience: data/ML, backend systems, or agentic/LLM-based workflows?: 5 years (Required) Location: Remote (Required) Work Location: Remote