Data Engineer

Title: Data Engineer Location : Rockville, MD or McLean, VA Target Start Date : ASAP Type: contract Pay Rate: DOE The Data Engineer works with moderate supervision across two equally weighted domains: (1) large-scale data pipeline development processing high-volume event data in a cloud environment, and (2) design and development of agentic AI systems, including LLM-powered data assistants, MCP servers, and agent harness architectures. This position contributes to overall product quality throughout the software development lifecycle. Responsibilities • Build and maintain ETL/ELT pipelines using Apache Spark, Hive, and Trino across S3-based data lake environments • Develop and optimize SQL for large-scale datasets, including window functions, multi-table joins, and complex aggregations • Build and engineer big data systems (EMR-on-EC2, EMR-on-EKS) and develop solutions on analytical platforms (SageMaker, Domino, Dataiku) • Participate in data quality monitoring, anomaly detection, and production incident investigation • Develop AI agent systems using AWS Bedrock and agent frameworks (e.g., Strands Agents SDK, LangChain/LangGraph, or equivalent) • Build agent harness architectures combining LLM reasoning with deterministic execution (e.g., RAG-based SQL generation and structured output validation) • Implement agent memory, context management, and tool integration (MCP servers, API connectors, data catalog lookups) • Build evaluation frameworks for agent accuracy (e.g., paraphrase robustness, routing precision, structural consistency) • Stay informed of advances in LLM frameworks and emerging AI capabilities • Write clean, well-tested code; contribute to CI/CD pipelines and infrastructure-as-code on AWS • Ensure secure handling of sensitive data across both data pipelines and AI agent outputs, including auditable execution traces • Adhere to internal standards for secure development practices and technology policies • Partner across teams, communicate technical information effectively, and maintain documentation • Actively learn from senior team members and contribute to process improvement Essential Technical Skills Data Engineering & Big Data Technologies • Experience building data pipelines using Apache Spark (PySpark preferred) and SQL • Experience with SQL query engines (Hive, Trino/Presto, or similar) and cloud platforms (AWS S3, EMR, Lambda) • Understanding of data skew, large-scale data processing challenges, and debugging strategies Generative AI & Agentic Systems • Experience building LLM-powered agent systems that use tools and produce structured outputs • Hands-on experience with agent frameworks (LangChain, LangGraph, AWS Strands, or equivalent) • Knowledge of prompt engineering, RAG architectures, and memory/context management • Experience with foundation model APIs (e.g., Anthropic Claude, Amazon Nova, OpenAI, or similar) Memory & Agent Design • Understanding of memory architectures (working, episodic, semantic memory) • Familiarity with agent harness patterns (tool routing, guardrails, verification loops, fallback handling) AI Tool Proficiency • Experience with AI development tools (e.g., GitHub Copilot, Q Developer, ChatGPT, Claude) • Experience with spec-driven development and AI-assisted coding workflows Cloud Technologies • Experience with AWS services such as S3, EMR, Lambda, Bedrock, Step Functions • Familiarity with monitoring/logging tools (CloudWatch, CloudTrail) • Exposure to platforms like Google Vertex AI or similar Programming (Python) • Strong Python skills for data engineering and automation • Ability to write clean, modular, and performant code • Understanding of functional programming concepts, concurrency, and memory management SQL • Strong proficiency in SQL (window functions, joins, aggregations) • Ability to optimize complex queries and handle edge cases (NULLs, duplicates, ordering) Nice to Have • Experience with agent frameworks and advanced patterns (evaluation harnesses, verification loops) • Model fine-tuning techniques (LoRA, PEFT, managed tuning platforms) • Vector databases (FAISS, Pinecone, OpenSearch) • Containerization and orchestration (Docker, Kubernetes, EKS) • Infrastructure as Code (Terraform, CloudFormation) • CI/CD tools (Jenkins, GitLab CI, GitHub Actions, ArgoCD) • Observability tools (Prometheus, Grafana, ELK stack) • Cloud or AI-related certifications Education / Experience • Bachelor's degree in Computer Science, Data Science, Information Systems, or related field, with 2+ years of relevant experience (or equivalent practical experience) • Experience delivering enterprise-quality software solutions using object-oriented and database technologies • Knowledge of modern software engineering practices (test automation, build automation, configuration management) • Strong written and verbal communication skills • Ability to build effective working relationships and collaborate across teams • Ability to learn new technologies quickly and work in a fast-paced environment Welcome to ConsultNet, a premier national provider of technology talent and solutions. Our expertise spans across project services, contract-to-hire, direct search, and managed services onshore, nearshore, and hybrid. For over 25 years, we have connected thousands of consultants with meaningful roles through a personal, communication-driven approach, partnering with a diverse client base to build high-performing teams and create lasting impact. Our comprehensive service offerings cover a wide range of technology and engineering positions across key markets nationwide. Learn more at www.consultnet.com . We champion equality and inclusivity, proudly supporting an Equal Opportunity Employer policy. We welcome applicants regardless of Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other status protected by law.