Data Engineer

Position Summary: Title: Data Engineer Duration: 12 Months – Long Term Location: Washington, DC 20433 Hybrid: 4 days onsite per week from day 1 Background and Context The Data Engineer in this role will support programs involving one or more of the following: • responsible for building persistent cloud data pipelines, integrating microservices, and structuring data to power Amazon OpenSearch, Amazon Neptune (Knowledge Graphs), and Amazon SageMaker for advanced analytics. Scope of Work Pipeline Development and Implementation • Build continuous, event-driven streaming pipelines using Amazon EventBridge and SQS. • Orchestrate complex ELT transformations using EKS and Databricks. • Develop automated data feeds for the enterprise data platform and downstream applications. Solution Design and Optimization • Design and populate graph data models for Amazon Neptune to support entity relationship tracking. • Build and optimize vector indexes for Amazon OpenSearch to power the platform's AI/ML and RAG Q&A capabilities. • Ensure real-time or near-real-time data latency targets are met for operational dashboards. Stakeholder Engagement and Change Management • Partner directly with AI/ML Data Scientists to ensure data is properly curated, partitioned, and served for real-time model inference. • Support front-end developers by building reliable, performant data APIs. • Present pipeline architectures during technical reviews. Governance, Ethics, and Risk • Implement fine-grained, row-level Access Control to secure sensitive procurement data. • Set up Amazon CloudWatch and Dynatrace for continuous pipeline monitoring, alerting, and telemetry. • Ensure data served to AI models is clean and unbiased according to institutional guidelines. Documentation and Reporting • Maintain architectural diagrams for streaming data flows. • Write API endpoint documentation and AI data prep runbooks. Required Qualifications and Experience Education • Bachelor’s or Master’s in Computer Science, Data Engineering, or a related quantitative field. Certifications (Preferred) • AWS Certified Data Engineer – Associate or Microsoft Certified Azure Data Engineer. OR AWS ML Specialty Mandatory Experience • 5+ years building continuous data pipelines, real-time streaming architectures, and preparing data for machine learning workflows. Technical Knowledge • Expert SQL and Python. • Deep expertise in AWS ecosystem (EKS, Lambda, SQS, OpenSearch, Neptune, Bedrock) and Apache Spark/Databricks. Core Competencies • Strong architectural mindset, capability to handle hight-veolocity data, and enthusiasm for integrating foundational AI/ML services. “Mindlance is an Equal Opportunity Employer and does not discriminate in employment on the basis of – Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans.”