Mindlance
Lead Data Engineer
⭐ - Featured Role | Apply direct with Data Freelance Hub
This role is for a Lead Data Engineer on a remote contract, requiring expertise in Python, ETL patterns, and cloud platforms (Azure, GCP). Candidates must possess strong skills in data integration, streaming technologies, and Agile methodologies, with hands-on experience in scalable data platforms.
🌎 - Country
United States
💱 - Currency
$ USD
-
💰 - Day rate
Unknown
-
🗓️ - Date
June 26, 2026
🕒 - Duration
Unknown
-
🏝️ - Location
Remote
-
📄 - Contract
Unknown
-
🔒 - Security
Unknown
-
📍 - Location detailed
United States
-
🧠 - Skills detailed
#Spark (Apache Spark) #SQL (Structured Query Language) #"ETL (Extract #Transform #Load)" #REST (Representational State Transfer) #Stories #Kafka (Apache Kafka) #Cloud #Kubernetes #Data Engineering #Agile #Batch #GCP (Google Cloud Platform) #Azure #AWS Kinesis #PostgreSQL #Data Integration #Python #SQS (Simple Queue Service) #MariaDB #AWS (Amazon Web Services) #Automation #Databases #Databricks #Scala #SQL Server #MongoDB #REST API #ML (Machine Learning) #MS SQL (Microsoft SQL Server) #Data Ingestion #AI (Artificial Intelligence) #Terraform #Data Processing #PySpark
Role description
Role: Lead Data Engineer
Location: Remote
Job Responsibilities:
• Own the product backlog: decompose epics and business vision into well-defined, developer-ready features and user stories
• Partner with engineering and architecture teams to design scalable, reusable enterprise data platform capabilities
• Define and govern batch and streaming data ingestion and integration patterns across the platform
• Drive and own ceremonies of SAFe Agile / LPM, including PI planning, iteration goals, and backlog refinement
• Incorporate agentic AI and LLM-based automation into platform development workflows
• Align cross-functional stakeholders on technical tradeoffs, platform roadmap, and delivery priorities
• Ensure platform capabilities are cloud-native, scalable, and built for reuse across enterprise consumers
• Must be hands-on technical with the ability to independently write proof-of-concepts (POCs) and reference implementations to guide and accelerate the development team
• Build reusable data integration and data ingestion platform capabilities that can be leveraged by all application teams to acquire data from systems of record and make it available in the data platform
• Build capabilities to catalog acquired data from systems of record, enabling discoverability and governance across the platform
• Expected to deeply understand the "what" — the business intent and product vision — and independently derive a very detailed and technical "how" — the implementation approach, architecture decisions, and engineering specifications ready for the development team
Technical Skills Required
• Data Processing Python, PySpark, PyFlink, ETL/ELT batch & streaming patterns
• Data Platforms Databricks, Databricks Lakeflow, structured & unstructured data engineering
• Databases & CDC MS SQL Server, PostgreSQL, MariaDB, MongoDB (CDC patterns), Kafka Connect, Debezium
• Streaming & Eventing Confluent Kafka, AWS EventBridge, AWS Kinesis, AWS SQS, Google Pub/Sub, Azure Event Hub
• Application Development REST APIs, Python-based service development
• Cloud & Infrastructure Azure, GCP, Terraform, Kubernetes
• Agile SAFe / LPM — PI planning, epic decomposition, backlog ownership
• AI/ML Agentic AI development patterns and integration workflows
Role: Lead Data Engineer
Location: Remote
Job Responsibilities:
• Own the product backlog: decompose epics and business vision into well-defined, developer-ready features and user stories
• Partner with engineering and architecture teams to design scalable, reusable enterprise data platform capabilities
• Define and govern batch and streaming data ingestion and integration patterns across the platform
• Drive and own ceremonies of SAFe Agile / LPM, including PI planning, iteration goals, and backlog refinement
• Incorporate agentic AI and LLM-based automation into platform development workflows
• Align cross-functional stakeholders on technical tradeoffs, platform roadmap, and delivery priorities
• Ensure platform capabilities are cloud-native, scalable, and built for reuse across enterprise consumers
• Must be hands-on technical with the ability to independently write proof-of-concepts (POCs) and reference implementations to guide and accelerate the development team
• Build reusable data integration and data ingestion platform capabilities that can be leveraged by all application teams to acquire data from systems of record and make it available in the data platform
• Build capabilities to catalog acquired data from systems of record, enabling discoverability and governance across the platform
• Expected to deeply understand the "what" — the business intent and product vision — and independently derive a very detailed and technical "how" — the implementation approach, architecture decisions, and engineering specifications ready for the development team
Technical Skills Required
• Data Processing Python, PySpark, PyFlink, ETL/ELT batch & streaming patterns
• Data Platforms Databricks, Databricks Lakeflow, structured & unstructured data engineering
• Databases & CDC MS SQL Server, PostgreSQL, MariaDB, MongoDB (CDC patterns), Kafka Connect, Debezium
• Streaming & Eventing Confluent Kafka, AWS EventBridge, AWS Kinesis, AWS SQS, Google Pub/Sub, Azure Event Hub
• Application Development REST APIs, Python-based service development
• Cloud & Infrastructure Azure, GCP, Terraform, Kubernetes
• Agile SAFe / LPM — PI planning, epic decomposition, backlog ownership
• AI/ML Agentic AI development patterns and integration workflows
