Senior Data Engineer — Azure, Databricks & ML Pipelines

We're looking for a Senior Data Engineer to design, build, and maintain scalable data pipelines and ML-ready infrastructure on Azure and Databricks. This is a hands-on engineering role: you'll own the full data pipeline lifecycle — ingestion, transformation, orchestration, and deployment — while supporting machine learning workflows with clean, reliable data. If you're comfortable owning infrastructure decisions and writing production-quality Python at scale, this role is built for that. What You'll Do • Design, build, and maintain data pipelines using Databricks and Azure-native data services • Develop and optimize ETL/ELT processes to support analytics and machine learning workloads • Build and maintain CI/CD pipelines for data engineering and ML deployment workflows • Write clean, efficient, production-quality Python for data processing and pipeline automation • Support machine learning teams with well-structured, high-quality datasets and feature pipelines • Design and manage data architecture across Azure services (e.g., Azure Data Factory, Azure Data Lake, Azure Synapse) • Monitor pipeline performance, troubleshoot data quality issues, and implement reliability improvements • Implement data governance, security, and access control best practices • Collaborate with data scientists, analysts, and software engineers to align data infrastructure with business needs • Participate in code reviews, architecture discussions, and technical planning What You Bring • Strong hands-on experience with Azure cloud data services • Proven experience building and maintaining pipelines on Databricks • Solid experience designing and managing CI/CD pipelines for data or ML workflows • Strong Python skills for data engineering and pipeline development • Working knowledge of machine learning workflows and how data engineering supports them • Experience with SQL and relational/distributed data systems • Understanding of data pipeline orchestration, monitoring, and reliability practices • Strong problem-solving skills and ability to work independently on complex data infrastructure challenges • Solid communication skills for collaborating with data science and engineering teams Nice to Have • Experience with MLOps practices and tools (MLflow, Azure ML) • Familiarity with Spark internals and performance tuning within Databricks • Experience with infrastructure-as-code (Terraform, Bicep, ARM templates) • Exposure to real-time/streaming data pipelines (Kafka, Event Hubs, Structured Streaming) • Relevant Azure or Databricks certifications