Data Engineer

We’re looking for a highly skilled Data Engineer with deep expertise in Azure Databricks and modern data engineering practices. This role is ideal for someone who thrives in fast‑moving environments, collaborates well with scientists and engineers, and can independently translate business needs into scalable data solutions. Responsibilities • Design, build, and optimize data pipelines using Azure Databricks, PySpark, Delta Lake, and Workflows • Develop robust ETL/ELT processes for both batch and streaming workloads • Ingest and integrate data from RESTful SaaS APIs, including sources with and without CDC capabilities • Implement strong governance and security practices using Unity Catalog, role-based access, data quality checks, and lineage tracking • Collaborate with scientists, engineers, and business stakeholders to understand workflows and translate them into scalable data models • Ensure reliability and performance of production pipelines, including monitoring, alerting, and SLA adherence • Contribute to CI/CD processes using Azure DevOps, Git, and automated deployment patterns • Support schema design and data modeling for scientific and laboratory data environments Required Qualifications (Technical Must‑Haves) • Hands-on experience with Databricks on Azure: Spark (PySpark), Delta Lake, Workflows • Strong skills in optimization, tuning, and debugging distributed data workloads • Expertise in ETL/ELT design, batch/stream processing, and scalable data architecture • Experience ingesting data from RESTful APIs • Solid understanding of data governance, security, and compliance best practices • Proficiency in Python, SQL, Git, and CI/CD pipelines (Azure DevOps) • Experience with Azure Data Lake Storage Gen2 and production monitoring/alerting • Proven ability to productionize pipelines with performance tuning and SLA management