Senior Data Engineer

Senior Data Engineer (Databricks) – Contract 📍 London, UK (2/3 days on site) 💷 £400 per day (Inside IR35) 🕒 6-month contract (Extendable) Travel: Occasional travel to Dublin may be required Our client, a top global organization, is seeking a Senior Data Engineer to design, build, and operate production-grade data products across customer, commercial, financial, sales, and enterprise data domains. This role is strongly focused on Databricks-based engineering, delivering trusted, governed, and scalable datasets that support reporting, analytics, and advanced use cases. Key Responsibilities • Design, build, and maintain Databricks pipelines using Delta Lake and Delta Live Tables (DLT) • Implement medallion architectures (Bronze / Silver / Gold) • Deliver reusable, well-documented, and discoverable data products • Ensure pipelines meet non-functional requirements (freshness, latency, scalability, reliability, and cost) • Own Databricks assets including Jobs/Workflows, notebooks, SQL, and Unity Catalog objects • Apply Git-based DevOps practices (branching, PRs, CI/CD) and Databricks Asset Bundles (DABs) • Implement monitoring, alerting, incident response, and root cause analysis • Support production operations with runbooks and operational standards • Enforce governance and security using Unity Catalog (lineage, classification, ACLs, row/column-level security) • Define and maintain data quality rules, expectations, and SLOs • Support investigation and resolution of data anomalies and production issues • Partner with Product Owners, Data Engineering Manager, Data Scientists, and business stakeholders to translate business requirements into functional and non-functional data solutions Essential Skills & Experience • 6+ years’ experience in data engineering or advanced analytics engineering • Strong hands-on expertise in Python and SQL • Proven experience building production pipelines in Databricks • Solid understanding of data modelling, performance tuning, and cost optimisation • High attention to detail with strong documentation and process-design skills Desirable Experience • Strong Databricks Lakehouse expertise (Delta Lake, DLT, batch & streaming pipelines) • Lakehouse monitoring, data quality, and observability • Unity Catalog governance and security in regulated environments • Databricks DevOps/DataOps with CI/CD and environment promotion • Performance and cost optimisation (autoscaling, Photon/serverless, OPTIMIZE/VACUUM) • Semantic layer or metrics engineering experience • Cloud-native analytics platforms (Azure preferred) If this is relevant to your experience, please apply with your CV and we’ll be in touch.