Lead Data Engineer

Lead Data Engineer Contract Outside IR35 6 Months Initially 2/3 days a week on-site - Birmingham ASAP Start My client are seeking an experienced Lead Data Engineer to lead the migration of customer data from a third-party platform onto the new Databricks environment. This is a hands-on leadership role requiring deep Databricks expertise combined with architectural ownership and delivery accountability. Key Responsibilities • Lead the migration of customer data from an external third-party provider to the new Databricks platform. • Define and implement the end-to-end data architecture strategy within Databricks. • Design scalable, secure, and high-performance data pipelines. • Establish governance, data modelling standards, and best practice frameworks. • Act as technical authority and SME for Databricks across the programme. • Provide architectural oversight while remaining hands-on with engineering delivery. Required Experience • Deep hands-on expertise in Databricks (architecture, optimisation, deployment). • Deep expertise in Azure data services (Databricks, Data Factory, ADLS) and strong understanding of distributed data processing. • Hands-on experience with CI/CD tooling (Azure DevOps/GitHub) and modern DevOps practices for data engineering. • Strong grasp of data architecture, ETL/ELT design, and performance optimisation. Desirable Experience • Proven experience leading enterprise data platform implementations. • Strong experience migrating customer data between platforms. • Experience operating in senior/lead/senior/lead/principle-level engineering roles. • Strong understanding of data governance, security, and compliance frameworks. • Comfortable leading small, high-performing teams. • Experience working within retail, eCommerce, or customer-focused environments (preferred).