Tech Lead - Azure Databricks Data Platform

Tech Lead - Azure Databricks Data Platform Duration: Until end of year Workload: Full time hours Setup: Day rate contract inside IR35 Location: London (Hybrid) We are looking for an experienced Technical Lead to drive the design, development, and optimisation of a large-scale Azure Databricks data platform supporting near real-time enterprise data and reporting capabilities. Key Responsibilities: • Lead the architecture, development, and delivery of scalable data solutions using Azure and Databricks. • Design and implement high-performance data pipelines leveraging Structured Streaming, Delta Live Tables (DLT), PySpark, Python, and SQL. • Establish and maintain data governance, data quality, and platform best practices. • Optimise Databricks performance, streaming workloads, and cloud cost efficiency. • Provide technical leadership across architecture, development, deployment, and production support. • Collaborate with business stakeholders to translate requirements into robust technical solutions. • Mentor engineering teams and drive technical decision-making across the programme. • Support cloud modernisation initiatives and enterprise-wide data strategy objectives. Must-Have Skills: • Strong technical leadership experience delivering enterprise-scale data platforms. • Expert knowledge of Azure Data Platform and Databricks. • Hands-on experience with Structured Streaming and Delta Live Tables (DLT). • Advanced PySpark architecture and engineering expertise. • Strong Python and SQL development skills. • Experience with data governance, data quality, and data architecture best practices. • Proven stakeholder management and communication skills. • Experience within financial trading environments and trading data models. Nice-to-Have: • Experience with cloud modernisation programmes. • Exposure to energy trading or commodities data platforms. • Knowledge of Endur, DEX, or similar trading applications. • Experience building near real-time reporting and analytics solutions.