Principal Data Engineer | Azure Data & Analytics Leadership

Principal Data Engineer | Azure Data & Analytics Leadership 📍 Hybrid Working 📅 Initial 6-Month Contract | Long-Term Programme (Up to 2 Years) Are you a visionary Data Engineering leader ready to shape the future of enterprise data and analytics? We are seeking an experienced Principal Data Engineer to lead the design, delivery, and optimisation of a strategic Azure-based data platform. This is a high-impact opportunity to drive technical excellence, strengthen data governance, and build scalable analytics solutions supporting a major enterprise transformation initiative. The Opportunity As Principal Data Engineer, you will provide both strategic direction and technical leadership across a large-scale cloud data ecosystem. You'll be responsible for driving data platform maturity, establishing best practices, and enabling business-critical insights through modern data engineering capabilities. Key Responsibilities • Architect, design, and implement scalable Azure Data Lake solutions • Lead the development of modern data pipelines using Databricks Delta Live Tables and Structured Streaming • Define technology roadmaps across Databricks, Unity Catalog, Qlik Replicate, and Azure services • Drive technical excellence, engineering standards, and solution quality • Optimise cloud resources to support high-volume data processing while maintaining cost efficiency • Establish and enhance enterprise data governance, lineage, metadata, and security practices • Monitor and improve production data pipelines and platform performance • Partner closely with business and technology stakeholders to align data initiatives with organisational goals • Translate technical challenges into clear business-focused recommendations for senior leadership • Mentor and develop engineering teams, fostering a culture of continuous improvement and innovation Required Experience • 15+ years of experience delivering enterprise-scale data and analytics solutions • Proven track record designing and implementing modern Data Lake architectures • Extensive experience leading cloud-based data transformation programmes • Strong background in large-scale data engineering and analytics platforms • Bachelor's or Master's degree in Computer Science, Information Technology, or a related discipline • Azure Data Engineering and/or Databricks certifications preferred Technical Skills • Azure Data Factory (ADF) • Azure Data Lake Storage (ADLS) • Azure Databricks • Delta Live Tables • Unity Catalog • Azure Stream Analytics • Azure Synapse Analytics • Advanced SQL • Python, Scala, and Spark-SQL • Enterprise ETL/ELT architecture and optimisation • Data migration (on-premises to cloud and cloud-to-cloud) • Data governance, metadata management, and data quality frameworks Nice to Have • Hadoop, Kafka, and Spark • Power BI or Qlik Sense • AI/ML, NLP, and Generative AI exposure • Agile and Kanban delivery experience About the Programme You will play a key role in the evolution of the Products Data Exchange (PDE), a strategic Central Data Repository initiative supporting the organisation's Vision 2030 objectives. This platform is designed to centralise and harmonise enterprise data, enabling improved accessibility, stronger governance, enhanced collaboration, and advanced analytics capabilities across the business. Why Apply? ✅ High-profile enterprise transformation programme ✅ Strategic leadership opportunity ✅ Cutting-edge Azure and Databricks environment ✅ Hybrid working model ✅ Initial 6-month contract with strong likelihood of extension for up to 2 years 📩 Interested? Apply now or contact me directly for a confidential discussion.