Lead Data Engineer

Lead & Senior Data Engineers (Contract) 📍 Remote (1 day per month in office) 💰 Competitive Day Rate | Outside IR35 ⏳ Initial 6-Month Contract I'm currently working with a leading digital consultancy to recruit Lead and Senior Data Engineers for a high-profile public sector digital transformation programme. This is an opportunity to join a multidisciplinary Agile team building and enhancing a modern, cloud-native data platform that supports critical digital services used by millions across the UK. The Role You'll be responsible for designing, developing and operating scalable data platforms, working alongside architects, software engineers and delivery teams to build high-quality data products within a cloud-native environment. The Lead position also offers the opportunity to provide technical leadership, mentor engineers and drive engineering best practices across the wider team. Key Responsibilities • Design and develop scalable cloud-native data platforms. • Build and optimise ETL/ELT data pipelines using modern engineering practices. • Develop solutions using Python, SQL and Spark/PySpark. • Work extensively with AWS services including Glue, S3, Athena and Lambda. • Design robust data models and implement data quality best practices. • Contribute to automation, CI/CD and engineering standards across delivery teams. • Collaborate with architects, product teams and stakeholders throughout the software delivery lifecycle. Essential Skills • Strong commercial experience with Python, SQL and Spark/PySpark. • Hands-on experience building cloud-native data platforms. • Strong knowledge of AWS services including Glue, S3, Athena and Lambda. • Experience developing ETL/ELT pipelines. • Understanding of Delta Lake, medallion architecture and modern data engineering principles. • Experience with GitHub, automation and CI/CD. • Strong communication skills and experience working within Agile delivery teams. Desirable Experience • Azure Databricks • Azure Data Factory • Unity Catalog • Great Expectations • Experience delivering within Government, GDS or other regulated environments What's on Offer? • Competitive day rate • Outside IR35 • Initial 6-month contract with strong extension potential • Remote working with just one day per month in office • Opportunity to work on a major public sector digital transformation programme • Collaborative Agile teams using modern cloud technologies If you're an experienced Data Engineer looking for your next contract and enjoy building modern data platforms at scale, I'd be happy to discuss the opportunity in more detail. Apply now or get in touch for a confidential conversation.