Lead Data Engineer

Lead Data Engineer Key Points • Location: London Or Reading, Hybrid • Duration: 12 months • Rate: Competitive Day Rate, Outside IR35 Overview This is an opportunity to join a leading organisation undertaking a major cloud transformation and data modernisation programme, delivering enterprise-scale data solutions across a complex technology landscape. As a Lead Data Engineer, you will provide technical leadership across the design, development, automation, and optimisation of cloud-based data platforms, ensuring secure, scalable, and high-performing solutions are delivered to support business and operational objectives. Working closely with architects, stakeholders, analysts, and engineering teams, you will drive best practice across data engineering, infrastructure automation, CI/CD, and cloud technologies while mentoring and leading other engineers within the delivery team. Key Responsibilities • Lead the design and delivery of enterprise-scale data engineering solutions within Google Cloud Platform (GCP). • Develop and maintain scalable ETL/ELT pipelines, data models, and integration frameworks. • Design and manage Infrastructure as Code (IaC) solutions using Terraform. • Build and support automated deployment processes through GitLab CI/CD pipelines. • Design, optimise, and maintain BigQuery datasets, schemas, and analytical workloads. • Develop and manage Cloud Storage, Cloud SQL, and Cloud Composer solutions. • Develop Python-based ETL applications, automation tools, and data services. • Implement real-time data integration and Change Data Capture (CDC) solutions using Striim. • Develop and support ETL workflows and reusable components using Ab Initio. • Ensure data quality, governance, security, and compliance standards are maintained. • Monitor platform performance, reliability, and operational efficiency across cloud environments. • Lead code reviews, mentor engineers, and establish engineering best practices. • Collaborate with cross-functional teams to deliver secure, reliable, and scalable data solutions. • Support architecture design, technical planning, estimation, and continuous improvement initiatives.