Lead Data Engineer

Lead Data Engineer (BigQuery Consolidation, Migration & Architecture) – Contract Location: Hybrid - London Contract Length: Initial 4-week engagement with extension Day Rate: £700 per day (Outside IR35) Start Date: ASAP An ambitious, data-driven organisation is seeking a highly experienced Lead Data Engineer to assess and potentially lead the consolidation of a complex, fragmented data landscape into a single Google BigQuery platform. The business currently operates across multiple technologies and data warehouses, creating challenges around data consistency, reporting accuracy, and stakeholder trust. The objective is to establish a scalable, future-proof architecture that delivers a true single source of truth while supporting an AI-enabled data strategy. Initially, this will be a four-week exploratory engagement focused on validating the migration approach, identifying risks, and proving technical feasibility. For the right individual, there is potential to lead the wider migration and transformation programme. The Challenge The current environment consists of multiple platforms, including: • Google BigQuery • Snowflake • Microsoft Fabric Over time, the data estate has become increasingly fragmented, resulting in: • Multiple versions of reporting across the business • Reduced confidence in data quality and consistency • Complex data flows and transformations • Increasing operational overhead and technical debt The organisation believes there is a significant opportunity to simplify the architecture and align around a strategic Google Cloud and AI-first approach. What You'll Be Doing Architecture & Strategy Assessment • Conduct a deep assessment of the existing data architecture, pipelines, and reporting landscape. • Challenge assumptions and identify architectural risks, bottlenecks, and potential migration pitfalls. • Define a target-state architecture centred on Google BigQuery. • Evaluate scalability, maintainability, performance, and future AI-readiness. Migration Validation • Perform hands-on analysis of critical data pipelines and transformations. • Validate that complex workloads can be successfully migrated to BigQuery without loss of fidelity, performance, or business value. • Assess migration complexity, business impact, and delivery timelines. • Develop a practical roadmap for consolidation. Data Quality & Governance • Define frameworks for validating data integrity and achieving data parity across platforms. • Establish measurable success criteria for migration and reporting accuracy. • Recommend approaches for automated testing and validation at scale. Executive Recommendations At the conclusion of the engagement, provide a clear assessment covering: • Migration feasibility • Key risks and mitigation strategies • Cost-benefit analysis • Delivery roadmap and timeline estimates • Go / No-Go recommendation for the wider programme What We're Looking For Essential Experience • Proven experience leading large-scale data platform migrations and consolidation programmes. • Deep expertise in Google BigQuery architecture and best practices. • Strong understanding of modern cloud data platforms including: • Google BigQuery • Snowflake • Microsoft Fabric • Experience designing scalable, enterprise-grade data architectures. • Strong understanding of modern data engineering tooling and practices, including dbt and cloud-native services. Technical Mindset • Highly hands-on and technically credible. • Comfortable diving into existing pipelines, transformations, and codebases rather than operating solely at a strategic level. • Able to quickly understand complex environments and identify practical solutions. • Strong analytical and problem-solving skills with the confidence to challenge assumptions. Desirable Experience • Retail or e-commerce experience. • Exposure to AI-enabled data platforms and modern analytics ecosystems. • Experience supporting organisations through cloud-first or AI-driven transformation initiatives. Technology Environment • Google Cloud Platform (GCP) • BigQuery • Gemini • AI-enabled analytics and decision-making • The successful consultant will play a key role in shaping the future direction of the organisation's data platform. Interview Process • Single-stage interview process. • Direct engagement with senior leadership. • Interviews will be arranged quickly, often within the same week of application. • This role would suit a senior, pragmatic data engineering leader who has successfully delivered complex migrations before and enjoys combining strategic thinking with hands-on execution.