Data Engineer

Data Engineer (Platform & Integration), London/Hybrid, £500-£550/day (Outside IR35) Role Purpose A hands-on engineer to build out the data lake that will become the single source of truth for the business — so that iPaaS pulls from one governed, canonically-modelled platform rather than integrating directly with five separate systems. Working to the architecture and standards set by the Data Lead, this role delivers the pipelines, models and integrations that make the lake real. Day to day on Microsoft (Azure / Fabric) today, with a likely move to Google Cloud, so portable, vendor-neutral build habits matter. What You'll Do • Build the lake: Develop ingestion pipelines and the landing → curated → serving layers, following the platform design and patterns set by the Data Lead. • Implement the canonical model: Map and transform data from the five source systems into the shared canonical model, so downstream consumers work from one consistent vocabulary. • Re-point iPaaS: Migrate integrations to source from the lake, building reusable ingestion/publishing flows and helping retire legacy point-to-point connections. • Data quality & reliability: Implement validation, monitoring and alerting; keep pipelines tested, documented and dependable. • Use AI in the build: Apply AI-assisted tooling — schema mapping, data-quality checks, code and pipeline generation — to work faster, and help prepare clean, well-structured data for AI/ML and analytics consumption. • Build portably: Use open table formats (Delta / Iceberg), SQL, Python and infrastructure-as-code so the Azure→GCP move is straightforward. What You'll Bring • 2–5 years of hands-on data engineering, ideally including work on a data lake or lakehouse. • Solid SQL and Python, with practical ELT/ETL experience (event streaming, CDC or API-led integration a plus). • Comfortable building data transformations to a defined model; exposure to canonical / dimensional modelling. • Hands-on with a cloud data platform — Azure / Fabric and/or GCP (BigQuery, Dataflow); willing to work across both. • Experience with, or genuine enthusiasm for, AI-assisted engineering tooling. • Works well to someone else's architecture and standards, asks good questions, and documents as they go.