Data Engineer

Data Engineer – Digital Transformation in Healthcare 6 Month Contract - Inside IR35 Fully Remote Must be eligible for SC Clearance Are you a skilled Data Engineer looking to make an impact on a national healthcare programme? We are seeking a talented professional to join a small, expert team focused on modernising and transforming the Breast Screening service in England through data-driven digital capabilities. In this role, you will work across multiple datasets and systems to identify and resolve data issues, shape future data structures, and build secure, repeatable pipelines for ingestion, transformation, and cleansing. Your work will directly support the delivery of auditable, reproducible data outputs that inform critical healthcare services. Key Responsibilities • Build and maintain data ingestion and transformation pipelines, including data profiling, cleansing, standardisation, and publishing. • Establish import/export patterns, handle schema discovery, incremental loads, and multiple source instances. • Apply advanced SQL for profiling, joins, deduplication, anomaly detection, and performance tuning. • Automate processes using Python (experience with Pandas, Polars, scikit-learn, matplotlib desirable). • Work with modern data tooling (Spark, Azure Data Factory) or implement equivalent solutions in code. • Handle geospatial data, coordinate reference systems, and spatial analysis workflows, applying geographical context in processing pipelines. • Work with official open datasets (ONS, census, deprivation indices, population estimates) to enrich analytical outputs. • Implement rules for data quality, exception handling, reconciliation flows, and versioned pipelines with logging and traceability. • Operate in a secure environment with sensitive data, adhering to strict access and transfer protocols. Essential Skills & Experience • Proven experience in data transformation-heavy pipelines and data wrangling. • Strong SQL and Python capabilities with maintainable code practices. • Experience handling geospatial datasets and publicly available official datasets. • Ability to implement data quality rules, versioned pipelines, and secure handling of sensitive data. • Strong communication, collaboration, and pro-active working style. Desirable • Experience with healthcare data, patient or longitudinal records. • Experience creating automated data profiling reports or dashboards. Personal Attributes • Positive, enthusiastic, and engaged attitude. • Collaborative, team-oriented approach with strong interpersonal skills. • Self-motivated and able to handle ambiguity and change effectively. Additional Requirements • Must be eligible for UK Security Clearance due to access to live systems and sensitive personal data.