Senior Data Engineer

AgileSource are seeking an experienced Senior Data Engineer to support a large-scale digital transformation programme within a government environment. The successful candidate will play a key role in assessing, migrating, and archiving approximately 1,000 digital assets (HTML pages, PDFs, and slide decks) from legacy hosting platforms to modern infrastructure, including cloud-based analytical systems and long-term archival storage . Key Responsibiliti • es Lead the design and execution of data migration strategies from legacy syst • emsAssess digital assets to determine archiving vs. cloud migration pathw • aysSupport migration of selected datasets to the Analytical Cloud platf • ormEnsure compliance with data governance, security, and archival standards (e.g., National Archives requiremen • ts)Develop and maintain data pipelines and transformation lo • gicPerform data validation, quality checks, and integrity assurance throughout migrat • ionCollaborate with Business Analysts, PMO, and wider stakeholders to align delivery with programme objecti • vesIdentify and mitigate data risks, dependencies, and technical challen • gesProduce clear technical documentation and handover materi als Required Skills & Experi • ence Strong experience in data engineering and migration projects, ideally within public sector or regulated environ • mentsProven ability to work with legacy systems and modern cloud plat • formsExperience designing and implementing ETL/ELT pipe • linesFamiliarity with cloud data platforms (e.g., Azure, AWS, or GCP – preferably Azure Analytical serv • ices)Knowledge of data governance, archiving standards, and compliance frame • worksStrong skills in data modelling, transformation, and optimis • ationProficiency in relevant tools/languages (e.g., Python, SQL, Spark, or equiva • lent)Experience handling unstructured and semi-structured data formats (HTML, PDFs, docum • ents)Excellent problem-solving and stakeholder communication s kills