Senior Data Engineer (Contract/InsideIR35)

AgileSource is seeking an experienced Senior Data Engineer to support a large-scale digital transformation programme within a government environment. The successful candidate will play a key role in assessing, migrating, and archiving approximately 1,000 digital assets (HTML pages, PDFs, and slide decks) from legacy hosting platforms to modern infrastructure, including cloud-based analytical systems and long-term archival storage. Key Responsibilitie • sLead the design and execution of data migration strategies from legacy system • sAssess digital assets to determine archiving vs. cloud migration pathway • sSupport migration of selected datasets to the Analytical Cloud platfor • mEnsure compliance with data governance, security, and archival standards (e.g. National Archives requirements • )Develop and maintain data pipelines and transformation logi • cPerform data validation, quality checks, and integrity assurance throughout migratio • nCollaborate with Business Analysts, PMO, and wider stakeholders to align delivery with programme objective • sIdentify and mitigate data risks, dependencies, and technical challenge • sProduce clear technical documentation and handover material s Required Skills & Experien • ceStrong experience in data engineering and migration projects, ideally within public sector or regulated environmen • tsProven ability to work with legacy systems and modern cloud platfor • msExperience designing and implementing ETL/ELT pipelin • esFamiliarity with cloud data platforms (e.g. Azure, AWS, or GCP – preferably Azure Analytical Service • s)Knowledge of data governance, archiving standards, and compliance framewor • ksStrong skills in data modelling, transformation, and optimisati • onProficiency in relevant tools/languages (e.g. Python, SQL, Spark, or equivalen • t)Experience handling unstructured and semi-structured data formats (HTML, PDFs, document • s)Excellent problem-solving and stakeholder communication skil ls