Data Engineer

Senior Data Engineer (SC Cleared) Location: UK (Remote-first) Employment Type: Permanent or 12-Month Fixed-Term Contract Travel: Remote with occasional travel (approximately one day per month) to Leeds or Newcastle Security Clearance: Active SC Clearance required Join Amber Labs At Amber Labs, we're a people-first technology consultancy delivering innovative digital services that help organisations solve complex challenges. We work with multidisciplinary teams across the public sector, partnering with our clients to deliver secure, scalable, and user-centred technology solutions. Collaboration is at the heart of everything we do. We foster an inclusive and supportive culture where knowledge sharing, continuous learning, and engineering excellence are encouraged. Our teams are empowered to challenge ideas, innovate, and deliver meaningful outcomes in environments where quality and trust matter. We're looking for an experienced Senior Data Engineer to join our growing data practice. If you're passionate about modern cloud-based data engineering, enjoy mentoring others, and thrive in Agile delivery environments, we'd love to hear from you. The Role As a Senior Data Engineer, you'll design, build, and operate robust data platforms that power mission-critical analytics within a complex UK Government environment. Working closely with architects, engineers, product teams, and stakeholders, you'll develop scalable data pipelines, modernise legacy data platforms, and ensure high standards of engineering across the full software delivery lifecycle. You'll also play an important role in mentoring colleagues and helping shape engineering best practices. A strong background in AWS, PySpark, and Informatica is essential, alongside experience delivering secure, scalable data solutions within UK Central Government. What You'll Be Doing • Design, build, and maintain production-grade data pipelines using AWS services including Amazon EMR, S3 and Lambda. • Develop scalable data processing solutions using PySpark, Python and SQL with a focus on performance, resilience, testing and observability. • Lead the migration and modernisation of legacy ETL and reporting workloads into cloud-native architectures. • Develop reusable data engineering components and shared frameworks. • Build and optimise data transformations and models supporting Management Information (MI), reporting and analytics, including downstream Power BI solutions. • Work with Informatica as part of enterprise data integration and migration activities. • Own engineering best practices including CI/CD, automated testing, version control and release governance. • Conduct code reviews and mentor engineers, promoting high-quality engineering standards across delivery teams. • Collaborate with architects, product owners and platform teams to translate business requirements into robust technical solutions. • Participate fully in Agile ceremonies using Jira and Confluence. • Embed security, governance and compliance throughout the engineering lifecycle, ensuring solutions meet UK Government security and data handling requirements. Essential Skills & Experience We're looking for someone with strong technical expertise combined with excellent communication and collaboration skills. You'll have: • Active SC Security Clearance (essential). • Proven experience delivering data engineering solutions within UK Central Government. • Strong commercial experience with Informatica. • Hands-on expertise with AWS services including Amazon EMR, S3 and Lambda. • Advanced PySpark, Python and SQL skills for large-scale data processing. • Experience building and maintaining enterprise-scale batch data pipelines. • Strong understanding of cloud-native data engineering architectures. • Experience modernising legacy ETL platforms and migrating workloads to AWS. • Experience implementing CI/CD pipelines, Infrastructure as Code (Terraform) and automated testing. • Strong knowledge of Git/GitLab, branching strategies and code review practices. • Experience designing and consuming APIs for secure data integration. • Strong understanding of data quality, governance and engineering best practices. • Experience working in Agile delivery teams using Jira and Confluence. • Excellent stakeholder management skills, with the ability to communicate effectively across both technical and non-technical audiences. • Experience producing clear technical documentation and contributing to architectural discussions. Desirable Experience It would be advantageous if you also have experience with: • Amazon Redshift and modern data warehousing. • Dimensional data modelling and dbt. • Power BI and supporting Business Intelligence teams with data models. • Wider AWS services including Athena, EC2, CloudWatch and IAM. • Event-driven architectures and modern integration patterns. Working at Amber Labs At Amber Labs, we believe our people are our greatest strength. We invest in creating an environment where talented individuals can collaborate, learn from one another, and deliver exceptional outcomes together. When you join us, you'll benefit from: • Permanent or 12-month fixed-term opportunities. • Meaningful projects supporting critical UK public sector services. • A collaborative, inclusive and supportive engineering culture. • Opportunities to work alongside highly experienced cloud, data and engineering professionals. • Continuous learning, mentoring and professional development. • Modern Agile delivery practices and access to cutting-edge cloud technologies. What We Offer • 25 days annual leave plus public holidays. • Private medical insurance with Bupa. • Remote-first working with a base at Liverpool Street whenever you want it. • Personal training budget to invest in your professional development. • Perkbox access, offering hundreds of perks and discounts. • EV scheme after one year of service. • Regular team socials with colleagues who genuinely enjoy working together. • Employer pension contributions. • The opportunity to help shape the future of Amber Labs as we continue to grow. Why Join Amber Labs? At Amber Labs, you'll be part of a collaborative consultancy delivering impactful digital solutions across the public sector. We focus on building long-term capability, maintaining high delivery standards, and creating meaningful outcomes for users. If you're an experienced SC Cleared Senior Data Engineer with a passion for cloud-native data engineering and delivering secure, high-quality solutions for UK Central Government, we'd love to hear from you.