Data Engineer

Data Engineer Salary: £70,000 - £80,000 Contract: 6 month fixed term contract Location: City of London / hybrid We are supporting a leading organisation within the specialty insurance market that is investing heavily in a major transformation programme across its data, technology and operating model. As part of this programme, they are looking for an experienced Data Engineer to join on a 6 month fixed term contract and play a key role in delivering a modern cloud based data capability. This is a hands on delivery role, suited to someone who can take ownership of a defined piece of work, build high quality data pipelines, and help create reliable datasets that support reporting, analytics and business decision making. The successful candidate will be working with modern Azure data technologies, helping to improve how data is ingested, transformed, validated and made available to the business. Key responsibilities • Design and build scalable data pipelines using Azure Databricks • Transform raw data into clean, structured and reliable datasets for reporting and analytics • Develop Databricks notebooks, jobs and workflows to support end to end data processing • Implement data quality checks, validation rules and monitoring processes • Troubleshoot pipeline issues and improve reliability, performance and maintainability • Work across development and production environments using CI/CD principles • Collaborate with data, technology and business stakeholders as part of a wider transformation programme • Produce clear technical documentation to support future ownership and handover Experience required • Strong commercial experience as a Data Engineer • Hands on experience with Azure Databricks • Strong Python, PySpark and Spark SQL skills • Experience building and optimising data pipelines • Good understanding of modern cloud data platforms and Lakehouse architecture • Experience working with Delta Lake or similar technologies • Understanding of data quality, validation, monitoring and pipeline performance • Experience with CI/CD processes, ideally using Azure DevOps • Ability to work independently and deliver within a project focused environment Desirable experience • Experience supporting reporting or BI platforms such as Power BI • Experience within insurance, financial services or another regulated environment • Exposure to enterprise transformation or data modernisation programmes This is a strong opportunity for a Data Engineer to join a business undergoing meaningful transformation and take ownership of a visible data delivery project using modern Azure technologies. Apply following the link Data & Application Notice We process personal data submitted as part of your application for recruitment purposes in line with UK GDPR. All applications are reviewed by a member of our team, we do not use automated decision-making in our selection process. Your application will be assessed against the requirements of this role. By applying to this advert, you consent to receiving future relevant roles and industry news and insights from us. To opt out of this, please contact: compliance@pioneer-search.com. For more information on how we use your data, including how long we retain it & your rights, please refer to our Privacy Policy: www.pioneer-search.com/privacy-policy.