Databricks Data Engineer

Databricks Data Engineer – Healthcare Project Contract: 6 Months Rate: £400–£450 per day, Inside IR35 Location: Fully Remote – UK Wide Sector: Healthcare About the Role Llooking for an experienced Databricks Data Engineer to join a large-scale healthcare data project on an initial 6-month contract. This is an excellent opportunity for a hands-on Data Engineer with strong experience in Databricks, Azure data technologies and large-scale data migration. You will play a key role in migrating existing SQL, Azure Synapse and Azure Data Factory (ADF) workloads into a modern Databricks-based data platform. Key Responsibilities • Design, develop and maintain scalable data engineering solutions using Databricks. • Migrate existing SQL workloads, Azure Synapse solutions and Azure Data Factory (ADF) pipelines to Databricks. • Assess existing data pipelines and transform them into efficient Databricks-based solutions. • Develop and optimise data pipelines using PySpark and SQL. • Work with large volumes of healthcare data while ensuring data quality, reliability and performance. • Implement effective data partitioning strategies to optimise performance and scalability. • Troubleshoot and optimise Databricks workloads and data pipelines. • Work with Azure data services and modern data engineering technologies. • Collaborate with data architects, engineers, analysts and wider project teams. • Follow appropriate data governance, security and quality standards, particularly within a healthcare environment. Essential Experience You must have demonstrable commercial experience with: • Databricks and large-scale data engineering. • Migrating SQL workloads to Databricks. • Migrating Azure Synapse workloads to Databricks. • Migrating Azure Data Factory (ADF) pipelines to Databricks. • Data partitioning, including understanding how to design and optimise partitioning for large datasets. Eligibility Candidates must have full and unrestricted right to work in the UK. Unfortunately, candidates requiring visa sponsorship or who do not have full right to work in the UK will not be eligible for this position. Contract Details • Duration: 6 months • Rate: £400–£450 per day • IR35: Inside IR35 • Location: Fully Remote – UK Wide • Sector: Healthcare • Start: ASAP / subject to availability If you are an experienced Databricks Data Engineer with proven experience migrating SQL, Azure Synapse and ADF pipelines to Databricks, we would be keen to hear from you.