Data Engineer

Job Title: Data Engineer (Databricks SME) Rate: DOE (outside IR35) Location: Remote Contract Length: 6 months A consultancy client of ours have secured a project requiring an Azure Databricks expert. This is an exciting opportunity to work on cutting-edge data projects, building scalable data pipelines and cloud-based systems that deliver real impact. Key Responsibilities: • Lead the design, development and optimisation of scalable data solutions using Azure Databricks • Provide subject matter expertise on Databricks architecture, best practices and performance tuning • Collaborate with data engineering, BI and analytics teams to deliver robust and reusable data pipelines • Drive the adoption of Databricks features such as Delta Lake, Unity Catalog, and MLflow where appropriate • Support the migration of legacy ETL processes to Databricks-based workflows • Ensure data quality, governance and security standards are met across all Databricks solutions • Mentor and upskill team members in Databricks usage and data engineering techniques • Troubleshoot complex technical issues and act as the escalation point for Databricks-related queries • Contribute to the continuous improvement of the data platform, tooling and engineering practices • Work closely with stakeholders to understand data needs and deliver fit-for-purpose solutions at pace Experience and Qualifications Required: • Extensive hands-on experience with Azure Databricks, including Delta Lake, notebooks, and job orchestration • Strong proficiency in Python, SQL and Spark for building and optimising data pipelines • Solid understanding of cloud architecture, ideally within Azure, including Data Lake, Data Factory and related services • Experience designing and implementing data solutions using dimensional modelling (e.g. Kimball methodology) • Proven track record of delivering data products in large-scale, enterprise environments • Familiarity with data governance, security, and compliance frameworks • Experience with CI/CD practices and DevOps tools in a data engineering context • Strong problem-solving skills and ability to troubleshoot complex data issues • Excellent communication and stakeholder engagement skills across technical and non-technical teams • Previous experience mentoring or upskilling engineers in Databricks or data engineering practices • Experience working in retail or finance sectors is highly desirable If this sounds like an exciting opportunity please apply with your CV.