Data Engineer

Job Title: Data Engineer Duration: 12 months (with potential extension) Workload: Full-time hours Setup: Freelance Inside ir35 Location: London Hybrid – (3 days onsite, 2 days remote) Overview: We are seeking a highly skilled Data Engineer to design and deliver robust, high-performance data pipelines and enable advanced analytics for business-critical insights. The Role As a Data Engineer, you will play a key role in designing, developing, and maintaining robust cloud-based data solutions. You'll collaborate closely with technical and business stakeholders to build efficient, reliable, and scalable data platforms that enable data-driven decision making. You'll work with cutting-edge Azure technologies, leveraging Databricks and Data Factory to create high-performing data pipelines and processing frameworks. Key Responsibilities • Design, develop, and maintain scalable data engineering solutions using Azure Databricks • Build and optimize enterprise-grade data pipelines using Azure Data Factory • Process and transform large-scale datasets containing millions of records • Drive performance tuning and optimization across distributed data processing environments • Collaborate with stakeholders to gather requirements and translate business needs into technical solutions • Participate in architecture discussions and contribute to best practice implementation • Ensure high standards of code quality, testing, documentation, and maintainability • Identify and resolve technical issues, escalating risks where appropriate • Work within Agile delivery teams to deliver high-quality data solutions Required Experience • 6+ years of experience in Data Engineering, Data Platform Engineering, or related fields • Strong hands-on experience with Azure Databricks • Deep understanding of Azure Data Factory • Proven experience developing and optimizing large-scale ETL/ELT processes • Strong knowledge of data processing concepts and distributed computing • Hands-on experience with Scala • Experience working within Microsoft Azure cloud environments • Strong analytical and problem-solving skills • Excellent communication and stakeholder management abilities