Lead Data Engineer

Overview: Synechron UK is looking for a highly experienced Lead Data Engineering Consultant to lead and develop cutting-edge data engineering platforms. The ideal candidate will bring a mix of technical expertise, leadership skills, and innovative thinking to support our data-driven initiatives. Key Responsibilities: • Lead the design, development, and optimisation of enterprise data platforms using tools such as Hadoop, Spark, and Splunk. • Develop and maintain robust ETL/ELT data pipelines to handle raw, structured, semi-structured, and unstructured data (SQL and NoSQL). • Integrate large, disparate datasets with modern tools and frameworks, ensuring smooth data flow across systems. • Collaborate closely with BI and Analytics teams in fast-paced environments, supporting their data needs. • Drive the creation of technical test plans, conduct unit and integration testing, and maintain high code quality within automated environments. • Promote a Site Reliability Engineering (SRE) culture by addressing operational challenges and ensuring service resilience and sustainability. • Support continuous integration, delivery, and deployment (CI/CD) pipelines, incorporating source control practices. Qualifications: • Extensive experience with Hadoop, Spark, and Splunk. • Proficiency in Python scripting, with experience in handling various data formats. • Strong background in building and optimising data pipelines and supporting BI/Analytics teams. • Experience with CI/CD pipelines and source control. • Knowledge of agile methodologies such as Scrum or Kanban. • Ability to develop and implement automated test plans and ensure high-quality deliverables.