Data Engineer - Pyspark - Palantir

Data Engineer - PySpark with Palantir Hybrid - London 2-3 days onsite 6 months - Inside IR35 SC Cleared or SC Clearable About the Role: We are working with a leading consultancy who are looking for an experienced Data Engineer with strong expertise in PySpark, Python, and SQL. The ideal candidate will have hands-on experience with the Palantir Foundry platform and a passion for building scalable data solutions. You will play a key role in designing and delivering data pipelines and warehouses, working with cutting-edge cloud technologies and driving impactful projects within a fast-paced environment. Key Responsibilities: • Develop and maintain data stores and data warehouse solutions • Design and implement data pipelines integrating diverse data sources into Azure Data Lake and Azure Databases • Collaborate with Product Owners and Solution Architects to translate business requirements into technical solutions • Lead design, estimation, and planning activities for projects and PoCs • Deliver high-quality solutions independently and as part of an agile team • Mentor and lead junior Data Engineers, fostering collaboration and knowledge sharing Skills & Experience: • Minimum 5 years’ experience in PySpark, Python, and SQL • Proven experience with Palantir Foundry platform • Strong background in enterprise data analytics and distributed computing frameworks (Spark/Hive/Hadoop preferred) • Demonstrated ability to design end-to-end data management and transformation solutions • Proficient in Spark SQL and familiar with cloud platforms such as Azure or AWS • Experience with Scrum/Agile methodologies • Knowledge of JavaScript/HTML/CSS and Kubernetes is a plus • Bachelor’s degree or equivalent in Computer Science, Data Science, or related field