Principal AI Data Engineer (AI/GenAI Data Engineer)

Role Summary Aubay UK is seeking an experienced Principal AI Data Engineer to join a major transformation programme within the energy sector. This is a 6-month contract engagement (inside IR35) with possible extension, focused on designing, developing and deploying enterprise-scale GenAI and Agentic AI solutions. The role combines advanced AI engineering, Databricks expertise, stakeholder engagement and delivery leadership, taking AI products from rapid prototyping through to production adoption. Type of Role Contract Inside IR35 (6 months with possible extension). Location and Travel Location: London Waterloo. Working pattern: On-site, 5 days per week. Travel requirements: None. Visa sponsorship available. Required Skills and Experience • 8-12+ years of experience in Data Science, AI or related disciplines. • Degree in Statistics, Mathematics, Econometrics or similar. • Deep understanding of LLMs including GPT, Llama, Claude and Mistral. • Strong Databricks expertise including DLT, Delta Lake and Unity Catalog governance. • Experience with Spark Structured Streaming and Autoloader. • Strong Python, SQL and/or Scala skills. • Proven data modelling, ETL/ELT and data architecture experience. • Azure cloud experience. • Hands-on experience with Azure OpenAI, AI Foundry, MLflow, Mosaic AI, Agent Bricks and Genie. • Experience with GenAI evaluation frameworks and benchmarking methodologies. • Strong Git and collaborative development practices. • Excellent communication, stakeholder management and problem-solving skills. Desired Skills and Experience • Experience with enterprise AI governance models. • Knowledge of LangChain, LangGraph, AutoGen and smolagents. • Experience with Docker and CI/CD pipelines. • Familiarity with ADF and Qlik Replicate. • Experience leading backlogs and technical roadmaps. • Commercial experience supporting trading, supply chain or energy businesses. Role Responsibilities • Develop and evaluate AI, GenAI and Agentic AI prototypes. • Build, tune and evaluate RAG solutions. • Design and deploy AI agents and scalable AI workflows. • Deliver production-ready GenAI solutions for business users. • Build and manage large-scale structured and unstructured datasets using Databricks. • Implement testing, benchmarking and reproducibility frameworks. • Deploy models using Azure AI services, Databricks and Docker. • Apply DevOps best practices including CI/CD, testing and GitHub workflows. • Collaborate with technical and business stakeholders to ensure value delivery. • Contribute to architecture, planning, sprint ceremonies and backlog development. Client Description Our client is one of the Super Major global energy companies with around 84,000 employees across 70+ countries who are working to power progress through cleaner energy solutions. Specialties include upstream and downstream operations, biofuels, integrated gas, new energies, chemicals, energy and trading. You will have the opportunity to work in a challenging and rewarding environment focused on delivering innovative solutions that support the transition to a lower-carbon energy future.