Generative AI Engineer

GenAI Data Engineer Location: London / Edinburgh (Hybrid - 2 days per week in the office) Day Rate: Market rate (Inside IR35) Duration: 6 months The Role We are seeking a highly skilled GenAI Data Engineer to join a forward-thinking team delivering advanced data and AI solutions. This role will focus on designing scalable data platforms and integrating Generative AI capabilities into enterprise systems. Key Responsibilities • Design, build and maintain scalable data pipelines using PySpark, Python and distributed computing frameworks • Architect and optimise AWS-based data and AI infrastructure for secure, high-performance data processing • Develop, fine-tune, benchmark and evaluate GenAI/LLM models, including custom training and inference optimisation • Implement and maintain Retrieval-Augmented Generation (RAG) pipelines, vector databases and document processing workflows • Build reusable frameworks for prompt management, evaluation and GenAI operations • Collaborate with cross-functional teams to integrate GenAI solutions into production environments • Ensure data quality, governance and operational reliability across systems Essential Skills & Experience • Strong experience with PySpark and large-scale distributed data processing (ETL/ELT pipelines) • Advanced SQL expertise, including schema design (star/snowflake), indexing and optimisation techniques • Proven experience implementing CDC and SCD (Type 1/2/3) in data warehousing environments • Advanced Python skills for data engineering, automation and AI/ML integration • Hands-on experience with AWS services (e.g. S3, Glue, Lambda, EMR, Bedrock or custom model hosting) • Practical experience developing and evaluating GenAI/LLM models • Strong understanding of RAG architectures, embeddings and vector databases • Experience handling structured and unstructured data (e.g. text, documents, logs, images) • Knowledge of scalable storage solutions such as Delta Lake, Parquet, Redshift and DynamoDB • Experience optimising AI models (e.g. quantisation, distillation, inference tuning) • Strong troubleshooting and performance optimisation skills across distributed systems Desirable Skills • Additional experience across PySpark, Python, SQL, AWS and GenAI technologies Personal Attributes • Strong analytical and problem-solving abilities • Effective communication skills and ability to work collaboratively • Experience working in agile, cross-functional environments