Data Scientist

Senior Data Scientist – AI & Conversational Systems (RAG / LLMs) - £800 per day Outside IR35 Location: London / Hybrid About the Role A leading financial media and data business is building a next-generation AI-powered conversational assistant designed to surface insights across both proprietary datasets and premium editorial content. The organisation is seeking a Senior Data Scientist to help evolve this platform into a high-quality, production-grade system. The role focuses on improving answer quality, retrieval performance, and real-time responsiveness - translating cutting-edge LLM capabilities into reliable, user-facing products. Responsibilities • Develop and optimise RAG (Retrieval-Augmented Generation) pipelines • Design approaches that combine: • Unstructured data (articles, reports, documents) • Structured data (databases, SQL systems) • Build and refine text-to-SQL capabilities • Evaluate and improve LLM outputs: • Accuracy, grounding, and hallucination reduction • Retrieval relevance and ranking performance • Design and implement evaluation frameworks (offline and online) • Collaborate with engineering teams to deploy models into production environments • Identify and solve real-world implementation challenges • Contribute to shaping and executing a focused AI roadmap Requirements Essential: • Proven experience delivering production-grade RAG systems • Deep expertise in working with LLMs for high-quality answer generation • Databricks experience • Strong experience with both: • Unstructured text data • Structured data / SQL systems • Experience in model evaluation, experimentation, and optimisation • Ability to translate complex problems into scalable, practical solutions