Data Engineer

Job Overview To assist in the design, implementation, and optimization of data pipelines to load and maintain the data warehouses, to support the Trust's data and analytics needs. The post holder will be a technical expert in data engineering tools and practices and will drive the development of robust and scalable data architectures to enable advanced analytics and reporting capabilities across the organization. The post holder will be responsible for the development and deployment of the Trust’s data within the data warehouse environment, which consists of multiple SQL Server databases that capture data from the Trust’s multiple data sources including Clinical systems, Electronic Patient Record systems, Electronic Staff Records, Finance, and other disparate internal and external data sources. Main duties of the job Job Summary • Develop, test & implement solutions built in the SQL Server Environment and business intelligence (BI) reporting software ensuring that it meets the requirements of the end users and the Data Engineering team’s development standards. • Supporting the development of technical BI skills across the wider Insight & Analytics department including, but not limited to T-SQL, ETL Tools (SSIS, Python or dbt) and Visualisation (Tableau, Power BI). • Work with the analytical teams and business users to develop high-quality data models for reporting and dashboarding purposes. • Establish and manage metadata repositories to support development pipelines and processes. • Ensure data quality across the data platform by managing assurance and providing guidance to resolve data quality issues with regional partners and divisional teams. • Support the Data Platform Modernisation (DPM) Programme, including migration of data pipelines, models and reporting objects from SQL Server to Snowflake, and contribute to Data Quality Improvement Project deliverables Working for our organisation Are you looking for an exciting opportunity to develop your data engineering skills while helping shape the future of healthcare analytics? We are seeking a motivated individual to join our Central Business Intelligence (BI) Data Engineering Team on a 12-month development secondment. This is an excellent opportunity to gain hands-on experience working with both our established SQL Server data warehouse and modern cloud platform as we migrate to Snowflake. Working alongside experienced Data Engineers, you will help develop and maintain data pipelines and warehouse solutions that support reporting and analytics across the Trust. You’ll gain exposure to clinical, operational and corporate data while developing experience in enterprise data architecture and information management. As part of the role, you will contribute to the Trust’s Data Platform Modernisation Programme, developing skills in SQL, ETL/ELT, Python, dbt and cloud technologies. You will also support the development of key National Operating Framework (NOF) metrics and nationally mandated datasets, gaining experience in NHS performance reporting. This secondment offers an opportunity to broaden your technical expertise, work on transformation projects and develop the skills needed for the next stage of your career within a supportive and collaborative team. If you are looking to challenge yourself, gain valuable experience and make a meaningful contribution across CNWL, we would love to hear from you. Please note: This vacancy does not meet the criteria for Skilled Worker sponsorship, unless you meet the criteria by temporary exemption from current changes to immigration rules put in force on 22/07/2025. “Separate provisions are applied to workers who have been sponsored and held continuously Skilled Worker visa since prior to 04/04/2024." As such, if you don’t meet the Transitional Provision (above) we are unable to consider your application unless you can provide documentary evidence of your right to work in the United Kingdom. If you believe you are eligible for sponsorship or already hold a valid right to work in the UK, please ensure you provide full details of your immigration status in the Pre-Screening Immigration section of your application form. Please note role eligibility also depends on whether the role meets the salary threshold for the relevant occupational code (SOC CODE). For further info please visit: Skilled Worker visa: Overview - GOV.UK Job Summary Detailed job description and main responsibilities • Develop, test & implement solutions built in the SQL Server Environment and business intelligence (BI) reporting software ensuring that it meets the requirements of the end users and the Data Engineering team’s development standards. • Supporting the development of technical BI skills across the wider Insight & Analytics department including, but not limited to T-SQL, ETL Tools (SSIS, Python or dbt) and Visualisation (Tableau, Power BI). • Work with the analytical teams and business users to develop high-quality data models for reporting and dashboarding purposes. • Establish and manage metadata repositories to support development pipelines and processes. • Ensure data quality across the data platform by managing assurance and providing guidance to resolve data quality issues with regional partners and divisional teams. • Support the Data Platform Modernisation (DPM) Programme, including migration of data pipelines, models and reporting objects from SQL Server to Snowflake, and contribute to Data Quality Improvement Project deliverables Information Management • Take lead responsibility for ensuring that the key mandatory NHS national returns (CSDS, CDS, IAPT and MHSDS) are submitted in line with the requirements and timescales of NHS England. • Manage the Trust’s NHS Dataset Change Notice (DSCN) Procedure and ensure that all relevant parties are notified as to the nature, impact and timetable for any changes. • Keep abreast of all national reporting developments and respond appropriately to new and/or revised statutory reporting requirements. • Reviewing the NHS England website for changes to dataset requirements regularly and ensuring that these changes are reflected appropriately in the Trust’s reporting. • Support the delivery of performance metrics required under the National Operating Framework, ensuring alignment between national submission requirements and internal performance reporting. Planning and Organisation • Demonstrate and champion how the new the data engineering tools can be used to develop and run ‘intelligent information’ reports that draw upon, integrate, analyse and make links between different information sources to support more effective service planning, monitoring and delivery. • Design, build, and maintain scalable data pipelines to ingest, transform, and load data from various sources (e.g., clinical systems, external data feeds) into the Trust's data platforms. • To develop and extend expertise in database design concepts and SQL server language, including T-SQL, ETL (SSIS, Python or dbt) and Visualisation (Tableau, Power BI). • To ensure high quality data warehouse security, putting in place robust security measures to allow adequate access for the required users whilst protecting sensitive patient-identifiable data. • Develop expert understanding of the IT systems operated by CNWL with regard to data models and structures so as to facilitate work to develop new and more complex reporting objects that require effective and intelligent linkage between sources of information held on different systems. • Support the migration of existing data pipelines and reporting objects from SQL Server to Snowflake as part of the Trust's Data Platform Modernisation Programme, ensuring continuity of national and local reporting throughout transition. • Ongoing requirement to act with minimum guidelines, setting standards for others, establishing how professional and administrative policies should be interpreted and implemented • To provide training and demonstration and ensure provision and translation of complex analytical information to non-specialist users and provision of support in their understanding of the analysis • Provision of technical support to business intelligence managers for the purpose of developing release schedules of required data items • Ensure all documentation relating to own area is complete and fit for purpose and all releases are controlled. • Lead in the development and maintenance of a culture of service provision and continuous improvement for own areas. • Supporting team members to maintain an up to date knowledge and understanding of NHS information requirements and IT developments as they apply to Trust systems. • Contribute to the strategic planning of team projects, identifying interdependencies across projects/functions, potential impacts on wider organisation, resource requirements and building in contingency and adjustments as necessary. Collaboration • Create strong and sustainable business relationships with senior colleagues both internally and externally. Establish and use a network of peers to ensure consistent and accurate information and approaches are used. Share and promote the sharing of information and knowledge. • Coordinate discussion within the team to solve a problem and utilise the knowledge and experience of team colleagues. • Participate and present in meetings as directed by BI colleagues, providing information advice and support where requested. • Seek out development opportunities for you to learn, both as an individual and as a team, sharing your knowledge with your peers • Working with and assisting Data Base Administrators (DBAs) ,to design and develop the structure, procedures and functions of the Trust’s data warehouse data models. Including data security, responsibility for the administration of permissions to the Trust’s data warehouse datasets. Learning and Development • Provide ongoing support, training and guidance to key internal stakeholders (and, potentially, some external stakeholders) so that they are able to use the new BI system to build and run their own information reports, adjusting the information content, parameters and presentation so as best to meet their needs. • Proactively identify appropriate training / development opportunities that could enhance the skills of the team to the benefit of the team’s stakeholders. • Plan, develop and evaluate methods and processes for gathering, analysing, interpreting and presenting data and information. Management And Leadership Responsibilities • Develop and empower all members of your team to perform to high standards and innovate. • Proactively participate in monthly 1:1 management supervision, annual objective setting and Personal Development Review (PDR) / appraisal processes. • Develop staff knowledge and skills to promote equality and diversity and address inequalities both in employment and service delivery. Ensure specific equality objective is included in PDRs. • Develop a culture that ensures that the standards of Improving Working Lives and Investors in People are achieved and maintained for all staff and that staff’s perception about their working lives are measured and improvements made. • Coordinate and ensure all aspects of work within the team (or project team) are well managed and delivered on time, helping to resolve day-to-day issues and provide immediate and detailed support to all levels within the team. • Provide relevant and timely specialist advice and guidance on own portfolio of projects and analytical areas. Rider Clause This is an outline of the post holder’s duties and responsibilities. It is not intended as an exhaustive list and may change from time to time to meet the changing needs of the Trust and Department. Person specification Education & Qualifications Essential criteria • Educated to master’s level in relevant subject or equivalent level qualification or significant experience of working at a similar level in specialist area. Experience & Knowledge Essential criteria • Significant experience in using common SQL-based query tools. • Proven experience in data engineering, with expertise in designing, building, and maintaining data pipelines, data warehouses, and/or data lakes • Experience of Using ETL tools like SQL Server Integration Services (SSIS) or an equivalent ETL/ELT tool. • Knowledge of data modelling, database design, and data integration patterns. • Familiarity with DevOps practices and tools for collaborative development, testing, and deployment. • Experience in the use of cloud based BI tools including Microsoft Azure BI solutions (e.g. Azure Data Factory, MS SQL server, Snowflake or equivalent tool) Desirable criteria • Knowledge of data governance principles, data quality practices, and data security measures. • Experience in the use of cloud based BI tools including Microsoft Azure BI solutions (e.g. Azure Data Factory, MS SQL server, Snowflake or equivalent tool) • Experience contributing to data quality improvement initiatives or data governance frameworks within a regulated or public sector environment. • Experience with other programming languages such as Python • Experience of using any reporting tool preferably Tableau • Previous experience of working within the NHS or social care services and/or Knowledge of the key performance indicators for NHS and social care mental health services. • Previous experience of working within mental health or substance misuse services Communication Skills Essential criteria • Clear communicator with excellent writing, report writing and presentation skills; capable of constructing and delivering clear ideas and concepts concisely and accurately for diverse audiences. • Engaging and comprehensive presentational skills for conveying complex concepts to a varied audience Analytical Skills Essential criteria • Excellent analytical skills with a proven ability to analyse and interpret information, pre-empt and evaluate issues, and recommend appropriate course of action to address the issues using complex, conflicting data from multiple sources. • Excellent numeric, critical reasoning, analytical skills – able to interpret, analyse, integrate and effectively communicate data and information from a range of sources using tables, graphs and text. • Ability to understand a broad range of complex information quickly and making decisions where opinions differ/no obvious solution Planning & Organisation Skills Essential criteria • Experience of the successful, implementation, use and maintenance of data warehouse solutions to support an organisation • Experience of setting up and implementing internal processes and procedures. • Able to influence junior staff team members -setting, communicating and demonstrating clear standards and supporting personal development. • Able to plan and coordinate the work of self and others to achieve agreed team objectives and meet tight and often conflicting deadlines. Desirable criteria • Evidence (using agile & other methods) of planning and delivering programmes and projects and services on time