Data Engineer - Outside IR35 (Scotland)

Gigged AI is an On demand talent marketplace specialising in the IT and technology sector. We have an opportunity live at the moment for a Data Engineer - Outside IR35 his is a Hybrid role in Scotland, flexible on onsite presence. Data Engineer Location: Scotland, occasional onsite presence Duration: Estimated 3-6 months, TBC IR35: Outside All proposals will be reviewed on the Gigged AI platform. In order to send a proposal please head to app.gigged.ai and create a free talent profile and apply to the role directly All applicant proposals will be reviewed directly on the Gigged AI platform by the hiring manager. • In order to send a proposal please follow the below steps • head to app.gigged.ai and create a free talent profile • once signed in, follow this link to the project - https://app.gigged.ai/find-a-gig/m4dGm4YDK52 • you will now see a 'submit proposal' button • please fill this in (intro, relevant experience, and attach CV to this) Overview Our client operates a cloud-based data platform that brings together trusted business information from a range of internal systems and applications. As the platform's capabilities continue to expand, they are looking for an experienced Data Engineer to build the next generation of data pipelines, covering data provisioning, modelling, performance tuning and optimisation. The role sits between IT development and business end-users, ensuring core datasets remain available, accurate and accessible, while also working with data and system owners to define access, security and data management processes. The ideal candidate can take a data requirement end-to-end: examining a source data structure and building the extraction, storage and pipeline work needed to deliver it. Key Responsibilities • Design, build and maintain data pipelines from internal databases and SaaS applications into the client's cloud data platform • Continuously develop and enhance the platform's data provisioning, modelling and performance capabilities • Produce and maintain clear architecture and systems documentation • Partner with business functions and other IT teams to scope data needs and deliver solutions • Provide ongoing technical support, monitoring and troubleshooting for the data platform • Act as a technical point of reference, sharing expertise and best practice across the team Required Skills • Degree qualified in a STEM discipline (or equivalent professional experience) • Strong proficiency in a modern programming language (Python, Java, JavaScript, Scala, etc.) • Strong SQL skills for querying and analysing structured data stores • Proven experience designing, developing and deploying data pipelines • Experience with a modern public cloud data platform (Azure, AWS or GCP) • Experience with version control tooling • Experience performing root-cause analysis of production incidents and debugging code defects • Strong interpersonal and stakeholder-management skills Desirable Skills • Experience with Big Data toolkits and NoSQL technologies • Experience with modern Business Intelligence toolkits • Experience designing and documenting data models • Experience optimising data processing workloads • Experience with message queuing and stream processing • Knowledge of modern data warehouse architectural principles • Knowledge of DevOps workflows and principles • Knowledge of Machine Learning/MLOps workflows and principles