Cloud Specialist

Senior DevOps / Data / Infrastructure / Web Engineer London Hybrid Long term contract. Reports to: Head of Engineering / Head of Data We are seeking a versatile Senior DevOps/Data/Infrastructure/Web Engineer to join a small, fast-moving engineering team supporting a data-centric product. This role is central to operating and improving our cloud infrastructure, maintaining data pipelines and analytical tooling, and building lightweight web visualizations. The successful candidate will be comfortable across infrastructure-as-code, backend scripts and SQL, container orchestration, and front-end visualization. This position will work on a confidential project; the employer identity will be shared with shortlisted candidates. Key responsibilities: Infrastructure & Cloud • Maintain and extend AWS infrastructure using AWS CDK and Projen (TypeScript). • Enforce Git-first workflows and source code quality standards. • Harden security for AWS resources and for workloads running in EKS. • Manage user access and IAM policies. Apache Superset & Analytics Tooling • Configure, optimize and operate Apache Superset instances in AWS. • Maintain and extend in-house Python tools that integrate with Superset. • Read and modify Superset source code when documentation is insufficient. Data Pipelines • Operate and improve a Python-based Elexon/Iris data grabber running in EKS; add new data grabbers as needed. • Implement robust monitoring, logging and alerting for data grabbers and pipeline components. • Automate data processing using PostgreSQL, dbt, and GitHub Actions. Database Operations • Monitor and scale AWS Aurora (PostgreSQL) clusters; implement partitioning strategies (pg\_partman) and other maintenance tasks. • Optimize large SQL queries — refactor queries and apply appropriate indexes. Web & Visualization • Implement automatic exports from Superset/Postgres to static JSON files for website consumption. • Build static analytical charts with ECharts (or similar) embedded in the company website. • Integrate SSO for Superset using Microsoft Active Directory / Azure AD or AWS AD Connector. Containers & Orchestration • Manage Docker images for arm64 and amd64; maintain images on AWS ECR. • Operate Kubernetes clusters in EKS (and occasionally k3s). Use tools such as kubectl, Lens, k9s, Helm. Networking & Security • Maintain site-to-site and client VPNs (WireGuard). • Implement and maintain monitoring, IAM, secrets management and secure networking best practices. Required skills & qualifications • Linux: Expert user and admin-level experience. • AWS: AWS Certified Solutions Architect (SAA-C03) or equivalent hands-on experience. • Experience with AWS CDK and Projen (TypeScript). • TypeScript / JavaScript: Advanced. Node.js, React experience required. • Familiarity with Apache ECharts (or equivalent visualization libs). • Python: Advanced. Comfortable with Jinja templates and environment management (conda/pip). • Databases / SQL: Strong SQL skills. Experience with PostgreSQL (AWS RDS/Aurora), PL/pgSQL; partitioning and indexing. • Data tooling: Practical experience with dbt and CI automation (GitHub Actions). • Containers / Kubernetes: Strong Docker experience (Alpine, Ubuntu images); Kubernetes expertise (EKS). Experience building multi-arch images (arm64 + amd64). • Monitoring & Observability: Experience with logging, metrics and alerting for production workloads. • Apache Superset: Able to read and modify Superset source and maintain custom Python tooling. • Networking: Experience with WireGuard or similar VPN tech. • Excellent troubleshooting skills and capacity to work across the stack. Nice-to-haves • Kubernetes certification (CKA/CKAD). • Experience with MS SQL Server. • Prior experience with energy-sector datasets (Elexon/Iris) or time-series utilities is a plus.