Data Engineer

Senior Data Engineer (Python Enterprise Developer) – London, UK Location: London (Canary Wharf – Hybrid, 4 days onsite) Contract Duration: 6 – 12 Months Experience Required: 8+ Years Preferred Domain: Energy Trading Job Description We are looking for a Senior Data Engineer / Python Enterprise Developer with strong expertise in Python-based data engineering, ETL development, cloud platforms, and enterprise-grade data solutions. The ideal candidate should have deep experience in building scalable data pipelines, web scraping frameworks, SQL optimization, and cloud-native workflows using AWS and Azure technologies. Responsibilities Design, develop, and optimize scalable ETL/data pipelines. Build enterprise-grade Python applications and automation frameworks. Develop scraping and processing solutions using Python libraries such as Pandas, NumPy, Polars, BeautifulSoup, Selenium, Requests, and pdfplumber. Work with PostgreSQL and optimize complex SQL queries. Develop and orchestrate workflows using Apache Airflow. Implement CI/CD pipelines using Azure DevOps, Jenkins, and Git. Work with AWS services including S3 and Lambda. Develop cloud-based solutions using Azure Databricks. Collaborate with cross-functional teams in Agile/Scrum environments. Coordinate with global teams across multiple time zones. Mandatory Skills Strong Python scripting and enterprise application development experience PostgreSQL / SQL expertise Azure Databricks AWS (S3/Lambda) Apache Airflow CI/CD tools (Azure DevOps, Jenkins) Git version control ETL/Data Engineering experience