AWS Data Engineer - SC Cleared

AWS Data Engineer - SC cleared - remote - 12 months - UK Key Responsibilities • Develop, maintain, and optimize ETL pipelines using AWS Glue (Informatica will be beneficial) • Build and manage cloud-based data pipelines leveraging AWS services (eg, EMR, S3, Lambda, Glue). • Implement scalable data processing workflows using Databricks, PySpark, Python, and SQL. • Design and support data ingestion, transformation, and integration processes across structured and unstructured data sources. • Collaborate with data architects, analysts, and business stakeholders to understand requirements and deliver reliable data solutions. • Monitor pipeline performance, troubleshoot issues, and ensure data quality and reliability. • Contribute to best practices for data engineering, including version control, CI/CD, and automation. Required Skills & Qualifications • Strong hands-on experience with ETL development and orchestration (AWS). • Solid AWS cloud experience, including working with core data services. • Expertise in building distributed data pipelines using EMR, PySpark, or similar technologies. • Strong data processing and transformation experience across large datasets. • Proficiency in PySpark, Python, and SQL for data manipulation and automation. • Understanding of data modelling, data warehousing concepts, and performance optimization. • Familiarity with CI/CD tools (DevOps, GitHub, GitLab). • Exposure to data governance, metadata management, and data quality frameworks. • Experience working in Agile environments is a plus.