Pyspark AWS Data Engineer

Job Title: AWS PySpark Data Engineer Location: Northampton (Hybrid) Job Type: Contract Industry: Banking / Financial Services Job Summary We are looking for an experienced AWS PySpark Data Engineer to join a high-performing data engineering team within a leading Banking client. The ideal candidate will have strong expertise in building scalable data pipelines on AWS, developing ETL solutions using PySpark and Python, and working with cloud-native data services. Experience in data warehousing, automation, and DevOps practices is highly desirable. The successful candidate will be responsible for designing, developing, optimizing, and maintaining enterprise-grade data platforms while ensuring high performance, security, and reliability. Key Responsibilities • Design, develop, and maintain scalable ETL/ELT pipelines using PySpark and Python. • Build and manage cloud-native data solutions on AWS. • Develop data ingestion, transformation, and orchestration workflows. • Work extensively with AWS data services to process large-scale datasets. • Create reusable and optimized data processing frameworks. • Develop SQL queries for data extraction, validation, and reporting. • Monitor and optimize ETL performance and troubleshoot production issues. • Collaborate with Solution Architects, Data Analysts, DevOps, and Business stakeholders. • Implement Infrastructure as Code (IaC) using CloudFormation. • Follow CI/CD best practices and version control processes. • Participate in code reviews and ensure coding standards are maintained. • Ensure data quality, governance, and security across the data platform. Must-Have Technical Skills AWS • Amazon S3 • AWS IAM • AWS Lambda • AWS Glue • AWS Step Functions • AWS Lake Formation • Amazon VPC & Subnets • Amazon Athena • Amazon Redshift • AWS Service Catalog • AWS CloudFormation • Amazon SNS • Amazon SQS Programming & Data Engineering • PySpark • Python • SQL • Data Pipeline Development • ETL/ELT Development Development Tools • Basic Unix Commands • Shell Scripting • Git Good-to-Have Skills • GitLab • Nexus Repository • Redshift Performance Optimization • Snowflake • dbt (Data Build Tool) Required Experience • 6+ years of experience in Data Engineering. • Strong hands-on expertise in PySpark, Python, and AWS. • Experience building scalable ETL pipelines and cloud-based data platforms. • Strong SQL development and query optimization skills. • Experience with Infrastructure as Code using CloudFormation. • Familiarity with Agile/Scrum methodologies. • Banking or Financial Services experience is highly desirable. Preferred Qualifications • Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline. • AWS Certification (Associate or Professional) is a plus. • Experience working in enterprise-scale cloud environments. Key Skills • AWS • PySpark • Python • SQL • S3 • Glue • Lambda • Athena • Redshift • Lake Formation • Step Functions • CloudFormation • IAM • VPC • SNS • SQS • Shell Scripting • Git • ETL • Data Engineering • Snowflake (Preferred) • dbt (Preferred) • GitLab (Preferred) Location: Northampton (Hybrid) Banking domain experience is highly preferred.