Data Engineer (Modern Stack — Python/PySpark/AWS)

Job Description We are seeking a highly motivated and experienced Data Engineer to join our team, focusing on building, optimizing, and deploying robust, scalable data solutions. The ideal candidate will possess deep expertise in Python and PySpark to drive complex data transformations and support high-volume, performance-critical data initiatives. Key Responsibilities • Design, build, and maintain high-performance ETL/ELT data pipelines using Python and PySpark • Apply expertise in Python data analysis libraries including Pandas and NumPy • Develop and manage data processing jobs leveraging PySpark for distributed computing across large-scale datasets • Implement DevOps practices and tooling for automated deployment and orchestration of Python applications • Utilize AWS or other cloud services to architect and maintain cloud-based data ecosystems • Write and optimize complex SQL queries for data extraction, integrity checks, and performance tuning • Collaborate with data scientists and analysts to ensure data quality, availability, and consistency Required Technical Skills • 5+ years of experience in Data Engineering • Expert-level proficiency in Python — Pandas, NumPy • Solid hands-on experience with PySpark for building scalable data workflows • AWS — S3, Glue, EMR, Snowflake, Lambda strongly preferred • Docker, Kubernetes, Terraform, CloudFormation • Advanced SQL and data warehousing knowledge • Apache Spark, Hadoop ecosystem • CI/CD tools — Jenkins, Git, Artifactory/Nexus • Strong knowledge of shell scripting and file systems Nice to Have • Healthcare or financial services domain experience • NoSQL databases — HBASE, DynamoDB • Microservices and service-oriented architecture • OpenShift, container orchestration platforms