Note • This is a W2 position. • It is a 100% on-site role based in Owings Mills, MD. • Candidates must have strong hands-on experience in PySpark, Apache Spark, Python, and AWS along with data engineering, cloud architecture, and ETL pipeline development. • Visa sponsorship and relocation assistance are not available. • Work Type: Full-time | On-site • Contract Duration: 6 Months • Notice Period: Immediate Joiner / Up to 30 Days Preferred Job Summary • We are hiring on behalf of a global IT services provider for a Lead PySpark Developer to build, optimize, and scale big data solutions using Apache Spark and AWS. The role will lead the design and deployment of ETL pipelines, collaborate with cross-functional teams, and drive cloud data platform initiatives. Key Responsibilities • Design and lead PySpark-based big data applications and data pipelines • Architect scalable ETL workflows for structured/unstructured data • Optimize Spark jobs for performance, caching, and partitioning • Collaborate with data engineers, scientists, and business users • Work with AWS services and cloud-native data platforms • Apply DevOps and CI/CD practices to data engineering workflows • Mentor junior developers, conduct code reviews, and enforce standards • Ensure data governance, security, and compliance Key Requirements • Bachelor’s degree in Computer Science or equivalent • 10+ years of experience in big data and distributed systems • 7+ years of AWS cloud computing experience • Expert in PySpark, Apache Spark, and Python • Proficient in SQL and NoSQL (DB2, PostgreSQL, Snowflake) • Experience with workflow tools like Airflow, and platforms like DBT • Exposure to CI/CD, Docker, Kubernetes, and DevOps pipelines • Excellent problem-solving and leadership skills • Strong communication and team collaboration abilities