Lead PySpark Engineer

• • • • • CONTRACT W2 ROLE • • • • • • • • • • CONTRACT W2 ROLE • • • • • • • • • • CONTRACT W2 ROLE • • • • • Job Title: Lead PySpark Engineer Location: Owings Mills, MD (LOCAL CANDIDATES ONLY!) Rate: $50 /hr on W2 (Benefits) Independent Visa Candidates Only. JD: Role Name - Lead PySpark Engineer ROLE\_DESCRIPTION - • 10+ years of experience in big data and distributed computing. • Very Strong hands-on experience with PySpark, Apache Spark, and Python. • Strong Hands on experience with SQL and NoSQL databases (DB2, PostgreSQL, Snowflake, etc.). • Proficiency in data modeling and ETL workflows. • Proficiency with workflow schedulers like Airflow • Hands on experience with AWS cloud-based data platforms. • Experience in DevOps, CI/CD pipelines, and containerization (Docker, Kubernetes) is a plus. • Strong problem-solving skills and ability to lead a team • Lead the design, development, and deployment of PySpark-based big data solutions. • Architect and optimize ETL pipelines for structured and unstructured data. • Collaborate with Client, data engineers, data scientists, and business teams to understand requirements and provide scalable solutions. • Optimize Spark performance through partitioning, caching, and tuning. • Implement best practices in data engineering (CI/CD, version control, unit testing). • Work with cloud platforms like AWS • Ensure data security, governance, and compliance. • Mentor junior developers and review code for best practices and efficiency Note : If you are interested please share me your resumes to srinivas@vmcsofttech.com or else reach me at 4804076930.