Job Description • 7+ years of experience in Amazon Web Service(AWS) Cloud Computing. • 10+ years of experience in big data and distributed computing. • Very Strong hands-on experience with PySpark, Apache Spark, and Python. • Strong Hands on experience with SQL and NoSQL databases (DB2, PostgreSQL, Snowflake, etc.). • Proficiency in data modeling and ETL workflows. • Proficiency with workflow schedulers like Airflow. • Hands on experience with AWS cloud-based data platforms. • Experience in DevOps, CI/CD pipelines, and containerization (Docker, Kubernetes) is a plus. • Strong problem-solving skills and ability to lead a team • DBT, AWS Astronomer • Lead the design, development, and deployment of PySpark-based big data solutions. • Architect and optimize ETL pipelines for structured and unstructured data. • Collaborate with Client, data engineers, data scientists, and business teams to understand requirements and provide scalable solutions. • Optimize Spark performance through partitioning, caching, and tuning. • Implement best practices in data engineering (CI/CD, version control, unit testing). • Work with cloud platforms like AWS. • Ensure data security, governance, and compliance. • Mentor junior developers and review code for best practices and efficiency. Must Have • 7+ years of experience in Amazon Web Service(AWS) Cloud Computing. • 10+ years of experience in big data and distributed computing. • Experience with PySpark, Apache Spark, and Python. • Experience with SQL and NoSQL databases (DB2, PostgreSQL, Snowflake, etc.). • Hands on experience with AWS cloud-based data platforms. • Experience in DevOps, CI/CD pipelines, and containerization (Docker, Kubernetes) is a plus. Skills: cloud computing,nosql,cd,pyspark developer,python,pyspark,ci/cd,apache spark,big data,airflow,etl workflows,sql,snowflake,ci,kubernetes,aws,postgresql,aws astronomer,distributed computing,docker,db2,devops,data modeling,dbt,aws cloud computing