Join our dynamic team in Great River, where innovation meets opportunity. We are seeking an experienced professional with a passion for big data and distributed computing. If you are a proactive individual with a strong background in PySpark and Apache Spark, this role is for you. Key Responsibilities • Lead the design, development, and deployment of PySpark-based big data solutions. • Architect and optimize ETL pipelines for both structured and unstructured data. • Collaborate with data engineers, data scientists, and business teams to understand requirements and deliver scalable solutions. • Optimize Spark performance through partitioning, caching, and tuning. • Implement best practices in data engineering, including CI/CD, version control, and unit testing. • Work with cloud platforms like AWS to enhance data solutions. • Ensure data security, governance, and compliance. • Mentor junior developers and review code for best practices and efficiency. Qualifications • 10 years of experience in big data and distributed computing. • Strong hands-on experience with PySpark, Apache Spark, and Python. • Proficient in SQL and NoSQL databases such as DB2, PostgreSQL, and Snowflake. • Experience in data modeling and ETL workflows. • Proficiency with workflow schedulers like Airflow. • Hands-on experience with AWS cloud-based data platforms. • Experience in DevOps, CI/CD pipelines, and containerization (Docker, Kubernetes) is a plus. • Strong problem-solving skills and ability to lead a team. • Experience with DBT and AWS Astronomer is advantageous. We are committed to fostering a diverse and inclusive workplace. We welcome applications from all qualified individuals, regardless of gender, race, or background. For immediate consideration please click APPLY or contact me directly at priyanka@thegreatriverfinancial.com