Big Data Developer

The Big Data Developer will play a critical role on the Big Data engineering team, designing and implementing large-scale data processing systems that power scientific research and innovation. The ideal candidate has hands-on experience building enterprise-grade data pipelines, working with distributed systems, and optimizing data processing workflows. This is a long-term project (12+ months) with high visibility, cutting-edge tools, and opportunities to influence technical direction. • • U.S. CITIZENSHIP OR PERMANENT RESIDENCY REQUIRED. VISA OR EAD STATUS NOT ACCEPTABLE FOR THIS POSITION. NO EXCEPTIONS • • What You’ll Do ✔ Data Pipeline Design & Development • Design, build, and deploy scalable data pipelines for ingesting, processing, transforming, and storing high-volume datasets. • Implement streaming and batch-processing solutions using Hadoop, Spark, and cloud-based tools. ✔ Data Architecture & Engineering • Develop and maintain data architecture and data flow models. • Ensure data reliability, accuracy, and integrity across all environments. • Support data warehousing strategies and best practices. ✔ Data Quality, Security & Compliance • Implement automated data validation, error handling, and monitoring. • Ensure compliance with internal security controls and regulatory standards. • Partner with governance teams to enforce data quality and security guidelines. ✔ Cross-Functional Collaboration • Work closely with data scientists, analysts, product teams, and application developers. • Translate business requirements into robust technical solutions. • Participate in Agile ceremonies and contribute to technical design discussions. ✔ Performance Optimization • Tune Spark applications, Hadoop jobs, and distributed data systems for performance and cost efficiency. • Troubleshoot bottlenecks and implement improvements to system performance. ✔ Technical Leadership • Provide mentorship to junior developers and contribute to coding standards, best practices, and technical documentation. Required Skills & Qualifications • 4+ years of Big Data Development experience in Hadoop ecosystems • 2+ years of hands-on development with Apache Spark • Proficiency in Java, Scala, or Python • Strong understanding of distributed systems, ETL, data warehousing, and data modeling concepts • Experience with large-scale datasets, performance tuning, and troubleshooting • Strong problem-solving, communication, and collaboration skills • Bachelor’s degree in Computer Science, Engineering, or related discipline Preferred Skills • Experience working with AWS cloud services (EMR, S3, Lambda, Glue, etc.) • Experience with Spark 3.x or 4.x • Exposure to Kubernetes, Airflow, or similar orchestration tools • Familiarity with CI/CD and DevOps automation for data engineering Why This Opportunity Stands Out • Long-term project stability (12+ months, likely extension) • Ability to work on high-impact scientific and research-driven datasets • Hands-on cloud modernization (AWS) and next-generation big data tooling • Collaborative and innovative engineering culture