Big Data Engineer

Overview: Join our dynamic team as a Big Data Engineer, where you'll leverage cutting-edge technologies to design, develop, and optimize large-scale data processing systems. This thrilling role will have you collaborating with cross-functional teams to architect data pipelines and implement integration solutions that drive decision-making and enhance operational efficiency. As a cornerstone of our operations, your expertise in big data technologies and distributed systems will help us harness the power of our data for business transformation. Required Skills • Big Data Technologies: Proficient in Hadoop, Spark, Hive, and Trino. • Container Orchestration & Kubernetes: Strong experience managing Kubernetes architecture and running Spark workloads on Amazon EMR on EKS. • Programming Languages: Proficiency in Python or Scala with a focus on modular and performant code. • SQL Skills: Expertise in window functions, multi-table joins, and writing complex queries. • Cloud Technologies: Familiarity with AWS services such as S3, Glue, Lambda, and CloudWatch. Nice To Have Skills • Experience with CI/CD pipelines (Jenkins, GitHub Actions). • Knowledge of Infrastructure as Code (Terraform, CloudFormation). • Familiarity with monitoring tools like Prometheus and Grafana. • AWS certifications (e.g., AI Practitioner, Solutions Architect). Preferred Education And Experience • Bachelor’s degree in Computer Science, Information Systems, or a related field, with at least five (5) years of relevant experience. A Master's degree and experience in the Financial Services industry are preferred. Other Requirements • Strong written and verbal communication skills. • Ability to work effectively in fast-paced environments. • Willingness to remain current with emerging technologies and best practices in the industry.