AWS Data Engineer

Join our team as an AWS Data Engineer and contribute to building scalable data solutions using advanced AWS-native platforms. Collaborate with a dynamic team dedicated to delivering innovative data architecture and engineering practices. Responsibilities • Design and implement efficient data storage and retrieval systems within AWS Lakehouse architectures. • Develop high-quality, production-grade data pipelines utilizing advanced Python programming. • Work with cross-functional teams to establish data architecture and best practices. • Implement Medallion Architecture for effective data processing and management. • Manage and ensure data integrity through open table formats and versioning strategies. • Maintain documentation and uphold high coding standards throughout the development process. Skills • Proficiency in Python, particularly for data engineering. • Experience with AWS services like S3, Glue, Athena, and Lake Formation. • Strong understanding of data architecture principles and best practices. • Experience implementing Slowly Changing Dimensions (SCD Type 2) in distributed systems. Preferred Skills • Familiarity with Apache Iceberg, Delta Lake, or Apache Hudi for managing open table formats. • Experience with orchestration tools such as EMR on EKS or Argo Workflows. • Knowledge of data visualization tools like AWS Quicksights, Tableau, or Cognos. At Horizontal, we foster an inclusive workplace where diversity, equity, and inclusion are fundamental principles. We welcome applicants from all backgrounds to join our team. By applying for this position, you acknowledge and agree that Horizontal Talent may contact you regarding your application using automated technology, including phone calls, SMS/text messages, or email, which may be delivered by our virtual AI recruiter, Alex.