Job Title: Data Engineer Location: Fort Mill, SC (Hybrid) Position Type: Contract Must Have Skills: AWS Glue, Snowflake, DBT, Good to Have: Lambda, Airflow, Fivetran, HVR Job Description: Responsible for designing, developing, and maintaining scalable data pipelines and ETL processes using AWS Glue. This role involves working closely with data engineers, analysts, and other IT professionals to ensure data is efficiently integrated, transformed, and made available for business use. 1. AWS services (Airflow): The candidate should have hands-on experience with AWS services, particularly Apache Airflow for orchestrating complex data workflows and pipelines. 1. AWS services (Lambda): Proficiency in AWS Lambda for serverless computing and event-driven architecture is essential for this role. 1. AWS services (Glue): The candidate should be well-versed in AWS Glue for ETL (Extract, Transform, Load) processes and data integration. 1. Snowflake, Snowpark: The candidate should have a deep understanding of Snowflake data warehousing platform and be proficient in using Snowpark for data processing and analytics. 1. DBT: Experience with DBT (Data Build Tool) for modeling data and creating data transformation pipelines is a plus. 1. Fivetran (HVR); Working knowledge and handson experience on Fivetran HVR. 1. Python: Strong programming skills in Python are required for developing data pipelines, data transformations, and automation tasks. Responsibilities: • Design, develop, and maintain data pipelines and ETL processes using Snowflake, AWS services, Python, and DBT. • Collaborate with data scientists and analysts to understand data requirements and implement solutions. • Optimize data workflows for performance, scalability, and reliability. • Troubleshoot and resolve data-related issues in a timely manner. • Stay updated on the latest technologies and best practices in data engineering. • Design and implement scalable ETL solutions using AWS Glue • Collaborate with data engineers and analysts to understand data requirements • Develop and manage data extraction, transformation, and loading processes • Optimize and improve existing data pipelines and ETL workflows • Ensure data quality and integrity during the transformation process • Monitor and troubleshoot issues with ETL jobs • Maintain detailed documentation of data workflows and processes • Stay updated with the latest AWS services and tools Qualifications: • Bachelor's degree in Computer Science, Engineering, or related field. • Proven experience in data engineering roles with a focus on Snowflake, AWS services, Python, and DBT. • Strong analytical and problem-solving skills. • Excellent communication and teamwork abilities. • Technical certifications are a plus. • Bachelor's degree in Computer Science, Information Technology, or a related field • 3+ years of experience working with AWS Glue and other AWS data services • Proven experience in designing and maintaining ETL processes • Strong knowledge of SQL and database management • Familiarity with data warehousing concepts and tools • Experience with Python or other scripting languages • Excellent problem-solving and analytical skills • Strong communication and teamwork abilities If you believe you are qualified for this position and are currently in the job market or interested in making a change, please email me the resume along with contact details at roshni@nytpcorp.com