Data Engineer

JOB DESCRIPTION: Minimum 4 years of hand on experience with • Building data pipeline using big-data stack (Hadoop, Hive, pySpark, python) • Amazon AWS S3 – Object storage, security, data service integration with S3 • Data modelling and database design. • Job Scheduler – Autosys • PowerBI, Dremio • Unix/shell scripting, CICD pipeline • Exposure in GCP cloud data engineering is a plus Manager Notes: -The contractors need to be proactive, they can't wait to be told what to do -Must be accountable along with the technical skills -The tech stack mentioned, these are the technologies being used to build data pipelines -They need to model, design the data, build pipelines, applying logic to the data to transform the data and troubleshoot -They should have strong understanding and implementation of Autosys -Ability to automate using spark, Python, Hadoop/Hive -Should have a fundamental background in database design (MySQL or any standard database) -Exposure to Cloud data engineering is a big plus, not required -Financial services experience is a plus but not required-having domain knowledge is helpful Technical Assessment -We need a clear understanding of tech work experience, they need to be able to describe the work they have done -Overall problem solving, so given a problem how efficiently does their thought process drive towards a solution?