Data Engineer

Contract position with a major investment firm. Hybrid 3 days/week in Newark, NJ. Environment includes several Informatica products - data catalogue, market place, data lineage, data quality. As well as Denodo and Ataccama. Any experience with any of these tools are a big plus. Qualifications: • Bachelor's degree in Computer Science, Software Engineering, MIS or equivalent combination of education and experience • 8+ years of experience as Data Engineer on AWS Stack with experience on DevOps tool • AWS Solutions Architect or AWS Developer Certification required • Solid experience of AWS services such as CloudFormation, S3, Athena, Glue, Glue DataBrew, EMR/Spark, RDS, Redshift, DataSync, DMS, DynamoDB, Lambda, Step Functions, IAM, KMS, SM, EventBridge, EC2, SQS, SNS, LakeFormation, CloudWatch, Cloud Trail • Implement high velocity streaming solutions and orchestration using Amazon Kinesis, AWS Managed Airflow and AWS Managed Kafka (preferred) • Solid experience building solutions on AWS data lake/data warehouse • Analyze, design, development, and implementation of data ingestion pipeline in AWS • Knowledge implementing ETL/ELT for data solutions • End-to-end data solutions (ingest, storage, integration, processing, access) on AWS • Knowledge implementing RBAC strategy/solutions using AWS IAM and Redshift RBAC model • Build & implement CI/CD pipelines for EDP Platform using CloudFormation and Jenkins • Programming experience with Python, Shell scripting and SQL • Knowledge of analyzing data using SQL Stored procedures • Build automated data pipelines to ingest data from relational database systems, file systems, NAS shares to AWS relational databases such as Amazon RDS, Aurora, and Redshift • Build Automated data pipelines to ingest data from Rest APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift. • Good Experience of DevOps practice • Experience with modern source code management and software repository systems (Bitbucket) • Experience with programming languages (Python) • Experience with scripting languages (Shell, Groovy) • Experience with API deployment for tooling integration • Experience using Jenkins, CloudBees (Pipeline as Code, Shared Libraries) • The ability to document exceptions/issues/action plans/meeting minutes/lessons learned accurately and in a timely fashion • Experience with administering DevOps tools in SaaS • Experience using DevOps Tools (SonarQube, Artifactory etc.) • Experience using build tools (Maven, MS Build and Gradle) • Experience using containers (Docker) • Experience using Atlassian suite (Jira, Confluence) • Experience of Infrastructure as Code using CloudFormation • Creating Jenkins CI pipelines to integrate Sonar/Security scans and test automation scripts • Part of DevOps QA and AWS team focusing on building CI/CD pipeline • Responsible for writing and maintaining Jenkins Pipelines Responsibilities: • Designing, building, and maintaining efficient, reusable, and reliable code • Ensure the best possible performance and quality of high scale data applications and services • Participate in system design discussions • Independently perform hands on development and unit testing of the applications • Collaborate with the development team and build individual components into enterprise data platform • Work in a team environment with product, QE/QA, and cross functional teams to deliver a project throughout the whole software development cycle. • Responsible to identify and resolve any performance issues • Keep up to date with new technology development and implementation • Participate in code review to make sure standards and best practices are met • Project management: Agile developers take responsibility for estimating, planning, and managing all tasks and report on progress • Software quality: The Agile developer is also responsible for the quality of the software he/she produces. The team takes responsibility for the quality of the work they produce instead of turning over some code to a separate and independent group for testing • Teamwork: This includes collaboration with all other team members with the aim to take shared responsibility for the overall efforts • Understanding user needs: This is about interacting with users as necessary to clarify requirements