Snowflake Data Engineer - ON SITE

Company and Role: Dedicated Tech Services, Inc. (DTS) is an award-winning IT consulting firm based in Columbus, OH. We now have an opening for a Snowflake Data Engineer - for critical ODM Enterprise Data warehouse. EDW M&O to migrate the production data (Weekly, Monthly, and yearly) from the current Big Data Environment to Snowflake Environment, run ELT jobs, and check the data quality from disparate data sources in Snowflake Environment to achieve ODM's strategic and long-term business goals. Highlights and Benefits: • Onsite – Columbus, Ohio • W2 hourly pay rate up to $70hr or salaried equivalent • Government Environment • Direct W2 hourly or salaried applicants only (no corp-to-corp subcontractors, third parties, or agencies) • Paid time off and holidays for salaried employees • 401K, billable bonus, and health, life, vision, dental and short-term disability insurance options for all • DTS is a proud Women Business Enterprise (WBE) and Woman Owned Small Business (WOSB)! • Check out our benefits and company information at www.dtsdelivers.com! Job Description: We are hiring an experienced Snowflake Data Engineer to work for us as our direct, W2 salaried or hourly employee to join our team. You will: • Participate in Team activities, Design discussions, Stand up meetings and planning Review with team. • Provide Snowflake database technical support in developing reliable, efficient, and scalable solutions for various projects on Snowflake. • Ingest the existing data, framework and programs from ODM EDW IOP Big data environment to the ODM EDW Snowflake environment using the best practices. • Design and develop Snowpark features in Python, understand the requirements and iterate. • Interface with the open-source community and contribute to Snowflake’s open-source libraries including Snowpark Python and the Snowflake Python Connector. • Create, monitor, and maintain role-based access controls, Virtual warehouses, Tasks, Snow pipe, Streams on Snowflake databases to support different use cases. • Performance tuning of Snowflake queries and procedures. Recommending and documenting the best practices of Snowflake. • Explore the new capabilities of Snowflake, perform POC and implement them based on business requirements. • Responsible for creating and maintaining the Snowflake technical documentation, ensuring compliance with data governance and security policies. • Implement Snowflake user /query log analysis, History capture, and user email alert configuration. • Enable data governance in Snowflake, including row/column-level data security using secure views and dynamic data masking features. • Perform data analysis, data profiling, data quality and data ingestion in various layers using big data/Hadoop/Hive/Impala queries, PySpark programs and UNIX shell scripts. • Follow the organization coding standard document, Create mappings, sessions and workflows as per the mapping specification document. • Perform Gap and impact analysis of ETL and IOP jobs for the new requirement and enhancements. • Create mockup data, perform Unit testing and capture the result sets against the jobs developed in lower environment. • Updating the production support Run book, Control M schedule document as per the production release. • Create and update design documents, provide detail description about workflows after every production release. • Continuously monitor the production data loads, fix the issues, update the tracker document with the issues, Identify the performance issues. • Performance tuning long running ETL/ELT jobs by creating partitions, enabling full load and other standard approaches. • Perform Quality assurance check, Reconciliation post data loads and communicate to vendor for receiving fixed data. • Participate in ETL/ELT code review and design re-usable frameworks. • Create Change requests, workplan, Test results, BCAB checklist documents for the code deployment to production environment and perform the code validation post deployment. • Work with Snowflake Admin, Hadoop Admin, ETL and SAS admin teams for code deployments and health checks. • Create re-usable framework for Audit Balance Control to capture Reconciliation, mapping parameters and variables, serves as single point of reference for workflows. • Create Snowpark and PySpark programs to ingest historical and incremental data. • Create SQOOP scripts to ingest historical data from EDW oracle database to Hadoop IOP, created HIVE tables and Impala views creation scripts for Dimension tables. • Participate in meetings to continuously upgrade the Functional and technical expertise. Required Skills and Experience: • Required Education: BS/BA degree or equivalent experience. • 9 years of experience with Big Data, Hadoop on Data Warehousing or Data Integration projects. • 9 years experience in Analysis, Design, development, support, and enhancements of ETL/ELT with Cloudera Big Data Technologies (Hadoop, MapReduce, Sqoop, PySpark, Hive, Impala, etc.) • 9 years of strong development experience in creating Sqoop scripts, PySpark programs, HDFS commands, StreamSets pipelines, Hive/Impala queries, and Unix shell scripts. • 9 years writing Hadoop/Hive/Impala scripts for gathering table stats post data load. • 6 years of hands-on experience with cloud databases. • 3 years of hands-on data migration experience from Big Data to Snowflake. • 3 years of hands-on experience with the Snowflake platform, including Snowpipe and Snowpark. • Proficiency in Data Warehousing, Data Migration, and Snowflake (Essential). • Strong experience in implementing, executing, and maintaining Data Integration solutions. • Strong SQL experience (Oracle and Hadoop – Hive/Impala). • Expertise implementing complex ETL/ELT logic. • Strong experience with Snow SQL, PL/SQL, and writing Snowflake procedures in SQL/Python/Java. • Experience with Snowflake performance optimization and monitoring. • Experience building datasets; familiarity with PHI and PII data. • Strong database architecture, problem-solving, and critical thinking. • Experience with the AWS platform. • Writing complex SQL queries and performance tuning (Hive/Impala explain plans). • Accountable for ETL/ELT design documentation. • Utilize ETL/ELT standards and centralized metadata practices. • Effective communication, presentation, and organizational skills. • Good experience with Visio, Excel, PowerPoint, Word. • Familiar with project methodologies: Agile and Waterfall. • Ability to prioritize and work independently with minimal supervision. • Basic knowledge of UNIX/Linux shell scripting. • Required excellent written and oral communication skills with technical and business teams. Desired Skills and Experience: • Snowflake certification (highly desirable). • Experience with Snowpark using Python (preferred). • Experience in the development work in both the Snowpipe and Snowpark. • Experience with Data Migration from Big data environment to Snowflake environment. • Strong understanding of Snowflake capabilities like Snowpipe, STREAMS, TASKS etc. • Knowledge of security (SAML, SCIM, OAuth, OpenID, Kerberos, Policies, entitlements etc.). • Has experience with System DRP for Snowflake systems • Demonstrate effective leadership, analytical and problem-solving skills. • Ability to work independently, as well as part of a team. • Stay abreast of current technologies in area of IT assigned. • Establish facts and draw valid conclusions. • Recognize patterns and opportunities for improvement throughout the entire organization. • Ability to discern critical from minor problems and innovate new solutions. US Citizens and those authorized to work in the US are encouraged to apply. We are unable to sponsor at this time. Dedicated Tech Services, Inc. is an Equal Opportunity Employer