Role : Data Engineer Location: Frisco, TX (100% Onsite) Mode : Contract (6 months) Mandate : AWS , S3 , Pyspark , python, end to end snowflake , Streamset pipeline Job Overview: We are seeking a highly skilled Lead Data Engineer to design, build, and maintain scalable data solutions on the Snowflake platform. The ideal candidate will have strong hands-on experience with StreamSets for data ingestion, expertise in Snowflake (including stored procedures, SnowSQL, Snowpark and job scheduling), and proficiency in SQL and Python (especially PySpark). Experience with performance tuning, end-to-end solution design, and data migration projects is essential. Key Responsibilities: • Design and Develop Data Pipelines: Build and maintain robust, scalable StreamSets jobs to ingest and load data into Snowflake databases from various sources. • Snowflake Development: Develop, optimize, and maintain Snowflake stored procedures, functions, and scripts using SnowSQL. • Job Scheduling: Set up and manage job scheduling within Snowflake for automated data loads and other processes. • Performance Tuning: Analyze, optimize, and tune data pipelines and Snowflake queries for optimal performance. • Solution Architecture: Design end-to-end data solutions, ensuring security, scalability, and best practices are followed across all data flows. • Data Migration: Plan and execute complex data migration projects, including transformation and validation, from legacy or external systems to Snowflake. • Programming: Write clean, efficient, and reusable SQL and Python code, with a focus on data processing using PySpark and Snow park. • Collaboration: Work closely with data architects, analysts, and other engineers to understand business requirements and translate them into technical solutions. • Documentation: Prepare and maintain clear documentation for all processes, pipelines, and architecture. Required Skills & Qualifications: • Bachelor’s or Master’s degree in Computer Science, Engineering, Information Systems, or a related field. • 10 to 15 years of experience in data engineering roles • Hands-on expertise in building StreamSets pipelines for data ingestion and transformation. • Advanced knowledge of Snowflake, including stored procedures, user-defined functions, and SnowSQL scripting. • Strong experience in job scheduling and orchestration within Snowflake. • Deep proficiency in SQL for complex queries, data transformations, and performance tuning. • Strong programming skills in Python, especially in building and optimizing data pipelines with PySpark. • End-to-end solution design and architecture experience on Snowflake. • Proven experience in large-scale data migration projects. • Excellent problem-solving, communication, and leadership skills. Preferred Qualifications: • Experience with cloud platforms (AWS) and their integration with Snowflake. • Familiarity with other ETL tools and orchestration frameworks. • Knowledge of data security, governance, and compliance standards. Thanks Harinath M Harinathm@vbeyond.com