Data Engineer

Data Pipeline Development • Design, build, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources into a centralized data platform. • Migrate current pipelines to AWS-Snowflake utilizing a range of cloud data technologies to automate data workflows and ensure data availability and quality. • Experience with AWS Glue, CDK, Python, enterprise version control (GIT based). ETL/ELT Processes • Implement and optimize Extract, Transform, and Load (ETL) and Extract, Load, and Transform (ELT) processes. • Develop data transformation logic to clean, enrich, and aggregate raw data, ensuring it meets business requirements and is sui for analysis and reporting. Data Warehouse Architecture • Architect, design, and manage enterprise-level data mesh, warehouses and aggregated data products (e.g., Amazon, Snowflake, MongoDB). • Define and enforce data governance standards, security policies, and best practices for data storage, access, and usage. Data Modeling • Create and maintain logical and physical data models (e.g., dimensional modeling, star schemas) that support business intelligence, analytics, and reporting needs. • Collaborate with business analysts and data scientists to understand data requirements and translate them into efficient data structures. Database Usage • Work with Relational Database Management Systems (RDBMS) like Oracle, SQL server, and cloud-based relational databases (e.g., Amazon RDS, Snowflake) to manage structured data. • Utilize Document Databases such as MongoDB for handling unstructured or semi-structured data, focusing on schema flexibility and scalability for specific use cases. Cloud-based Solutions • Leverage and integrate various cloud services on AWS, Snowflake, MongoDB to build and deploy data solutions. • Manage cloud pipelines, monitor performance and optimize costs associated with data storage and processing services with support from enterprise data team, Collaboration • Demonstrates a commitment to teamwork and fosters a collaborative work environment amongst engineers, infrastructure teams, and stakeholders. • Comfort with pair programming, peer reviews, and working in cross-functional teams. Mentorship • Provide technical leadership and mentorship to junior data engineers, fostering a culture of best practices and continuous learning. Analytical Thinking • Strong problem-solving skills and ability to analyze complex systems. Effective Communication • Strong verbal and written communication skills to collaborate with cross-functional teams, share ideas, and document processes. Diversity And Inclusion • Promotes a culture that values diverse ideas and opinions, listens courageously, and remains curious. Technology Exploration • Motivated to research and adopt new technologies through diligent investigation. Documentation • Capable of writing clear and comprehensive technical documentation. Agile Methodologies • Experienced in Agile environments, including using tools like JIRA and Confluence. Adaptability And Flexibility • Willingness to adapt to changing project requirements and priorities, and ability to work effectively in a fast-paced environment while maintaining focus on delivering quality work. Learning Agility • Proven ability to quickly learn new technologies, frameworks, and tools, and apply them effectively in projects. Autonomous Working Style • Capability to work independently, especially when requirements are vague, demonstrating initiative and resourcefulness. Services Description • Design and develop efficient and scalable data solutions and pipelines to meet the operational and analytical data requirements of the business. • Operates as a core team member with other data engineers on team within the Agile Scaled Framework within the IS Operation, supporting our Workplace Solutions business segment. (See Team Description below) • Collaborate with enterprise stakeholders on data management, architectural design, governance policies, and security measures to develop solutions that adhere to established standards, enhancing production environments and practices. • Work with business owners to improve data utilization and maximize information value within supported systems. • Develop, test and integrate new data features and functionality as defined by the product owners and business teams. • Implement data orchestration pipelines, data sourcing, cleansing, augmentation and quality control processes within supported data systems. • Lead the data pipeline development/migration, testing, and integration of modern applications and systems that enhance capabilities for products and services across Workplace solutions, aligning with enterprise growth initiatives. • Facilitate brainstorming sessions to generate innovative solutions, developing high-performance applications, APIs, and data solutions that drive technological advancement. • Adapt to various roles, whether as an individual contributor, leader, teacher, or mentor, fostering a collaborative and versatile team environment. • Build a deep understanding of business objectives and challenges within ESS, proactively delivering solutions that meet and exceed expectations. • Continuously assess and optimize applications, balancing performance and scalability to deliver impactful, sustainable results. Team Description We are seeking a high performer with some serious smarts, intense curiosity, and determination. This is a data team that builds and manages aggregated data products supporting our a group insurance small business unit. This team is currently modernizing our Informatica, Teradata pipelines to cloud data solutions (using AWS, Snowflake, MongoDB, Python). We are looking for experienced data engineers with database, data warehousing, ETL/ELT knowledge with experience designing, architecting and implementing data solutions in AWS and Snowflake. We also started our journey on Document database, and the experience will be a huge plus. Looking for someone who is passionate about data, modernization and enthusiastic about innovation. Deliverables Description (if Applicable) • Lead our tech modernization efforts to migrate current pipelines to cloud technologies (AWS, Snowflake and MongoDB). • Data modeling and warehousing on Snowflake • Work on assigned stories and deliver on commitments • Ensure NFRs are met and the data is accurate, available • Delivered code should be covered by automated integration/interaction/end-to-end tests following our standards. • Code and documentation should be consistent, clear, and concise and must be understandable, tes, and maintainable adhering to our defined standards. • Actively contribute to Agile team processes, including sprint planning, reviews, and retrospectives, fostering a collaborative environment where ideas can be shared freely, and feedback is welcomed for continuous improvement. • If hired permanent, will be responsible for providing timely and effective technical assistance during off-hours as part of on-call rotation. • Follow security policies, standards and expectations • Follow communication expectations and contingency plans • Leverage communication platforms (Microsoft Teams, Jira, email) to ask questions, raise issues, and share updates. • Escalate blockers with impact assessment immediately. • Proactive in identifying potential blockers or risks early on. • Follow schedule for live collaboration between onshore and offshore during overlapping hours. • Take full ownership of assigned stories, ask questions and seek clarification as needed before and during sprint • Comfor with collaboration across functions/teams.