Lead Data Engineer - (Automotive Exp)

Role: Sr Technical Lead Duration: 12+ Month Contract Location: Torrance, CA (4 days Onsite) Daily Tasks Performed: • Lead the design, development, and deployment of a scalable, secure, and high-performance CDP SaaS product. • Architect solutions that integrate with various data sources, APIs, and third-party platforms. • Design, develop, and optimize complex SQL queries for data extraction, transformation, and analysis • Build and maintain workflow pipelines using Digdag, integrating with data platforms such as Treasure Data, AWS, or other cloud services • Automate ETL processes and schedule tasks using Digdag's YAML-based workflow definitions • Implement data quality checks, logging, and alerting mechanisms within workflow Leverage AWS services (e.g., S3, Lambda, Athena) where applicable to enhance data processing and storage capabilities • Ensure best practices in software engineering, including code reviews, testing, CI/CD, and documentation. • Oversee data privacy, security, and compliance initiatives (e.g., GDPR, CCPA). • Ensure adherence to security, compliance, and data governance requirements. • Oversee development of real-time and batch data processing systems. • Collaborate with cross-functional teams including data analysts, product managers, and software engineers to translate business requirements into technical solutions • Collaborate with the stakeholders to define technical requirements to align technical solutions with business goals and deliver product features. • Mentor and guide developers, fostering a culture of technical excellence and continuous improvement. • Troubleshoot complex technical issues and provide hands-on support as needed. • Monitor, troubleshoot, and improve data workflows for performance, reliability, and cost-efficiency as needed • Optimize system performance, scalability, and cost efficiency. What this person will be working on: As the Senior Technical Lead for our Customer Data Platform (CDP), the candidate will define the technical strategy, architecture, and execution of the platform. They will lead the design and delivery of scalable, secure, and high-performing solutions that enable unified customer data management, advanced analytics, and personalized experiences. This role demands deep technical expertise, strong leadership, and a solid understanding of data platforms and modern cloud technologies. It is a pivotal position that supports the CDP vision by mentoring team members and delivering solutions that empower our customers to unify, analyze, and activate their data. Position Success Criteria (Desired) - 'WANTS' • Bachelor's or Master's degree in Computer Science, Engineering, or related field. • 8+ years of software development experience, with at least 3+ years in a technical leadership role. • Proven experience building and scaling SaaS products, preferably in customer data, marketing technology, or analytics domains • Extensive hands-on experience with Presto, Hive, and Python • Strong proficiency in writing complex SQL queries for data extraction, transformation, and analysis • Familiarity with AWS data services such as S3, Athena, Glue, and Lambda • Deep understanding of data modeling, ETL pipelines, workflow orchestration, and both real-time and batch data processing • Experience ensuring data privacy, security, and compliance in SaaS environments • Knowledge of Customer Data Platforms (CDPs), CDP concepts, and integration with CRM, marketing, and analytics tools • Excellent communication, leadership, and project management skills • Experience working with Agile methodologies and DevOps practices • Ability to thrive in a fast-paced, agile environment • Collaborative mindset with a proactive approach to problem-solving • Stay current with industry trends and emerging technologies relevant to SaaS and customer data platforms.