Lead Data Engineer

Role: Lead Data Engineer - 12+ Location: San Francisco, CA - Hybrid Hybrid - 3 days in office Experience level:12+ Skills: Lead Data Engineer, email marketing systems and marketing technology (MarTech) platforms. Key Responsibilities • Lead end-to-end MarTech engineering initiatives across orchestration, data processing, and activation pipelines. • Architect scalable, event-driven systems that power real-time marketing experiences and automated customer journeys. • Design and implement orchestration workflows using Adobe Campaign or equivalent enterprise-grade tools. • Develop high-performance big data applications using Scala, Databricks, Spark SQL, Spark Streaming, and Python. • Build and optimize cloud-native data pipelines on Azure, including ADF-based ingestion, transformation, and orchestration. • Apply modern design patterns to ensure reliability, maintainability, and scalability across distributed systems. • Drive AI-assisted engineering practices including Vibe Coding and other generative AI development accelerators. • Collaborate with product, marketing, and data teams to translate business needs into robust technical solutions. • Mentor engineers and elevate engineering standards, code quality, and operational excellence within the POD. Required Skills & Experience • Deep expertise in MarTech platforms with hands-on experience in Adobe Campaign or similar orchestration tools. • Strong proficiency in big data technologies: Scala, Databricks, Spark SQL, Spark Streaming, Python. • Cloud engineering experience with Azure services, including Azure Data Factory. • Advanced system design capabilities including event-driven architectures and distributed design patterns. • Experience with AI-augmented development such as Vibe Coding or comparable frameworks. • Proven ability to lead engineering teams in a fast-paced, cross-functional environment. • Strong communication and stakeholder alignment skills with the ability to translate technical concepts into business impact. Preferred Qualifications • Experience in large-scale marketing ecosystems (ESP, CDP, personalization engines, real-time decisioning). • Background in high-volume data processing supporting customer engagement or growth marketing. • Familiarity with DevOps practices including CI/CD, observability, and automated testing. • Exposure to modern AI/ML pipelines for personalization, segmentation, or content automation.