Lead Data Engineer - AI Data Products

Lead Data Engineer – AI Data Products | Databricks | Vector Databases Contract-to-Permanent Hire (W-2) 100% Remote (CST Work Hours) Our Fortune 50 healthcare client is seeking a Lead AI Data Engineer to drive the strategy and execution of scalable AI data products. This is a ground-floor opportunity for a leader who thrives on mentoring others, shaping engineering practices, and building robust, cloud-native data pipelines enabling enterprise AI/ML capabilities. The role balances hands-on implementation (~50%) with leadership and strategic oversight as the team and AI product portfolio grow. You’ll guide engineers, conduct code reviews, and collaborate cross-functionally to build AI-ready data products. Responsibilities: • Own the technical strategy for data engineering and AI product development. • Design and implement scalable data pipelines in Databricks and ETL/ELT workflows, leveraging Medallion Architecture and cloud-native best practices. • Develop data engineering solutions using Python, RDE/RDS, and related core technologies. • Define, document, and evolve engineering standards, architecture, and development workflows. • Mentor and guide a growing team of engineers, including senior and entry-level talent, providing technical oversight and supporting professional growth. • Conduct code reviews to ensure maintainable, secure, and high-quality code delivery. • Balance hands-on implementation (about 50%) with leadership and strategic responsibilities as the team and AI product portfolio scale. • Partner with cross-functional teams to deliver scalable, production-ready AI data products. Requirements: • Bachelor of Science in Computer Science or Data Science required; MS degree preferred. • 10 years of professional experience in data engineering, including integrating AI/ML solutions. • 2 years of experience as a Lead Engineer, conducting code reviews and mentoring both senior and junior engineers. • Hands-on experience with Databricks and cloud-based data engineering. • Expertise in Python, ETL/ELT pipeline development, and Medallion Architecture. • Experience with vector databases (FAISS, Pinecone, Weaviate, Milvus) and building AI-ready data assets (embeddings, vector stores, feature tables, semantic AI context assets). • Experience driving architectural decisions and technical standards for scalable data & AI solutions. • Ability to define and drive engineering practices in a dynamic, data & AI-focused environment, applying DevOps and code-first principles to data workflows. • US Citizens & Green Card holders only •