Data Engineer with AI/ML

This role bridges traditional data engineering (ETL/ELT) and data science, ensuring data is clean, accessible, and optimized for model training, real-time inference, and Generative AI (GenAI) workloads. Responsibilities • AI/ML Pipeline Development: Design and maintain robust ETL/ELT pipelines specifically for feeding data into machine learning models. • Data Preparation & Feature Engineering: Automate data preprocessing (normalization, encoding, augmentation) and collaborate with data scientists to create feature stores. • Infrastructure Optimization: Implement data lakes, warehouses, and vector databases (e.g., Pinecone, Weaviate) optimized for AI workloads. • MLOps & Deployment: Implement MLOps practices, including CI/CD, model versioning, monitoring model performance, and automating retrain workflows. • Real-time Streaming: Implement real-time data streaming for live inference using tools like Kafka or Flink. • Data Governance & Security: Ensure data quality, lineage tracking, and compliance with privacy standards (GDPR, CCPA) in AI contexts. Qualifications • Education: Bachelor’s or Master’s in Computer Science, Data Engineering, or a related field. • Experience: Generally 3–7+ years in data engineering or related ML roles. Required Skills • Programming: High proficiency in Python and SQL; experience with Scala or Java is often preferred. • Big Data Technologies: Experience with Apache Spark, Hadoop, Hive, and Kafka. • Cloud Platforms: Proficiency in AWS, Azure, or GCP cloud services. • ML Frameworks: Familiarity with TensorFlow, PyTorch, or Scikit-learn. • Data Tools: Experience with Airflow, dbt, and MLflow.