Data Engineer (Databricks/ Scala/ Spark)

General Info: Must be a US Citizen or GC Holder Fully Remote Contract to Hire Summary: The Data Engineer will design, build, and optimize scalable data platforms to support analytics and business decision-making. This role focuses on Databricks and cloud-based data engineering, integrating structured and unstructured data using Scala, Apache Spark, and both batch and streaming architectures. You will work closely with data scientists, architects, and client stakeholders in a consulting environment. Key Responsibilities: • Design and deliver end-to-end big data and AI-enabled analytics platforms • Build and optimize batch and streaming data pipelines • Apply Databricks, Spark, and cloud best practices • Serve as technical lead for complex solutions; document and support delivery • Support estimation, technical risk management, and knowledge transfer Required Experience: • 3+ years as a Data Engineer / Big Data Engineer • Hands-on Databricks experience in production environments • Strong Scala and Apache Spark expertise • Experience with batch and streaming pipelines (Kafka, Structured Streaming) • Experience with DataOps, CI/CD, and cloud migrations • Experience with at least two cloud platforms: AWS, Azure, or GCP • Bachelor’s degree or equivalent experience Core Technical Skills: • Scala & Spark: Functional programming, immutability, performance tuning • Streaming: Kafka, watermarking, windowed aggregations • Data Engineering: ETL/ELT, data modeling, orchestration • Databricks: Notebooks, Jobs, Delta Lake, CI/CD, performance optimization Nice to Have: • ML frameworks (Spark MLlib, TensorFlow, PyTorch) • Experience with StreamSets, MapReduce • Databricks, Azure, AWS, or GCP certifications