Soho Square Solutions
Senior Data Engineer
⭐ - Featured Role | Apply direct with Data Freelance Hub
This role is for a Senior Data Engineer focused on Databricks, requiring 7+ years of experience with Databricks, Spark, Delta Lake, Python, and SQL. Contract length is unspecified, with a pay rate of "unknown," and location is "remote."
🌎 - Country
United States
💱 - Currency
$ USD
-
💰 - Day rate
Unknown
-
🗓️ - Date
June 2, 2026
🕒 - Duration
Unknown
-
🏝️ - Location
Unknown
-
📄 - Contract
Unknown
-
🔒 - Security
Unknown
-
📍 - Location detailed
United States
-
🧠 - Skills detailed
#Data Engineering #SQL (Structured Query Language) #Monitoring #Python #Datasets #Delta Lake #Databricks #AI (Artificial Intelligence) #Data Quality #ML (Machine Learning) #"ETL (Extract #Transform #Load)" #Spark (Apache Spark) #Scala
Role description
Senior Databricks Data Engineer – AI & Data Platform
Key Responsibilities
• Design and optimize Databricks Medallion Architecture (Bronze, Silver, Gold) pipelines using Delta Lake, Spark, Python, and SQL.
• Ingest, process, and transform structured and unstructured data from enterprise systems.
• Build scalable, AI-ready datasets and data models for analytics and LLM-powered applications.
• Develop data quality, validation, and monitoring frameworks.
• Support RAG pipelines, embeddings, and data preparation for AI agents.
• Manage Unity Catalog governance, access controls, and schema management.
• Collaborate with business and technical teams to deliver scalable data solutions.
Required Qualifications
• 7+ years of experience with Databricks, Spark, Delta Lake, Python, and SQL.
• Strong experience implementing Medallion Architecture at scale.
• Expertise in processing unstructured data (documents, PDFs, text, images).
• Experience supporting AI/ML, Generative AI, or LLM-based solutions.
• Knowledge of embeddings, RAG, and AI agent workflows.
• Strong problem-solving and stakeholder management skills.
Senior Databricks Data Engineer – AI & Data Platform
Key Responsibilities
• Design and optimize Databricks Medallion Architecture (Bronze, Silver, Gold) pipelines using Delta Lake, Spark, Python, and SQL.
• Ingest, process, and transform structured and unstructured data from enterprise systems.
• Build scalable, AI-ready datasets and data models for analytics and LLM-powered applications.
• Develop data quality, validation, and monitoring frameworks.
• Support RAG pipelines, embeddings, and data preparation for AI agents.
• Manage Unity Catalog governance, access controls, and schema management.
• Collaborate with business and technical teams to deliver scalable data solutions.
Required Qualifications
• 7+ years of experience with Databricks, Spark, Delta Lake, Python, and SQL.
• Strong experience implementing Medallion Architecture at scale.
• Expertise in processing unstructured data (documents, PDFs, text, images).
• Experience supporting AI/ML, Generative AI, or LLM-based solutions.
• Knowledge of embeddings, RAG, and AI agent workflows.
• Strong problem-solving and stakeholder management skills.
