Nova Web Technologies LLC
Data Engineer – Databricks + DevOps
⭐ - Featured Role | Apply direct with Data Freelance Hub
This role is for a Data Engineer – Databricks + DevOps in Santa Clara, CA, for over 6 months at a competitive pay rate. Requires 8+ years of experience, strong skills in Databricks, AWS, and DevOps, with a focus on large-scale data processing.
🌎 - Country
United States
💱 - Currency
Unknown
-
💰 - Day rate
Unknown
-
🗓️ - Date
February 14, 2026
🕒 - Duration
More than 6 months
-
🏝️ - Location
On-site
-
📄 - Contract
Unknown
-
🔒 - Security
Unknown
-
📍 - Location detailed
Santa Clara, CA
-
🧠 - Skills detailed
#Lambda (AWS Lambda) #Data Pipeline #"ETL (Extract #Transform #Load)" #API (Application Programming Interface) #Leadership #GitLab #Scala #GIT #Data Governance #Databricks #SQL (Structured Query Language) #Delta Lake #AWS (Amazon Web Services) #Data Processing #Spark (Apache Spark) #DevOps #Data Engineering #PySpark #IAM (Identity and Access Management) #Security #Storage #Terraform #Deployment #S3 (Amazon Simple Storage Service)
Role description
Location: Santa Clara, CA Experience: 8+ Years Employment Type: Full-time / Contract
Position Overview:
NovaSoft is seeking a highly experienced Databricks Data Engineer with strong DevOps expertise to design, implement, and optimize large-scale Lakehouse architectures on AWS.
This role requires deep architectural understanding of compute vs. serving layer separation, low-latency data/API access strategies, and multi-terabyte data processing. The ideal candidate combines hands-on engineering excellence with technical leadership — a true player-coach mindset.
You will work closely with cross-functional teams to build scalable, secure, automated, and high-performance data platforms using modern DevOps practices.
Key Responsibilities:
Design and implement scalable Databricks Lakehouse architectures on AWS
Build and optimize ETL/ELT pipelines using PySpark, Spark, and SQL
Implement Delta Lake best practices (partitioning, optimization, schema evolution)
Develop and manage CI/CD pipelines and automated deployments using DevOps tools
Optimize Spark workloads for performance, cost efficiency, and low-latency access
Implement data governance and security using Unity Catalog
Collaborate with cross-functional teams and provide technical leadership
Technical Skills (Required):
Strong hands-on experience with:
Databricks (Delta Lake, Unity Catalog, Delta Live Pipelines, Workflows, Runtime)
PySpark, Spark, Advanced SQL
Lakehouse & Medallion Architecture
AWS expertise including:
S3, IAM, Glue / Glue Catalog
Lambda
Secrets Manager
(Kinesis is a plus)
DevOps expertise:
Git-based workflows
CI/CD pipelines
Databricks Asset Bundles
Terraform (preferred)
Experience handling multi-terabyte workloads
Strong understanding of performance tuning, partitioning, and storage optimization
Preferred Experience:
Structured Streaming / real-time data pipelines
Advanced Databricks runtime configuration
Real-time or near real-time data solutions
Exposure to GitLab CI/CD pipelines
Certifications (Optional)
Databricks Certified Data Engineer (Associate / Professional)
AWS Certified Data Engineer or Solutions Architect
Location: Santa Clara, CA Experience: 8+ Years Employment Type: Full-time / Contract
Position Overview:
NovaSoft is seeking a highly experienced Databricks Data Engineer with strong DevOps expertise to design, implement, and optimize large-scale Lakehouse architectures on AWS.
This role requires deep architectural understanding of compute vs. serving layer separation, low-latency data/API access strategies, and multi-terabyte data processing. The ideal candidate combines hands-on engineering excellence with technical leadership — a true player-coach mindset.
You will work closely with cross-functional teams to build scalable, secure, automated, and high-performance data platforms using modern DevOps practices.
Key Responsibilities:
Design and implement scalable Databricks Lakehouse architectures on AWS
Build and optimize ETL/ELT pipelines using PySpark, Spark, and SQL
Implement Delta Lake best practices (partitioning, optimization, schema evolution)
Develop and manage CI/CD pipelines and automated deployments using DevOps tools
Optimize Spark workloads for performance, cost efficiency, and low-latency access
Implement data governance and security using Unity Catalog
Collaborate with cross-functional teams and provide technical leadership
Technical Skills (Required):
Strong hands-on experience with:
Databricks (Delta Lake, Unity Catalog, Delta Live Pipelines, Workflows, Runtime)
PySpark, Spark, Advanced SQL
Lakehouse & Medallion Architecture
AWS expertise including:
S3, IAM, Glue / Glue Catalog
Lambda
Secrets Manager
(Kinesis is a plus)
DevOps expertise:
Git-based workflows
CI/CD pipelines
Databricks Asset Bundles
Terraform (preferred)
Experience handling multi-terabyte workloads
Strong understanding of performance tuning, partitioning, and storage optimization
Preferred Experience:
Structured Streaming / real-time data pipelines
Advanced Databricks runtime configuration
Real-time or near real-time data solutions
Exposure to GitLab CI/CD pipelines
Certifications (Optional)
Databricks Certified Data Engineer (Associate / Professional)
AWS Certified Data Engineer or Solutions Architect
