On-Shore Streaming Tech Lead
⭐ - Featured Role | Apply direct with Data Freelance Hub
This role is for an On-Shore Streaming Tech Lead on a long-term contract, remote. Key skills include Apache Flink, AWS, and CI/CD DevOps. Requires 5-8 years of streaming data engineering experience and preferred certifications in Databricks and AWS.
🌎 - Country
United States
💱 - Currency
$ USD
-
💰 - Day rate
-
🗓️ - Date discovered
May 31, 2025
🕒 - Project duration
Unknown
-
🏝️ - Location type
Remote
-
📄 - Contract type
Unknown
-
🔒 - Security clearance
Unknown
-
📍 - Location detailed
United States
-
🧠 - Skills detailed
#Terraform #Spark (Apache Spark) #Cloud #DevOps #Delta Lake #Observability #Kubernetes #Scala #Data Engineering #Monitoring #S3 (Amazon Simple Storage Service) #IAM (Identity and Access Management) #Databricks #AutoScaling #AWS (Amazon Web Services) #Java #Python #Agile
Role description
Title: On-Shore Streaming Tech Lead
Location: Remote
Job type: Long term Contract
Must skills:
• Streaming Structure with Apache Flink
• Cloud Background
• AWS, CICD Devops
Core Responsibilities:
• Rapid PoC of 3 critical pipelines (hi-, mid-, low-fi) in RT-mode
• Design work-arounds for stateful-join & operator-autoscale gaps
• Performance-tune Spark Structured Streaming for sub-second SLAs
• Pair-program with Rivian engineers; create annotated notebooks & docs
• Liaise with Databricks engineering / PS for feature flags & bug triage
Hard Skills:
• Expertise: Apache Flink (v1.15+) & Spark Structured Streaming/RT-mode
• Languages: Scala (required), Java/Python (nice-to-have)
• Latency Engineering: state management, watermarks, LRU eviction, checkpointing
• Autoscaling: Kubernetes & Databricks autoscale strategies
• Cloud: AWS (EKS, S3, IAM); Databricks Unity Catalog & Delta Lake
• DevOps: CI/CD via Databricks Repos & Terraform; monitoring with Lakehouse Observability
Soft Skills:
Strong client-facing communication, agile delivery mindset, knowledge-transfer coaching.
risk-first problem solving.
Experience:
5-8 yrs streaming data engineering; delivered at least one < 200 ms production pipeline on Flink and one on Structured Streaming.
Certifications (preferred):
Databricks Data Engineer Professional; AWS Solutions Architect Associate.
Title: On-Shore Streaming Tech Lead
Location: Remote
Job type: Long term Contract
Must skills:
• Streaming Structure with Apache Flink
• Cloud Background
• AWS, CICD Devops
Core Responsibilities:
• Rapid PoC of 3 critical pipelines (hi-, mid-, low-fi) in RT-mode
• Design work-arounds for stateful-join & operator-autoscale gaps
• Performance-tune Spark Structured Streaming for sub-second SLAs
• Pair-program with Rivian engineers; create annotated notebooks & docs
• Liaise with Databricks engineering / PS for feature flags & bug triage
Hard Skills:
• Expertise: Apache Flink (v1.15+) & Spark Structured Streaming/RT-mode
• Languages: Scala (required), Java/Python (nice-to-have)
• Latency Engineering: state management, watermarks, LRU eviction, checkpointing
• Autoscaling: Kubernetes & Databricks autoscale strategies
• Cloud: AWS (EKS, S3, IAM); Databricks Unity Catalog & Delta Lake
• DevOps: CI/CD via Databricks Repos & Terraform; monitoring with Lakehouse Observability
Soft Skills:
Strong client-facing communication, agile delivery mindset, knowledge-transfer coaching.
risk-first problem solving.
Experience:
5-8 yrs streaming data engineering; delivered at least one < 200 ms production pipeline on Flink and one on Structured Streaming.
Certifications (preferred):
Databricks Data Engineer Professional; AWS Solutions Architect Associate.
