IntraEdge
Senior Data Engineer
⭐ - Featured Role | Apply direct with Data Freelance Hub
This role is for a Senior Data Engineer with a contract length of "unknown", offering a pay rate of "unknown". Key skills include Databricks, PySpark, Scala, SQL, and Apache Airflow. Experience in data lakehouse solutions and real-time data processing is required.
🌎 - Country
United States
💱 - Currency
$ USD
-
💰 - Day rate
Unknown
-
🗓️ - Date
June 12, 2026
🕒 - Duration
Unknown
-
🏝️ - Location
Unknown
-
📄 - Contract
Unknown
-
🔒 - Security
Unknown
-
📍 - Location detailed
Phoenix, AZ
-
🧠 - Skills detailed
#Deployment #"ETL (Extract #Transform #Load)" #PySpark #Data Lakehouse #Leadership #Data Lake #Metadata #Security #Databricks #Airflow #SQL (Structured Query Language) #Data Processing #Batch #Data Quality #DevOps #Scala #Data Modeling #Cloud #Apache Airflow #Spark (Apache Spark) #Automation #Data Engineering
Role description
Key Responsibilities
• Implement enterprise-grade Lakehouse solutions using Databricks.
• Design and deliver end-to-end data engineering pipelines, including batch and real-time streaming solutions.
• Lead implementation of: Cloud-based data lakehouse platforms integrating diverse data sources.
• Real-time data processing pipelines for operational and analytical use cases.
• Develop scalable ETL/ELT pipelines using PySpark, Scala, and SQL.
• Implement advanced data modeling solutions including 3NF, dimensional modeling, and enterprise data warehousing strategies.
• Design and build incremental data loading frameworks and metadata-driven ingestion pipelines.
• Establish data quality frameworks and governance standards.
• Implement and manage Unity Catalog, including fine-grained security and access controls.
• Leverage Databricks components such as:
• Delta Live Tables
• Autoloader
• Structured Streaming
• Databricks Workflows
• Integration with orchestration tools (e.g., Apache Airflow)
• Drive CI/CD automation, deployment strategies, and DevOps best practices.
• Optimize performance of pipelines, Spark jobs, and compute resources.
• Provide architectural guidance and technical leadership across cross-functional teams.
• Engage with stakeholders and clients to translate business requirements into scalable technical solutions
Key Responsibilities
• Implement enterprise-grade Lakehouse solutions using Databricks.
• Design and deliver end-to-end data engineering pipelines, including batch and real-time streaming solutions.
• Lead implementation of: Cloud-based data lakehouse platforms integrating diverse data sources.
• Real-time data processing pipelines for operational and analytical use cases.
• Develop scalable ETL/ELT pipelines using PySpark, Scala, and SQL.
• Implement advanced data modeling solutions including 3NF, dimensional modeling, and enterprise data warehousing strategies.
• Design and build incremental data loading frameworks and metadata-driven ingestion pipelines.
• Establish data quality frameworks and governance standards.
• Implement and manage Unity Catalog, including fine-grained security and access controls.
• Leverage Databricks components such as:
• Delta Live Tables
• Autoloader
• Structured Streaming
• Databricks Workflows
• Integration with orchestration tools (e.g., Apache Airflow)
• Drive CI/CD automation, deployment strategies, and DevOps best practices.
• Optimize performance of pipelines, Spark jobs, and compute resources.
• Provide architectural guidance and technical leadership across cross-functional teams.
• Engage with stakeholders and clients to translate business requirements into scalable technical solutions
