Compunnel Inc.
Data Engineer
⭐ - Featured Role | Apply direct with Data Freelance Hub
This role is for a Data Engineer with a contract length of "unknown" at a pay rate of "unknown." Required skills include 2+ years with Databricks and Collibra, 3+ years with Python and PySpark, and AWS experience.
🌎 - Country
United States
💱 - Currency
$ USD
-
💰 - Day rate
Unknown
-
🗓️ - Date
December 2, 2025
🕒 - Duration
Unknown
-
🏝️ - Location
Unknown
-
📄 - Contract
Unknown
-
🔒 - Security
Unknown
-
📍 - Location detailed
San Francisco Bay Area
-
🧠 - Skills detailed
#Agile #NoSQL #Scala #S3 (Amazon Simple Storage Service) #Databricks #PySpark #Unit Testing #Data Engineering #Datasets #Big Data #Spark (Apache Spark) #Python #AWS (Amazon Web Services) #Monitoring #Jupyter #Data Warehouse #"ETL (Extract #Transform #Load)" #Airflow #Databases #Computer Science #Redshift #Security #Data Pipeline #Collibra #Data Science #Snowflake #Automation #Cloud
Role description
Job Summary:
The Data Engineer will be responsible for collecting, parsing, managing, analyzing, and visualizing large datasets to transform information into actionable insights. The role involves building scalable, repeatable, and secure data pipelines across various platforms. The engineer will ensure that data workflows support multiple users while maintaining high performance, security, and reliability.
Key Responsibilities:
• Design, develop, and maintain robust and efficient data pipelines to ingest, transform, catalog, and deliver curated, trusted, and quality data into the Common Data Platform.
• Actively participate in Agile ceremonies and follow Scaled Agile processes defined by the CDP Program team.
• Deliver high-quality data products and services following SAFe Agile practices.
• Identify and resolve issues related to data pipelines and analytical data stores.
• Implement monitoring and alerting for pipelines and data stores, enabling auto-remediation to ensure uptime and reliability.
• Apply a security-first approach with strong testing and automation practices.
• Collaborate with product managers, data scientists, analysts, and business stakeholders to understand data needs and provide tools and infrastructure.
• Stay updated with modern data engineering trends and evaluate new tools, frameworks, and technologies to improve efficiency.
Required Skills:
• Bachelor’s degree in computer science, Information Systems, or related field, or equivalent experience.
• 2+ years’ experience with Databricks, Collibra, and Starburst.
• 3+ years’ experience with Python and PySpark.
• Experience using Jupyter notebooks for coding and unit testing.
• Strong recent experience with relational and NoSQL data stores, STAR and Dimensional Modeling methods.
• 2+ years’ experience with a modern data stack (S3, Spark, Airflow, Lakehouse architecture, real-time databases).
• Experience with cloud data warehouses such as RedShift or Snowflake.
• Overall data engineering experience across traditional ETL and Big Data (on-prem or cloud).
• AWS data engineering experience, detailing services/tools used.
• Experience building end-to-end pipelines for unstructured and semi-structured data using Spark.
• Ability to work with confidential supervisory information (Must meet Protected Individual requirements).
Education:
Bachelor’s degree in computer science, Information Systems, or related field (or equivalent experience).
Job Summary:
The Data Engineer will be responsible for collecting, parsing, managing, analyzing, and visualizing large datasets to transform information into actionable insights. The role involves building scalable, repeatable, and secure data pipelines across various platforms. The engineer will ensure that data workflows support multiple users while maintaining high performance, security, and reliability.
Key Responsibilities:
• Design, develop, and maintain robust and efficient data pipelines to ingest, transform, catalog, and deliver curated, trusted, and quality data into the Common Data Platform.
• Actively participate in Agile ceremonies and follow Scaled Agile processes defined by the CDP Program team.
• Deliver high-quality data products and services following SAFe Agile practices.
• Identify and resolve issues related to data pipelines and analytical data stores.
• Implement monitoring and alerting for pipelines and data stores, enabling auto-remediation to ensure uptime and reliability.
• Apply a security-first approach with strong testing and automation practices.
• Collaborate with product managers, data scientists, analysts, and business stakeholders to understand data needs and provide tools and infrastructure.
• Stay updated with modern data engineering trends and evaluate new tools, frameworks, and technologies to improve efficiency.
Required Skills:
• Bachelor’s degree in computer science, Information Systems, or related field, or equivalent experience.
• 2+ years’ experience with Databricks, Collibra, and Starburst.
• 3+ years’ experience with Python and PySpark.
• Experience using Jupyter notebooks for coding and unit testing.
• Strong recent experience with relational and NoSQL data stores, STAR and Dimensional Modeling methods.
• 2+ years’ experience with a modern data stack (S3, Spark, Airflow, Lakehouse architecture, real-time databases).
• Experience with cloud data warehouses such as RedShift or Snowflake.
• Overall data engineering experience across traditional ETL and Big Data (on-prem or cloud).
• AWS data engineering experience, detailing services/tools used.
• Experience building end-to-end pipelines for unstructured and semi-structured data using Spark.
• Ability to work with confidential supervisory information (Must meet Protected Individual requirements).
Education:
Bachelor’s degree in computer science, Information Systems, or related field (or equivalent experience).
