Dice is the leading career destination for tech experts at every stage of their careers. Our client, Collaborate Solutions, Inc., is seeking the following. Apply via Dice today! Title of Position: Sr Data Engineer Location: Charlotte, NC Hybrid (2-3 days a week, local candidates only) Duration: 12 months Top Skills Required/Notes: Required Skills (top 3 non-negotiables): • PySpark • Google Dataproc • Google BigQuery Nice to have: • Airflow • Scala • Hadoop/Hive Description: • Convert product requirements into clear and actionable technical designs. • Design, build, test, and optimize data pipelines (batch or real-time), while handling data ingestion, processing, and transformation tasks. • Implement built-in and customized data monitoring and governance mechanisms. • Identify and remediate problems related to data ingestion, transformation, quality, or performance. • Write Infrastructure-as-Code (IaC) for deployments and Maintain CI/CD pipelines for data workflows. Required Testing: Strong grasp of data structures and algorithms (arrays, hash maps, etc.), with proven ability in parsing and transforming data. Skilled in developing ETL-like scripts using Python, Scala on Spark environment and proficient in writing optimized, high-performance SQL queries that leverage data aggregation, window functions, and subqueries. Software Skills Required: Language- Python, Scala Database - BigQuery, Hive, Trino, PostgreSQL, MongoDB etc. Data Platforms - Spark, Airflow, Kafka, Google Cloud Platform (BigQuery, Dataflow). Modelling and Transformation - ELT/ETL framework