Data Engineer

We are seeking a Data Engineer with Spark & SCALA ; Streaming skills builds real-time, scalable data pipelines using tools like Spark, Kafka, and cloud services (GCP) to ingest, transform, and deliver data for analytics and ML. As a Senior Data Engineer, you will: • Design, develop, and maintain ETL/ELT data pipelines for batch and real-time data ingestion, transformation, and loading using Spark (PySpark/Scala) and streaming technologies (Kafka, Flink). • Build and optimize scalable data architectures, including data lakes, data warehouses (BigQuery), and streaming platforms. • Performance Tuning: Optimize Spark jobs, SQL queries, and data processing workflows for speed, efficiency, and cost-effectiveness • Data Quality: Implement data quality checks, monitoring, and alerting systems to ensure data accuracy and consistency. Required Skills & Qualifications: • At least 8 years of IT experience • 4 + years of recent GCP experience • Programming: Strong proficiency in Python, SQL, and potentially Scala/Java. • Big Data: Expertise in Apache Spark (Spark SQL, DataFrames, Streaming). • Streaming: Experience with messaging queues like Apache Kafka, or Pub/Sub. • Cloud: Familiarity with GCP, Azure data services. • Databases: Knowledge of data warehousing (Snowflake, Redshift) and NoSQL databases. • Tools: Experience with Airflow, Databricks, Docker, Kubernetes is a plus. We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, sex, gender, gender expression, sexual orientation, age, marital status, veteran status, or disability status. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform crucial job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.