Key Skills: experience building Data Pipelines from multiple data sources, experience with Databricks in an AWS environment, Spark/PySpark, Airflow, Kafka, Hive. Must have experience working with Large Scale Data warehouses (multiple TBs). Must have experience working with DAGs in production, must have experience working with large-scale data pipeline in a production environment LOCAL candidates only. Face to Face Interview required. Location: San Jose, CA (onsite 2X a week) In Person Interview Required Duration: 12+ month contract Requirements: Background Check Description We’re looking for a Data Pipeline Engineer with deep experience building and orchestrating large-scale ingestion pipelines. This role is ideal for someone who enjoys working across high-volume telemetry sources, optimizing data workflows, and solving schema drift challenges in real-world distributed environments. You’ll be part of the Security Data Platform and ML Engineering team, helping to onboard and normalize security data that powers analytics, detection, and ML workflows across the BU. Key Responsibilities • Design and build scalable batch and streaming data pipelines for ingesting telemetry, log, and event data • Develop and maintain orchestration workflows using tools like Apache Airflow or similar schedulers • Onboard new data sources, build connectors (API/Kafka/file-based), and normalize security-related datasets • Monitor and manage schema drift across changing source systems and formats • Implement observability into pipelines — logging, metrics, and alerts for health and performance • Optimize ingestion for performance, resilience, and cost-efficiency • Collaborate across detection, threat intel, and platform teams to align ingestion with security use cases Required Qualifications • 5+ years of experience in data engineering or infrastructure roles focused on pipeline development • Strong experience with Python and distributed data processing tools like Apache Spark or PySpark • Hands-on experience with orchestration frameworks like Apache Airflow, Dagster, or similar • Deep understanding of ingestion best practices, schema evolution, and drift handling • Experience working with Kafka, S3, or cloud-native storage and messaging systems • Experience in cloud environments (AWS, Azure, or GCP) • Bonus: Familiarity with security tools (e.g., Crowdstrike, Wiz), OCSF, or compliance-related data #techjobs The base pay range above represents the low and high end of the base compensation range we reasonably expect to pay for this position. Actual base compensation will vary and may be above or below the range based on various factors including, but not limited to, geographic location, actual experience, and job performance. This job posting is not a promise of any specific pay for any specific employee. The range listed is just one component of the total compensation package for our employees. Based on the details of your position, we provide a variety of benefits to our employees, including medical, dental, and vision plans, pre-tax savings plans, pre-tax parking and commuter plans, supplemental health and welfare plans, a retirement savings plan, an employee assistance program, pet insurance, and paid holidays. Other rewards may include short-term incentives and paid time off. After you have applied, download our Staffmark Group WorkNOW App to receive real-time job offers and apply for additional opportunities. You can download it from the App Store or get it on Google Play. About Advantage Technical With company roots going back over 30 years, Advantage Technical is an engineering and information technology services company and a national leader in the provision of technical resources today. These services include Staff Augmentation, Direct Placement, Project Resourcing and Outsourcing – delivered from 40 key market locations, by over 3500 specialized contractors, to over 500 clients across North America. Advantage Technical is a Best of Staffing Diamond Award winner for both Clients and Talent. For more information about the industries and services offered by Advantage Technical, please visit AdvantageTechnical.com. Advantage Technical is committed to providing equal employment opportunity for all persons regardless of race, color, religion (including religious dress and grooming practices), sex, sexual orientation, gender, gender identity, gender expression, age, marital status, national origin, ancestry, citizenship status, pregnancy, medical condition, genetic information, mental and physical disability, political affiliation, union membership, status as a parent, military or veteran status or other non-merit based factors. We will provide reasonable accommodations throughout the application, interviewing and employment process. If you require a reasonable accommodation, contact your local branch. Advantage Technical is an E-Verify employer. This policy is applicable to all phases of the employment relationship, including hiring, transfers, promotions, training, terminations, working conditions, compensation, benefits, and other terms and conditions of employment. All employees are directed to familiarize themselves with this policy and to act in accordance with it. All decisions with respect to employment matters and other phases of employer-temporary employee relationships will be in keeping with this policy and in accordance with all applicable laws and regulations. To read our Privacy Notice for Candidates and Employees/Contractors, click here. By applying for this job, you agree that you may receive both AI-generated and non-AI generated calls, text messages, or emails from Staffmark Group and/or its affiliates, and contracted partners. Frequency varies for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You can reply STOP to cancel and HELP for help. You can access our general Privacy Policy at Privacy Policy - Staffmark