Data Engineer

Data Engineer (API Ingestion & Data Lake) We are seeking a hands-on, mid-level Data Engineer to support a security-focused data lake initiative. This is a contract, project-based role focused on building reliable API ingestion pipelines, creating structured SQL datasets, and preparing data for downstream analytics and future AI use cases. This role is execution-heavy and ideal for a builder who enjoys owning data pipelines end-to-end. What Success Looks Like: You bring 2+ years of experience building API-based ingestion pipelines at scale, managing data across multiple sources (14 total) and supporting large, complex datasets with thousands of fields. Role Details - Contract Length: 6 months (likely extension, no guaranteed conversion) - Location: Remote (U.S.-based) - Work Type: Individual contributor, hands-on technical role - Start: ASAP -Interview: Wed. Feb 4th and Thurs. Feb 5th Core Requirement (Non-Negotiable) What You Will Do - Build and own API-based ingestion pipelines pulling data from third-party systems - Handle authentication, pagination, retries, failures, and schema changes - Ingest, transform, and structure raw data into SQL-based datasets - Validate incoming data for accuracy, completeness, and consistency - Create and maintain clear documentation for datasets, fields, and ingestion logic - Manage and support datasets across multiple data sources (approx. 14 total) - Prepare clean, well-structured data for downstream analytics and AI consumption - Operate independently and take ownership of deliverables with minimal hand-holding Required Skills & Experience - 2+ years of hands-on experience writing and maintaining API ingestion pipelines in a big data environment, managing datasets across 14 known data sources and supporting datasets with 10–15K+ fields. - 3–6 years of hands-on experience as a Data Engineer or similar role - Strong experience building API-based ingestion pipelines - Proficiency with SQL / SQL Server and relational data modeling - Experience working in AWS (S3, Lambda, EC2, RDS, or similar) - Ability to validate and QA large, complex datasets - Comfortable working with large field counts and evolving schemas - Strong documentation and communication skills Nice to Have - Exposure to security, device, endpoint, or asset data - Experience with tools such as Qualys, CrowdStrike, Intune, JAMF, Wiz, or SCCM - Familiarity with data lake concepts (raw vs curated layers) - Experience supporting analytics or AI teams (data prep only, not model building) What This Role Is NOT - Not an architecture-only or strategy-only role - Not focused on AI/ML model development - Not a streaming/Kafka-heavy platform engineering role - Not a people management position Ideal Candidate Profile The ideal candidate is a practical builder who enjoys solving messy data problems, can clearly explain how they’ve built ingestion pipelines in the past, and is comfortable working in a fast-moving, project-based environment.