Palantir Foundry Engineer

Job Title: Palantir Foundry Engineer Location: Nashville, TN Job Description Hands-on Foundry specialist who can design ontology-first data products, engineer high-reliability pipelines, and operationalise them into secure, observable, and reusable building blocks used by multiple applications (Workshop/Slate, AIP/Actions). You’ll own the full lifecycle: from raw sources to governed, versioned, materialised datasets wired into operational apps and AIP agents. Core Responsibilities Ontology & Data Product Design: Model Object Types, relationships, and semantics; enforce schema evolution strategies; define authoritative datasets with lineage and provenance. Pipelines & Materialisations: Build Code Workbook transforms (SQL, PySpark/Scala), orchestrate multi-stage DAGs, tune cluster/runtime parameters, and implement incremental + snapshot patterns with backfills and recovery. Operationalization: Configure schedules, SLAs/SLOs, alerts/health checks, and data quality tests (constraints, anomaly/volume checks); implement idempotency, checkpointing, and graceful retries. Governance & Security: Apply RBAC, object-level permissions, policy tags/PII handling, and least-privilege patterns; integrate with enterprise identity; document data contracts. Performance Engineering: Optimise joins/partitions, caching/materialization strategies, file layout (e.g., Parquet/Delta), and shuffle minimisation; instrument runtime metrics and cost models. Data Engineering & SDLC: Use Git-based code repos, branching/versioning, code reviews, unit/integration tests for transforms; template patterns for reuse across domains. Observability & Alerting: Expose metrics/dashboards to Workshop/Slate apps via Actions and AIP agents to governed datasets; publish clean APIs/feeds to downstream systems. Reliability & Incident Response: Own on-call for data products, run RCAs, create runbooks, and drive preventive engineering. Documentation & Enablement: Produce playbooks, data product specs, and runbooks; mentor engineers and analysts on Foundry best practices. Required Qualifications • 7+ years in data engineering/analytics engineering with 4+ years of hands-on Palantir Foundry at scale. • Deep expertise in Foundry Ontology, Code Workbooks, Pipelines, Materialisations, Lineage/Provenance, and object permissions. • Strong SQL and PySpark/Scala in Foundry; comfort with UDFs, window functions, and partitioning/bucketing strategies. • Proven operational excellence: SLAs/SLOs, alerts, data quality frameworks, backfills, rollbacks, blue/green or canary data releases. • Frequent Git, CI/CD for Foundry code repos, test automation for transforms, and environment promotion. • Hands-on with cloud storage & compute (AWS/Azure/GCP), file formats (Parquet/Delta), and cost/perf tuning. • Strong grasp of data governance (PII, masking, policy tags) and security models within Foundry. Nice to Have • Building Workshop/Slate UX tied to ontology objects; authoring Actions and integrating AIP use cases. • Streaming/event ingestion patterns (e.g., Kafka/Kinesis) are materialised into curated datasets. • Observability stacks (e.g., Datadog/CloudWatch/Prometheus) for pipeline telemetry; FinOps/cost governance. • Experience establishing platform standards: templates, code style, testing frameworks, and domain data product catalogues. Success Metrics (90–180 Days) • 99.5% pipeline success rate, with documented SLOs and active alerting. • <20% runtime/cost reduction via optimisation and materialisation strategy. • Zero P1 data incidents and <4h MTTR with playbooks and automated remediation. • 3+ reusable templates (ETL, CDC, enrichment) adopted by partner teams. • Ontology coverage for priority domains with versioned contracts and lineage. Example Work You’ll Own • Stand up incremental CDC pipelines with watermarking & late-arrivals handling; backfill historical data safely. • Define a business-ready ontology for a domain and wire it to Workshop apps and AIP agents that trigger Actions. • Implement DQ gates (null/dup checks, distribution drift) that fail fast and auto-open incidents with context. • Build promotion workflows (dev → staging → prod) with automated tests on transforms and compatibility checks for ontology changes.