Support Data Engineer

Overview The Princeton School of Public and International Affairs seeks proposals for data engineering support focused on large-scale, text-based Internet logging data within our Databricks environment. This engagement will span from Jan - March 2026 and is expected to require approximately 20 hours/week. Objectives Ensure stability, performance, and documentation of text-based datasets approximately 25TB in size stored as Delta tables in our Databricks environment. Support ingestion of new or refreshed data, maintaining data quality, optimizing query performance, controlling compute/DBU costs, generating aggregations to assist researchers, and responding to bug fixes and other general user requests. Role will not include major changes to pipelines or existing architecture but is more in support of a steady state for these large datasets. Establish a sustainable operational foundation to minimize future engineering intervention. Required Skills Expertise in Databricks, PySpark, Delta Lake, and large-scale ETL. Familiarity with Unity Catalog, Azure/AWS storage optimization, and job orchestration. Strong debugging, documentation, and communication skills. Interested vendors should submit: A brief company profile and relevant experience. Proposed approach and timeline. Key personnel and their qualifications. Budget estimate for the 3-month engagement. References from similar projects (if available). All proposals must be submitted by December 12th. Work Location: Remote