Data Engineer

JD: Core engineering & data skills • Strong in Python, ideally using official SDKs to call REST APIs. • Experience building ETL / ingestion pipelines: incremental syncs, idempotent upserts, backfills, error handling, retries and rate-limit aware design. • Comfort with JSON schema, pagination, and large-volume data movement. Azure SharePoint / M365 integration • Practical knowledge of SharePoint Online / OneDrive: • Sites, document libraries, folders, item permissions, sensitivity labels, and how these affect what Glean can crawl and show. • Experience integrating apps with Azure AD + M365: • App registrations, Graph/SharePoint permissions, admin consent, and secure secret handling. • Ability to debug Glean SharePoint/OneDrive connector behavior using admin console status (crawl progress, indexing errors, filters). Authentication, security, and governance • Hands-on with OAuth / SSO flows (esp. Entra ID / Azure AD) and configuring SaaS integrations: • Understanding of bearer tokens, consent, scopes, redirect URIs. • Experience with Glean-issued tokens: • User vs global tokens, scopes (SEARCH, CHAT, DOCUMENTS, AGENTS, etc.), and X-Glean-ActAs for global tokens. • Ability to map source-system ACLs to Glean permissions, ensuring permission-aware search results (SharePoint/OneDrive sites, Confluence spaces, Jira projects). Custom ingestion using Glean’s APIs • Design and implementation of custom connectors where no native data source exists: • Creating data sources, pushing documents + metadata + permissions via Glean’s indexing API. • Building robust ingestion jobs: • Delta detection from upstream systems, resume/replay, dead-letter handling, and monitoring (logs/metrics) for ingest health.