Sr Data Engineer

Sr Data Engineer

Location: Philadelphia, PA/ Durham, NC

Core engineering data skills. Strong in Python ideally using official SDKs to call REST APIs. Experience building ETL/ingestion pipelines: incremental syncs, idempotent upserts, backfills, error handling, retries and rate-limit aware design. Comfort with JSON schemas, pagination, and large-volume data movement.

Azure SharePoint / M365 integration. Practical knowledge of SharePoint Online / OneDrive: Sites, document libraries, folders, item permissions, sensitivity labels, and how these affect what Glean can crawl and show. Experience integrating apps with Azure AD + M365: App registrations, Graph/SharePoint permissions, admin consent, and secure secret handling. Ability to debug Glean SharePoint/OneDrive connector behavior using admin console status (crawl progress, indexing errors, filters).

Authentication, security, and governance. Hands-on with OAuth / SSO flows (esp. Entra ID/Azure AD) and configuring SaaS integrations: Understanding of bearer tokens, consent, scopes, redirect URIs. Experience with Glean-issued tokens: User vs global tokens, scopes (SEARCH, CHAT, DOCUMENTS, AGENTS, etc.), and X-Glean-ActAs for global tokens. Ability to map source-system ACLs to Glean permissions, ensuring permission-aware search results (SharePoint/OneDrive sites, Confluence spaces, Jira projects).

Custom ingestion using Glean's APIs. Design and implementation of custom connectors where no native datasource exists: Creating datasources, pushing documents + metadata + permissions via Glean's Indexing API. Building robust ingestion jobs: Delta detection from upstream systems, resume/replay, dead-letter handling, and monitoring (logs/metrics) for ingest health. Using Client API search/chat to verify indexed content and permissions as part of QA tooling. Prior work integrating other enterprise SaaS (ServiceNow, Salesforce, GitHub, etc.) to appreciate patterns around connectors, webhooks, and rate limits.

AI tools may assist in the recruitment process; however, all hiring decisions are made by the recruitment team based on a comprehensive evaluation of candidates. *** is an Equal Employment Opportunity employer. We promote and support a diverse workforce at all levels of the company. All qualified applicants will receive consideration for employment without regard to race, religion, color, sex, age, national origin, or disability. All applicants will be evaluated solely based on their ability, competence, and performance of the essential functions of their positions with or without reasonable accommodations. Reasonable accommodation also is available in the hiring process for applicants with disabilities. Candidates can request a reasonable accommodation by contacting the company ADA Coordinator at ***.