Senior Dataiku Platform Engineer

Job Description

Job Description

ComResource is looking for a Senior Dataiku Platform Engineer.

We need someone to lead the engineering, automation, governance, and enterprise integration of our Dataiku DSS platform within our modern data ecosystem, including Snowflake, AWS, dbt, and Microsoft SharePoint.

This is a hands-on platform engineering role, not a traditional data science position. The engineer will strengthen platform reliability and security, automate deployment and administrative processes, and establish governance-as-code practices that enable business teams, including Finance, to develop and operationalize data products safely.

The ideal candidate will have extensive experience with Dataiku DSS administration, Python-based automation, APIs, deployment engineering, and enterprise integrations, with the ability to independently design, implement, and support production-grade solutions.

Responsibilities:

Platform Engineering and Administration

• Administer and maintain Dataiku DSS environments, including Design and Automation nodes, connections, code environments, plugins, permissions, and platform configurations.
• Lead platform upgrades, including planning and supporting the upgrades.
• Troubleshoot complex platform issues, analyze logs, perform root-cause analysis, and improve platform stability and performance.

Deployment Automation and Governance

• Enhance existing Python and Dataiku API-based automation to streamline project promotion from Design through Deployer to Automation.
• Automate bundle creation, publishing, deployment, configuration verification, and post-deployment health checks.
• Validate connections, code environments, plugin versions, deployment bundles, and Project Standards before deployment, blocking releases when critical checks fail.
• Implement governance-as-code through Project Standards, automated data quality checks, and production-readiness controls.
• Improve automation reliability through structured logging, error handling, credential management, and repeatable execution.

Security and Identity Automation

• Automate onboarding and offboarding of users, teams, connections, and functional accounts using Dataiku APIs.
• Support migration to Okta/Active Directory group-based entitlements and automate cleanup of obsolete access.
• Integrate HashiCorp Vault for secure API key retrieval and partner with Information Security on Fortanix tokenization and field-level encryption.
• Maintain secure enterprise integrations, including Microsoft SharePoint Online certificate-based authentication and credential rotation.

Snowflake and Enterprise Integrations

• Build and maintain Dataiku integrations with Snowflake and AWS to support enterprise data products.
• Automate deployment of Data Products from Design to Automation, including connection validation, permissions, and post-deployment verification.
• Support integration with dbt, Snowflake Cortex AI, Dataiku data lineage, and the enterprise Data Exchange marketplace.
• Partner with data engineering and architecture teams to establish reusable patterns for secure, governed data products.

Operational Excellence and Platform Enablement

• Integrate Dataiku monitoring and pipeline failure notifications with ServiceNow to improve incident management and operational visibility.
• Automate license reclamation, cleanup of unused projects and scenarios, disabled-user cleanup, and platform configuration checks.
• Establish monitoring, documentation, operational procedures, and repeatable support practices.
• Support Finance users through knowledge transfer, technical office hours, Lunch & Learn sessions, and guidance on production-ready Dataiku solutions.
• Coordinate with EDO, EuOps, Information Security, Finance, and Snowflake administration teams to resolve dependencies and deliver platform improvements.

Essentials:

• Extensive hands-on experience administering and supporting enterprise Dataiku DSS environments, including Design and Automation nodes.
• Strong Python programming and experience using Dataiku REST APIs/SDK for platform administration and automation.
• Proven experience with Dataiku bundles, Deployer, deployment workflows, and automated promotion between environments.
• Experience managing Dataiku connections, code environments, plugins, project permissions, and Project Standards.
• Experience implementing automated validation, governance controls, and production deployment checks.
• Strong AWS and Snowflake integration experience, including connectivity, authentication, and troubleshooting.
• Linux/Unix administration, scripting, log analysis, production support, and root-cause analysis.
• Understanding of enterprise security, RBAC, API keys, secrets management, and secure integration patterns.
• Ability to independently lead technical implementation and collaborate across infrastructure, security, operations, and business teams.

Desired:

• HashiCorp Vault, Fortanix, Okta/Active Directory, and enterprise identity lifecycle automation.
• Terraform or other infrastructure-as-code tools.
• GitHub, Jenkins, CI/CD pipelines, automated testing, and release management.
• ServiceNow integration, monitoring, and observability tools such as Splunk, Dynatrace, or AWS CloudWatch.
• Microsoft SharePoint Online certificate-based authentication and integration.
• dbt, Snowflake Cortex AI, data lineage, and enterprise data marketplace integration.
• Dataiku plugin development, advanced deployment automation, and data quality frameworks.
• Tesseract OCR, parallel processing, or document-processing workloads.

Req ID: AM6189331021