Senior Data Engineer

Senior Data Engineer

At Dematic, we are revolutionizing our data landscape in support of cutting-edge Artificial Intelligence (AI) use cases. We are forming multiple teams that will spearhead the creation of the platform's foundational components. These teams go beyond traditional data ingestion; they are architects of a microservices-driven platform, providing abstractions that empower other teams to seamlessly extend the platform. We are seeking a dynamic and highly skilled Senior Data Engineer who has extensive experience building self-service enterprise scale data platforms with microservices architecture and leading these foundational efforts. This role demands someone who not only possesses a profound understanding of the data engineering landscape but also has experience with software engineering design patterns and microservices frameworks. The ideal candidate will be 100% hands-on, deep in the code and individual contributor who will contribute significantly to platform development and actively shape our data ecosystem.

What You Will Do in This Role:

• As a senior data engineer, you will be responsible for ideation, architecture, design and development of our key data platform components.
• Create and maintain essential data platform SDKs and libraries, adhering to industry best practices.
• Design and develop connector frameworks and modern connectors to source data from disparate systems both on-prem and cloud.
• Design and optimize data storage, processing, and querying performance for large-scale datasets using industry best practices while keeping costs in check.
• Design and develop data quality frameworks and processes to ensure the accuracy and reliability of data.
• Collaborate with data scientists, analysts, and cross functional teams to design data models, database schemas and data storage solutions.
• Proactively identify and contribute towards platform resiliency.
• Design and develop observability and data governance frameworks and practices.
• Stay up to date with the latest data on engineering trends, technologies, and best practices.
• Drive the deployment and release cycles, ensuring a robust and scalable platform.
• Partner with AI enablement teams across the organization as well as KION IT Cloud Infrastructure, AI Platform, and security teams to ensure AI/ML capabilities align with IT framework and guidelines within KION.
• Partner with Business Transformation Data Management teams to align on master data.

What We Are Looking For:

• 5+ years of proven experience in modern cloud data engineering and software engineering.
• Proven ability to build end-to-end data platforms and data services (beyond ETL).
• Strong cloud experience, preferably GCP; Azure (ADLS) and Databricks are strong pluses.
• Hands-on experience with Databricks (Delta Lake, Spark, ML/ETL workflows).
• Experience working in multi-cloud environments (GCP / Azure).
• Proficiency with platforms such as BigQuery, Dataflow, Dataform, Cloud Run, DBT, Dataproc, SQL, Python, Airflow, Pub/Sub, and equivalent Azure tooling such as ADLS, Azure Functions, and Databricks.
• Experience with microservices architectures (Kubernetes, Docker).
• Deep experience with batch and streaming data infrastructures.
• Strong hands-on experience with metadata management, data catalogs, data lineage, data quality, and data observability frameworks.
• Strong understanding of data modeling, data architecture, and data governance.
• Solid experience with DataOps, CI/CD, and test automation.
• Excellent experience with observability tooling.
• Experience building data platforms supporting AI use cases and machine learning
• Production level experience with universal semantic layers.
• Production level experience with implementing either 3rd party or open-source metadata management platforms.
• Production level experience with data platform resiliency.
• Experience building large-scale data platforms on Azure, including Azure Data Lake and Databricks.