Senior Data Engineer

Job Description Ciklum is looking for a Senior Data Engineer to join our team full-time in Poland. We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and product owners, we engineer technology that redefines industries and shapes the way people live. About The Role As a Senior Data Engineer, become a part of a cross-functional development team engineering experiences of tomorrow. The Legacy Code Semantic Documentation Project is a 26-week enterprise initiative for a global industrial automation leader. The primary objective is to engineer an automated, AI-assisted pipeline to generate structured, system-level Markdown documentation directly from an undocumented ~400K LOC codebase (spanning IEC 61131-3 languages and ANSI C/C++). Operating within a dedicated, zero-data-egress secure tenant, the technical architecture combines Tree-sitter AST parsing, Neo4j knowledge graphs, Qdrant vector search, and self-hosted open-weight LLMs (Llama 3.1 / Mixtral family) with NLI-based validation. Delivery is structured across 7 work packages executing over a 6-month period, governed by strict contractual KPIs: ≥95% code coverage, ≥92% NLI-verified factual precision, and ≥90% SME validation acceptance. All outputs align with EU Cyber Resilience Act (EU CRA) requirements for SBOM and source-level traceability. Responsibilities Data Pipeline Architecture: Build and optimize high-throughput data processing pipelines linking static code analysis outputs, graph databases, vector indexes, and LLM orchestration layers Knowledge Graph Engineering: Design and maintain Neo4j graph schemas to map code dependencies, function calls, and module hierarchies across 400K+ LOC AST Integration: Integrate Abstract Syntax Tree (AST) parser outputs (Tree-sitter) into downstream knowledge graphs and vector embeddings (Qdrant) Data Lineage & Integrity: Implement bidirectional cross-reference tracking and graph snapshot versioning to ensure data consistency across batch documentation runs Pipeline Testing & Monitoring: Establish data validation protocols and automated testing for pipeline health, throughput, and state persistence Technical Collaboration: Partner closely with AI/ML Engineers and Architects to optimize retrieval quality and context assembly for RAG pipelines Requirements Professional Experience: 5+ years of Data Engineering experience building complex, production-grade data pipelines and ETL/ELT workflows Domain Specialization (Must cover at least ONE key domain): Option 1 (Static Analysis Focus): Hands-on experience with Abstract Syntax Tree (AST) parsing, static code analysis (e.g., Tree-sitter, language grammars), and code structure modeling Option 2 (Knowledge Graph Focus): Direct experience with graph database design and optimization (Neo4j/Cypher), documentation processing pipelines, and vector search integration (Qdrant) Core Technical Stack: High proficiency in Python, graph query languages, vector storage, and modern data engineering toolkits Infrastructure & Deployment: Familiarity with containerized execution (Docker, Kubernetes) in secure, private, or air-gapped environments Quality & Engineering Standards: Strong focus on code quality, automated pipeline testing, CI/CD, and robust versioning Language: Full professional proficiency in English (C1+) What’s in it for you? Strong community: Work alongside top professionals in a friendly, open-door environment Growth focus: Take on large-scale projects with a global impact and expand your expertise Tailored learning: Boost your skills with internal events (meetups, conferences, workshops), Udemy access, language courses, and company-paid certifications Endless opportunities: Explore diverse domains through internal mobility, finding the best fit to gain hands‑on experience with cutting‑edge technologies Flexibility: Enjoy flexibility – full remote working possibilities Care: We've got you covered with company-paid premium medical package through Luxmed Benefits: Access the MyBenefit cafeteria platform, allowing you to choose perks that best suit your lifestyle and needs About Us At Ciklum, we are always exploring innovations, empowering each other to achieve more, and engineering solutions that matter. With us, you’ll work with cutting-edge technologies, contribute to impactful projects, and be part of a One Team culture that values collaboration and progress. With delivery centers in Wrocław and Gdańsk, our 300+ professionals in Poland drive forward-thinking solutions for global clients. Join a community where collaboration sparks innovation—and your impact reaches millions. Explore, empower, engineer with Ciklum! #J-18808-Ljbffr Ciklum