Lead Data Engineer (Databricks / AI Solutions) - Washington, D.C. (Hybrid)

Lead Data Engineer (Databricks / AI Solutions) – Washington, D.C. (Hybrid)

Randstad is seeking a highly skilled Lead Data Engineer to architect and deliver next-generation data and AI solutions for a premier client in the transportation sector. In this hybrid, hands-on leadership role, you will drive the design and deployment of enterprise-scale Databricks-based data platforms while mentoring a team of engineers. You will act as the critical link between executive vision and technical execution—ensuring scalable, secure, and production-ready AI systems that align with enterprise governance and modernization goals.

Key Responsibilities

• Technical Leadership
• Lead the end-to-end architecture, design, and implementation of enterprise data and AI solutions on Databricks
• Define best practices for scalable data engineering, AI workflows, and cloud-native architectures
• Hands-on Development
• Build and optimize complex data pipelines using Python, Scala, or Java
• Design and implement robust ETL/ELT frameworks and real-time/batch processing systems
• MLOps & AI Delivery
• Oversee model deployment pipelines and MLOps practices from development through production
• Ensure seamless integration of machine learning models into enterprise systems
• Team Mentorship
• Provide technical guidance through code reviews, pair programming, and coaching
• Foster a high-performance engineering culture focused on quality and innovation
• Stakeholder Collaboration
• Partner with architects, product owners, and governance teams to align solutions with business strategy and compliance requirements
• Translate business needs into scalable technical solutions
• Optimization & Governance
• Improve platform performance, reliability, observability, and data quality
• Ensure adherence to enterprise data governance, security, and compliance standards

Required Qualifications

• Experience: 6–8 years in data engineering, AI systems, or cloud data platform development with increasing leadership responsibility
• Technical Skills:
• Advanced proficiency in SQL and Python (or Scala/Java)
• Strong experience with Databricks ecosystem
• Expertise in building scalable data pipelines and distributed systems
• Architecture: Proven experience designing enterprise-scale data integration and AI-driven solutions
• MLOps: Hands-on experience with model deployment, CI/CD pipelines, and production AI systems
• Methodologies: Strong understanding of Agile development practices and data governance frameworks
• Education: Bachelor's degree in Computer Science, Information Systems, or related field (or equivalent experience)

Preferred Skills

• Experience in cloud platforms (Azure, AWS, or GCP)
• Knowledge of streaming architectures (Kafka, Spark Streaming)
• Exposure to data governance tools and frameworks
• Experience working in transportation, logistics, or large-scale enterprise environments

Soft Skills

• Strong analytical and problem-solving abilities
• Excellent communication skills with both technical and non-technical stakeholders
• Ability to balance hands-on engineering with strategic leadership
• Proven ability to thrive in fast-paced, complex environments