Senior Databricks Data Engineer

Job Title:Senior Databricks Data Engineer
Location: Chicago, IL
Duration: 6 months
Experience Required: 8-10

Summary


In this role, you will autonomously, with minimal supervision, utilize your outstanding organizational and critical thinking skills. You will leverage your experience with agile methodologies software development practices. You will solve complex problems and bring a new perspective to the usage of existing solutions.

Responsibilities :
• Design, develop, enhance, code, test, deliver, and debug software.
• Create software applications in alignment with business requirements.
• Assist in troubleshooting, root cause determination, and resolution of defects and issues in various applications.
• Ensure code quality and maintainability through code reviews and adherence to coding standards.
• Develop deliverables to meet project deadlines and fulfill functionality expectations.
• Identify opportunities to improve procedures and standards.1
• Mentor entry and mid-level developers to enhance their skills and productivity.
• Design and implement scalable data engineering solutions using Databricks and Apache Spark to support analytics and AI-driven and machine Leaning healthcare applications.
• Architect, optimize, and maintain distributed data pipelines within Databricks to ensure performance, reliability, and scalability across environments.
• Establish best practices for CI/CD, testing, monitoring, and cost optimization of Databricks workloads in cloud environments.
• Troubleshoot complex data processing and performance issues and implement scalable solutions to improve system stability and efficiency.

Qualifications :
• Relevant degree in Computer Science or a related field preferred.
• 5 or more years of relevant work experience required.
• Experience designing and implementing scalable data engineering solutions using Databricks and Apache Spark required.
• Strong expertise in Delta Lake architecture, distributed computing, and large-scale data processing frameworks.
• Experience building and optimizing ETL/ELT pipelines in Databricks within Azure cloud environments.
• Experience supporting machine learning model deployment and lifecycle management within Databricks preferred.
• Proficiency in Python or Scala for data engineering and platform development.

Role Descriptions: Databricks
Essential Skills: Databricks
Desirable Skills:
Keyword:
Skills: Digital : Databricks