Data Engineer

Position: Data Engineer
Location: Quincy MA (Onsite)
Duration: Full-time
Salary Range: $110000 - $126000 /Annum

Must Have Technical/Functional Skills
8 years of extensive Data engineering experience in Databricks ETL/ELT using data pipelines .
SQL/Procesurr experience.
8 years of hands-on development proficiency in implementing Databricks solutions using scala and spark with data frames and notebooks/sql
8 years of hands-on development and performance tuning/ enhancement experience in scala/pyspark and sql
Must have strong experience with Databricks including developing and Eligible to workimizing spark jobs data transformations and data processing workflows
Must have experience and a good understanding on the above topics
Databricks delta lake storage
Unity catalog
Autoloader
Lakehouse architecture
Medallion architecture
Databricks serverless Eligible to workions
Dataset and dataframe
Databricks clusters configuration and sizing
Databricks job orchestration
Spark structured streaming

Roles & Responsibilities:
Must have experience with complex and large volume data streaming and data transformations using Azure Databricks
Must have extensive experience in datawarehouse / data lakehouse implementation
Must have experience with monitoring debugging and resolving issues in kive Databricks jobs
Must have strong hands on expertise in troubleshooting devops pipelines azure and AKS services
The candidate shd be able to provide data ingestion streaming and transformations solutions and do development work
Experience with apache big data implementation with programming proficiency like pyspark R or Java prior to
Databricks is advantageous
Good to have experience with containerization technologies such as docker and kubernetes

Disclaimer : Diverse Lynx LLC is an Equal Opportunity Employer. All applicants and employees are evaluated without discrimination based solely on their qualifications ability competence and performance. This email and its attachments may contain confidential or proprietary information and is intended only for the recipient(s). If you received this message in error please disregard it and notify the sender. If you no longer wish to receive our communications you may at any time.
Security Notice : Our official website is We do not operate or authorize any other websites representing Diverse Lynx LLC.

Required Experience:

IC