ML Architect with Data Bricks and Azure ADF

ML Architect With Data Bricks And Azure ADF

Location: Santa Clara, CA - onsite

Duration: Long term

Must Have Skills

Skill 1 – ML Architect with Experience on Data Bricks Development

Skill 2 - Azure Data factory, Azure Databricks, Azure Datalake

Skill 3 – Have good experience configuring Microservices using Docker, Kubernetes on Azure Data

Good To Have Skills –

Domain Experience (If any) – Data science

Client Job Description:

• Prior Experience Hands on experience on working in Azure cloud platform.
• Experience in designing and implementing scalable and secure Azure cloud solutions using Data Bricks ML platform.
• Understanding of implementation of architecture within Azure Big Data Analytics Al tools.
• Expert level in Designing and Architect solutions in Azure Data factory, Azure Databricks, Azure Datalake, Delta Lake and Azure synapse analytics implementation.
• Experience in Azure cloud technologies like PySpark, Synapse, ADF, Databricks, Python, Scala and SQL.
• Have good experience configuring Microservices using Docker, Kubernetes on Azure Data Bricks Extensive Experience on working on Azure AI services including Data Bricks and Azure cognitive services.
• Highly preferred experience on working with Azure OpenAI services. Hands-on experience on design, and optimizing LLM, natural language processing (NLP) systems, frameworks, and tools.
• Hands-on AI/ML modeling experience of complex datasets combined with a strong understanding of the theoretical foundations of AI/ML.
• Expertise in most of the following areas: supervised & unsupervised learning, deep learning, reinforcement learning, federated learning, time series forecasting, Bayesian statistics, and optimization.
• Experience in creating and deploying code libraries using functions and classes in Python in AI product-focused development.
• Well-versed in software development and code quality best industry practices. Experience working with Agile Development methodologies.

Responsibilities:

• Your primary responsibility is to Architect, design and develop AI application on Azure cloud. Lead the azure cloud architecture design for various applications depending on business requirements.
• Prototypes and do proof of concepts (PoC) in one or all the following areas:
• LLM, NLP, DL (Deep Learning), ML (Machine Learning), object detection/classification, tracking, etc.
• Stay up to date with the latest advancements in LLM, NLP, deep learning, machine learning, and object detection algorithms, and proactively identify opportunities to leverage new technologies for improved solutions.
• You will work with a team of researchers and engineers to build state-of-the-art language models that can understand and generate human-like text.
• Work with business users to translate functional requirements into data specifications Collaborate with data scientists and business users to transform domain know-how into functional requirements.

Data Analysis and Preprocessing:

• Collect and preprocess large-scale text corpora to train and fine-tune language models.
• Conduct data analysis to identify patterns, trends, and insights that can inform model development and improvement.

Language Model Development:

• Utilize off-the-shelf LLM services, such as Azure OpenAI, to integrate LLM capabilities into applications, and collaborate with the research team to design and develop state-of-the-art language models.

Model Training and Evaluation:

• Train and fine-tune language models using appropriate machine-learning frameworks and tools. Research and Innovation:
• Stay up to date with the latest advancements in the field of natural language processing and machine learning.
• Collaboration and Communication: Collaborate effectively with cross-functional teams, including researchers, DevOps architects/engineers and project managers.
• Communicate complex ideas and technical concepts clearly to stakeholders and team members supported by Large Language Model (LLM) technology.

Documentation and Knowledge Sharing:

• Document code, algorithms, and processes to ensure effective knowledge transfer and maintain codebase quality.

Requirements:

• B.E./ B. Tech / M. Tech/ MCA (Computer Science/Electronics & Communication/Electrical), a master's in computer science, artificial intelligence, or a related field, or a specialization in natural language processing is preferred.
• 4-6 years of IT experience with a min of 3+ years in Data Science (AI/ML) with a specialization in NLP and Azure Data Bricks for development and Azure Services.
• Strong Expertise in the Azure cloud solutions. Strong programming skills in Python.
• Experience with deep learning frameworks (e.g., TensorFlow, PyTorch) is highly desirable. Solid understanding of natural language processing techniques, including language modeling, text classification, sentiment analysis, and machine translation.
• Familiarity with statistical and machine learning approaches, such as neural networks, recurrent neural networks (RNN), and transformer models.
• Experience with large-scale data processing, including data collection, cleaning, and preprocessing.
• Proficiency in working with UNIX/Linux environments and command-line tools. Strong analytical and problem-solving abilities.
• Excellent communication skills, interpersonal, written, visual (presentations), and verbal.