Principal Software Engineer - Machine Learning

Principal Software Engineer - Machine Learning

Location: Dallas, TX/ Atlanta, GA/ Plano, TX onsite day1 (Any of these three locations can be given)

This is a senior level role where this person is responsible for the development of high performance, distributed modeling tasks using Machine Learning and Data Science.

Responsibilities include:

• Programming Languages: SQL(MySQL/ PostgreSQL), Java, Scala, Python, No-Sql technologies (Cassandra/MongoDB, Redis)
• DevOps: Docker, Kubernetes, Jenkins, CI/CD, Git, Jira, Azure DevOps
• Data exploration, analysis, summarization, visualization using necessary tools like Tableau, excel, etc.
• Experience with tools like Snowflake, Talend, and Informatica for extracting data from various sources
• Expertise in Extract, Transform, Load (ETL) processes using tools like Apache NiFi, Talend, and Informatica
• Knowledge of building and managing data pipelines with tools like Apache Kafka, Apache Flume, and Apache Storm, Apache Flink, BI Analytics and Databricks
• Experience with REST services, MQ/Rabbit, Redis/Hazelcast
• Proficiency in Python, Java, or Scala
• Understanding of data warehousing concepts and platforms like Snowflake
• Knowledge of Telecom Domain

A Machine Learning (ML) Engineer plays a crucial role in designing, implementing, and maintaining machine learning models and systems. They bridge the gap between data science and software engineering, ensuring that ML models are scalable, efficient, and integrated into production environments.

Key roles and responsibilities include:

• Model Development and Training
• Algorithm Selection: Select and implement appropriate machine learning algorithms and models based on the problem and data characteristics.
• Feature Engineering: Develop and transform features from raw data to improve model performance. This includes data preprocessing, normalization, and feature selection.
• Model Training: Train machine learning models using historical data, optimizing model parameters to achieve the best performance.
• Model Evaluation and Tuning
• Model Evaluation: Evaluate model performance using metrics such as accuracy, precision, recall, F1 score, AUC-ROC, and others. Compare different models and select the best-performing one.
• Hyperparameter Tuning: Optimize model hyperparameters using techniques such as grid search, random search, or Bayesian optimization to improve model performance and generalizability.
• Cross-Validation: Implement cross-validation techniques to ensure the robustness and reliability of the model.
• Model Deployment and Integration
• Model Deployment: Deploy machine learning models into production environments, ensuring they are scalable, efficient, and reliable.
• API Development: Develop APIs to expose machine learning models as services that can be consumed by other applications or systems.
• Integration: Integrate machine learning models with existing systems, applications, or workflows. Collaborate with software engineers and IT teams to ensure seamless deployment and integration.
• Monitoring and Maintenance
• Model Monitoring: Monitor the performance of deployed models in real-time, tracking metrics such as latency, throughput, and prediction accuracy.
• Model Maintenance: Update and retrain models as new data becomes available to ensure they remain accurate and relevant. Address issues such as model drift and data drift.
• Error Analysis: Analyze model errors and misclassifications to identify areas for improvement and refine the model.
• Infrastructure and Tooling
• Infrastructure Management: Set up and manage the infrastructure required for training and deploying machine learning models, including cloud platforms, GPUs, and distributed computing resources.
• Automation: Automate repetitive tasks such as data preprocessing, model training, and deployment using scripting languages (e.g., Python) and workflow orchestration tools (e.g., Apache Airflow).
• Tooling: Utilize and maintain ML frameworks and libraries such as TensorFlow, PyTorch, scikit-learn, and others to streamline the development and deployment process.
• Collaboration and Communication
• Cross-Functional Collaboration: Work closely with data scientists, software engineers, product managers, and other stakeholders to understand their requirements and ensure alignment on project goals.
• Documentation: Create and maintain comprehensive documentation for machine learning models, pipelines, and processes. Ensure documentation is accessible and up-to-date.
• Stakeholder Communication: Communicate progress, issues, and solutions effectively with stakeholders. Provide regular updates on machine learning activities and projects.

Programming Languages: Python, SQL, Java/Scala, shell-scripting

Cloud Technologies: Azure ML, Databricks, Snowflake and Palantir Foundry

DevOps: Docker, Azure Kubernetes Service (AKS), Jenkins, CI/CD, Git

ML Frameworks: Numpy, Pandas, Scikit-learn, OpenCV, TensorFlow, PyTorch, Hugging Face's Transformers, Spacy & NLTK

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.