Senior Data Platform Engineer

Company Name New Vision Softcom & Consultancy Pvt. Ltd

Function Business Units (BU)

Department/Practice AI & Analytics

Region USA

Country US

Base office location San Francisco

Working model Hybrid

Weekly Off US standard

State California

Skills

Skill

Platform Development Platform Architecture Platform Integration Database Architecting Performance Tuning Big Data

CERTIFICATION

Working Language

Job Description

Location - US Remote. Candidate to be based out of US.

Mandatory Skills

-designing, optimizing, and managing enterprise-grade data ecosystems.

-mastery of database architecture, performance tuning, and transaction management,

- hands-on experience in cloud-native platforms, big data frameworks, and advanced analytics pipelines.

The ideal candidate will be comfortable working across distributed systems, modern data engineering tools, and collaborating with architects, DevOps, and business teams to deliver scalable, secure, and high-performing solutions.

KEY RESPONSIBILITIES

• Design and implement robust, scalable, and secure database architectures for transactional and analytical workloads.
• Define partitioning strategies, indexing, and normalization for optimal performance and maintainability.

Performance Tuning & Optimization

• Conduct advanced query optimization and execution plan analysis.
• Implement proactive monitoring and troubleshooting for high-volume, mission-critical systems.
• Deploy and manage microservices-based applications on Azure Service Fabric clusters.
• Ensure high availability, fault tolerance, and scalability of distributed services.
• Optimize communication patterns and stateful/stateless service design for data-intensive workloads.
• Design and orchestrate complex ETL/ELT pipelines for batch and streaming data.
• Build and optimize big data processing workflows using Apache Spark on Databricks.
• Implement Delta Lake for ACID transactions on large-scale data lakes.
• Develop scalable solutions for machine learning and advanced analytics.
• Manage versioned data storage for reliability and reproducibility.
• Optimize schema evolution and data compaction strategies.
• Collaborate with BI teams to design semantic models and optimize data for reporting.
• Ensure data pipelines deliver clean, consistent, and performant datasets for visualization.
• Work closely with architects, DevOps, and business stakeholders to align technical solutions with strategic objectives.
• Provide mentorship and technical guidance to junior engineers and developers.

REQUIRED SKILLS & QUALIFICATIONS

Database Mastery

• Advanced SQL, query optimization, partitioning, and transaction handling.

Performance Tuning

• Proven ability to diagnose and resolve complex performance bottlenecks.
• Hands-on experience with Azure Service Fabric, Azure Data Factory, Azure Databricks, and Azure Storage.
• Strong knowledge of Spark, Delta Lake, and distributed data processing.
• Familiarity with Power BI and data modeling best practices.

Troubleshooting & Optimization

• Ability to resolve issues across multi-tiered systems under pressure.

Cross-Functional Collaboration

• Excellent communication and stakeholder management skills.

PREFERRED QUALIFICATIONS

• Experience with CI/CD pipelines for data solutions.
• Familiarity with data governance, security, and compliance frameworks.
• Exposure to machine learning workflows and real-time streaming architectures.