Data Engineer

Title- Data Engineer
Location - Redmond, WA (Open for WFM -Need to work in PST time zone)
Skills-:
Design, develop, and maintain scalable big data solutions using
Apache Spark
and
Scala .
Implement complex
SQL
queries for data transformation and analytics.
Develop and optimize
Power BI dashboards
for business reporting and visualization.
Collaborate with cross-functional teams to integrate data pipelines and reporting solutions.
Ensure data quality, security, and compliance across all systems.
Required Skills
Experience
Strong proficiency in Apache Spark
with hands-on experience in
Scala .
Solid understanding of
SQL
for data manipulation and analysis.
Experience in
Power BI
for creating interactive reports and dashboards.
Familiarity with distributed computing concepts and big data ecosystems (Hadoop, Hive, etc.).
Ability to work with large datasets and optimize data workflows.
Highly Desirable
Spark Performance Tuning Expertise : Proven ability to optimize Spark jobs for efficiency and scalability.
Knowledge of cluster resource management and troubleshooting performance bottlenecks.
Experience with Azure cloud services
for big data solutions (e.g., Azure Data Lake, Azure Databricks, Synapse Analytics).
Exposure to other cloud platforms (AWS or GCP) is a plus.
Experience working in financial domain or with ERP systems
(e.g., SAP, Oracle ERP).
Understanding of compliance and regulatory requirements in financial data processing.
Additional Qualifications
Strong problem-solving and analytical skills.
Excellent communication and collaboration abilities.
Bachelor's or master's degree in computer science, Engineering, or related field.
Questioner is mandate while submitting the profiles- Update on the resume while submitting the profile...
Skill sets reviewed (All Rating are out of 5)
[Associate Name]
Scala - Hands on programming experience
Spark - Hands on programming experience, Spark Architecture, Debugging, Optimizing
ADF, Synapse
SQL (Queries, Stored Proc, Backup)
Azure Fabric (One Lake, Semantic Model), Delta Lake
Power BI
Azure
Microsoft Ecosystem Expertise
Scala
Spark
Can you describe a project where you used Scala with Apache Spark, and explain how you approached debugging and performance optimization?
ADF
Synapse
Walk me through how you have designed or orchestrated data pipelines using Azure Data Factory or Synapse, highlighting any challenges you solved.
SQL Expertise
Tell me about a time you wrote complex SQL queries or stored procedures, and how you managed database backups or recovery in that environment.
Azure Fabric
Delta Lake
What is your understanding of Azure Fabric's OneLake and Semantic Model, and how have you applied Delta Lake to ensure reliability and scalability in data solutions?
Power BI
Microsoft Ecosystem
How have you integrated Power BI with Azure services such as Synapse, ADLS, or Delta Lake, and what benefits did this bring to the overall Microsoft ecosystem solution