Data Engineer

Data Engineer

Location: Houston, TX (Onsite) Duration: Contract Experience: 10+ Years

Skills | Must-Haves:

• .NET and C# experience
• Spark experience
• experience with large data sets
• PowerBI experience
• experience using CI/CD DevOps pipelines

Job Description Skills/Domain: Microsoft Azure, Synapse, Spark, Python, Angular, C#,.NET, DevOps, Azure Function, Microservice/API Development, Power BI

Roles and Responsibilities:

• Promotes and adheres to Client's architectural standards and guidelines and works closely with the Enterprise Architects to develop target architectures and technology standards
• Provides explicit architecture consulting to project teams, across corporate functions, understands and frames the business problem to be solved. Identify available problem-solving approaches and methods
• As part of an Agile delivery approach, works with the product owner to write epics and stories to be delivered and participate in Agile SAFe delivery via standups, sprint planning and PI planning.
• Develop solutions by utilizing Microsoft Synapse,.Net and Spark to retrieve data from on-premises systems, manipulate large datasets, and perform intricate data wrangling including logging, error handling and rerun capabilities
• Implement role based and OBO authentication to enhance security management for the business.
• Develop CI/CD DevOps pipelines using ansible roles to enable continuous deployment and regression tests.
• Develop Data Manager Application using Angular and NgRx.
• Design synapse data warehouse using data modelling guidelines to enable optimal performance for Power BI reporting and develop Power BI reports
• Develop Rest APIs following REST guidelines for interacting with source systems and ingesting data into Azure Data Manager for Energy.