ETL Developer

SR ETL DeveloperThe Virginia Department of Transportation (VDOT) Information Technology Division (ITD) is seeking a master data analyst with demonstrated experience in data analytics to work as a key member of the enterprise data asset team. This analyst will support teams working in agile (sprint) to analyze datasets to be made available in a cloud-based data management platform that will support the agency to produce master data with data governance. Responsibilities include analyzing source systems which contain a spatial component for candidate datasets; documenting business processes and data lifecycle; developing data requirements, user stories and acceptance criteria; and testing strategies. Develop ETL to extract business data and spatial data and load it into a data warehousing environment. Design and test the performance of the system. Consult with various teams to understand the company's data storage needs and develop data warehousing options. Deep knowledge of coding languages, such as python, Java, XML, and SQL. Well-versed in warehousing architecture techniques such as MOLAP, ROLAP, ODS, DM, and EDW.VDOT is a fast-paced organization with very high standards for work quality and efficiency. This position is expected to handle multiple projects, and remain flexible and productive, despite changing priorities and processes. Ongoing improvement and efficiency are a part of our culture, and each team member is expected to proactively contribute to process improvements.Responsibilities include:Work with the project team members and business stakeholders to understand business processes and pain pointsDevelop expertise in source system datasets and data lifecycleProfile source data which may contain a spatial component; review source data and compare content and structure to dataset requirements; identify conflicts and determine recommendations for resolutionConduct entity resolution to identify matching and merging and semantic conflictsElicit, record, and manage metadataDiagram current processes and proposed modifications using process flows, context diagrams and data flow diagramsDecompose requirements into epics and features and create clear and concise user stories that are easy to understand and implement by technical staffUtilize progressive elaboration; map stories to data models and architectures to be used by internal staff to facilitate master data managementIdentify and group related user stories into themes, document dependencies and associated business processesClient and document requirements and user stories with a focus on improving both business and technical processingAssist product owner in maintaining the product backlogCreate conceptual prototypes and mock-upsCollaborate with staff, vendors, consultants, and contractors as they are engaged on tasks to formulate, detail and test potential and implemented solutionsPerform quality analyst functions such as defining test objectives, test plans and test cases, and executing test casesCoordinate and facilitate user acceptance testing with business and ensure project managers/scrum masters are informed of the progressDesigns and develops systems for the maintenance of the data asset program (data hub), ETL processes, ETL processes for spatial data, and business intelligence.Develop a new data engineering process that leverages a new cloud architecture and will extend or migrate existing data pipelines to this architecture as needed.Design and supports the DW database and table schemas for new and existing data sources for the data hub and warehouse. Design and development of data marts.Work closely with data analysts, data scientists, and other data consumers within the business in an attempt together and populate data hub and data warehouse table structure, which is optimized for reporting.The data developers partner with data modeler and data architect in an attempt to refine the business's data requirements, which must be met for building and maintaining data assets.Qualifications required include:Minimum of 10 years of experience delivering business data analysis artifacts5+ years of experience as an agile business analyst; strong understanding of Scrum concepts and methodologyExperience organizing and maintaining product and sprint backlogsExperience translating client and product strategy requirements into dataset requirements and user storiesProficient with defining acceptance criteria and managing acceptance processExceptional experience writing complex SQL queries for SQL Server and OracleExperience with Azure Databricks, Azure data factory, snowflakeExperience with ESRI ArcGISExperience with enterprise data managementExpertise with Microsoft Office products (Word, Excel, Access, Outlook, Visio, PowerPoint, Project Server)Experience with reporting systems operational data stores, data warehouses, data lakes, data martsExceptional written and oral communications skills and proven ability to work well with a diverse set of peers and customersPreferred skills include:Advanced understanding of data integrationsStrong knowledge of database architecturesStrong analytical and problem solving skillsAbility to build strong relationships both internally and externallyAbility to negotiate and resolve conflictsAbility to effectively prioritize and handle multiple tasks and projectsStrong written and verbal communication skillsDesire to learn, innovate and evolve technologyComputer skills/MS Office/software:Excellent computer skills and be highly proficient in the use of MS Word, PowerPoint, MS Excel, MS Project, MS Visio, and MS Team Foundation Server, which will all be necessary in the creation of visually and verbally engaging ETL, data designs and tables as well as the communication of documentation and reporting.Deep passion for data analytics technologies as well as analytical and dimensional modeling. The candidate must be extensively familiar with ETL (Extraction, Transformation & Load), data warehousing, and business intelligence tools such as business objects, PowerBI and Tableau.The candidate must also have vast knowledge of database design and modeling in the context of data warehousing.Experience with key data warehousing architectures including Kimball and Inmon, and has a broad experience designing solutions using a broad set of data stores (e.g., HDFS, Azure Data Lake Store, Azure Blob Storage, Azure SQL Data Warehouse, Azure Cosmos DBTechnologies required:Data Factory v2, Data Lake Store, Data Lake Analytics, Azure Analysis Services, AZURE SynapseIBM Datastage, Erwin, SQL Server (SSIS, SSRS, SSAS), ORACLE, T-SQL, Azure SQL Database, Azure SQL Datawarehouse.Operating system environments (Windows, Unix, etc.)Scripting experience with Windows and/or Python, Linux shell scripting