Sr. Big Data Engineer

Software Engineer IV - Big Data Engineer

Location: Durham, NC (Day 1 onsite)

Duration: Long Term

Responsibilities:

• Ability to assemble large, complex sets of data that meet non-functional and functional business requirements
• Ability to identify, design and implement internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes
• Build required infrastructure for optimal extraction, transformation and loading of data from various data sources using Azure and SQL technologies
• Building analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition
• Working with stakeholders including data, design, product and executive teams and assisting them with data-related technical issues
• Strong programming and analytical skills, problem determination and debugging, attention to detail, teamwork, and excellent communication skills.
• Knowledge of software development concepts/methodology/life cycle, especially Agile methodology
• Proficiency with code versioning tools like Git
• Proficiency with Windows- and Linux-based operating systems, including software installation, light administration and simple scripting.

Required Skills:

• Bachelor’s Degree in Computer Science or related field
• Strong analytical and Java programming skills
• Deep technical skills in software development technologies and tools such as socket communication, Python, DI, CI, Spring Boot, XML/JSON, Java, JavaScript, Docker, Dagger2, maven
• Experience in Cloud and Big Data technologies such as PowerShell, SQL, ADLS/Blob, Spark/SparkSQL, Databricks.
• 8+ years professional experience in software development
• 3+ years of cloud based or client/server development experience.
• 1+ years of Big Data development experience.

Preferred Skills:

• Experience implementing data migration and data processing using Azure services: ADLS, Azure Data Factory, Azure Functions, Synapse/DW, Azure SQL DB, Event Hub, IOT Hub, Azure Stream Analytics, Azure Analysis Service, HDInsight, Databricks Azure Data Catalog, Cosmo Db, ML Studio, AI/ML, etc.
• Experience in developing data ingestion, data processing and analytical pipelines for big data, relational databases, NoSQL and data warehouse solutions
• Experience in Big data partitioning strategies, Azure ADF, Azure Databricks.
• Knowledge of cloud computing technologies
• Knowledge of non-relational databases
• Experience working on Azure SQL Server.