Big Data Software Engineer

In this key role as a Software Engineer, you'll get to transform a Streamlit-built UI prototype into a polished, production-ready interface, with a roadmap to extend its reach and build it into an interface with future plans to make the interface available external to the corporate enterprise.

 

Requirements

What You’ll Be Doing (We don’t love the bullet points, but we love the work!)

Required Skills and Experience (SW Swiss Army knife with a keyboard)
• Using the Linux CLI and Linux tools
• Developing Bash scripts to automate manual processes
• Recent software development experience using Python and Java
• Familiar with Streamlit, a Python framework for building interactive web applications
• Experience developing front-end applications using TypeScript, HTML, and CSS
• Experience with JavaScript frameworks such as React, Angular, and Vue
• Familiar with Distributed Big Data processing engines including Apache Spark 
• Experience using Jupyter Notebook
• Experience with data wrangling and preprocessing using tools such as pandas, NumPy
• Experience working with structured, semi-structured, and unstructured data such as Parquet, JSON, CSV, XML
• Familiarity with data quality concepts, data validation, and anomaly detection
• Experience with Git Source Control System

Position Desired Skills 
• Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows
• Experience with SQL technologies such as MySQL, MariaDB, and PostgreSQL for querying, joining, and aggregating large datasets
• Familiar with HPC Job Scheduling tools including Slurm
• Experience using the Atlassian Tool Suite (JIRA, Confluence) 
• Appreciates a sense of humor and the occasional well-timed joke.