Data Engineer

Senior Data Engineer This position is all about extracting RAW data. Must have solid ELT (extract, load, transact) experience. Experience you have: Production experience in Spark, with either Python or Scala Must have solid experience with ELT (extract, load, transact) Expertise in Data related Engineering You understand the tradeoffs between different types of databases and storage options Experience designing production environments Enjoy working closely with Product to build new products and support existing products 3+ years’ experience in a software engineering role Experience in Cloud Architecture, ideally AWS. Experience in cloud administration, cost management, and cloud security Experience contributing to a best-in-class engineering culture and process: version control, CI/CD, Code Reviews, QA/QC, Production engineering You have written Spark jobs and have experience working with distributed data pipelines. Bonus if you have written Spark jobs in scala You have written software designed to run in the cloud What you’ll do: Improve production workflows by creating and incorporating new features Develop a deep understanding of the company's data and our massive geospatial data pipeline Deep-dive into the infrastructure underlying our data pipeline: Spark, AWS, Databricks, and Airflow. Know the internals of these technologies. Develop fluency in Terraform, ECS, serverless apps in AWS, serverless databases, and Kubernetes Develop subject matter expertise in geospatial data analytics. Prototype open source software and the latest geospatial libraries such as Apache Sedona (geospark) and Geomasa, and demonstrate how they can be utilized in our future product pipeline Participate in the scoping and development of the next generation of our product Assist in design of next generation technical architecture for the company products Oversee (as a part of the team) the daily production deliveries and historical data sharing Help build us into a great company Zortech Solutions