Data Pipeline Architect - Spark, Python & SQL

If opportunity doesn’t knock, build a door. - Milton Berle Come build your door with us. It’s what we do here. At Paradigm, we make world-class software and deliver high-quality professional services for the building products industry. Our success is tied directly to our customers’ success, so we do what it takes to make sure they’re successful. And we know that we couldn’t do it without our awesome employees. We are looking for a Data Engineering Developer to join our team. If you love data - cleaning it, digging in it, understanding what type of insights stakeholders want from it and helping them navigate through it - let’s talk. What You Will Do: You will be responsible for creating and maintaining optimal data pipeline architecture using Spark, Python, SQL, and other tools. You will assemble large, complex data sets that meet functional/non-functional business requirements. You will work with technical and non-technical teams to provide data-driven insights. You will identify issues related to data, data sources and processing. You will monitor reporting solutions & make sure that data sets are correct & troubleshoot any issues. You will be responsible for customer issue tickets - including reviewing existing solutions and looking holistically at our services to identify trends that may need a larger review. You will document methodologies, variations, and QA processes. You will intake, identify, document, and share findings and insights to stakeholders, both internal and external. You may work cross departmentally to be a data SME for other teams. What You Need to Succeed: You have a bachelor’s degree (ideally in Comp Sci, Math, Economics, or Statistics). You have 5 years’ experience in programming or data engineering. You are very knowledgeable in Python. Knowledge of other JVM languages like Kotlin, Java and Scala, is a plus. You have advanced working SQL knowledge and experience working with relational databases, query authoring (SQL), as well as working familiarity with a variety of databases. You have a deep understanding of Apache Spark, especially as part of Azure Databricks. You are familiar with the Unity Catalog, Spark Declarative Pipelines (aka Delta Live Tables), and Delta Sharing. You have knowledge of technology related to data transformation and data structures. You think critically and are not afraid to be wrong (but you love learning from your successes and your mistakes). You have excellent requirement gathering and independent problem-solving skills. You understand AI as a tool to leverage, with a deep understanding of the required output. You have excellent communication skills (both written & verbal) - including the ability to translate technical concepts to people without technical expertise. You are organized with strong attention to detail and excellent follow-through. You are agile and able to adapt to changing demands and manage multiple priorities in a fast-paced environment. You are self-motivated and resourceful, with strong interpersonal skills; able to work individually as well as with a team. Ready to Join? Apply now! MyParadigm.com/careers #J-18808-Ljbffr