Data Engineer

Data Engineer

This position is for a Data Engineer. This position sits 100% remote. They are looking for someone with pyspark developer that can build out data pipelines.

Responsible for construction and development of "large-scale cloud data processing systems". The Data Engineer must have considerable expertise in data warehousing and the job requires proven coding expertise with Python, Java, SQL, and Spark languages. Must be able to implement enterprise cloud data architecture designs, and will work closely with the rest of the scrum team and internal business partners to identify, evaluate, design, and implement large scale data solutions, structured and unstructured, public and proprietary data. The Data Engineer will work iteratively on the cloud platform to design, develop and implement scalable, high performance solutions that offer measurable business value to customers.

Required Work Experience:

• 5+ years of experience in an engineering role using Python, Java, Spark, and SQL.
• Some of the minimum experience requirement may be met with Masters or other advanced degree
• Cloud Experience Required
• Coding experience with Python, Java, Spark, and SQL
• Strong Linux/Unix background and hands on knowledge.
• Past experience with big data technologies including HDFS, Spark, Impala, Hive
• Experience with Shell scripting and bash.
• Experience with version control platform github
• Experience unit testing code.
• Experience with development ecosystem including Jenkins, Artifactory, CI/CD, and Terraform.
• Works on problems of diverse scope and complexity ranging from moderate to substantial.
• Assists senior professionals in determining methods and procedures for new tasks.
• Leads basic or moderately complex projects/activities on semi-regular basis.
• Must possess excellent written and verbal communication skills.
• Ability to understand and analyze complex data sets.
• Exercises independent judgment on basic or moderately complex issues regarding job and related tasks.
• Makes recommendations to management on new processes, tools and techniques, or development of new products and services.
• Makes decisions regarding daily priorities for a work group; provides guidance to and/or assists staff on non-routine or escalated issues.
• Decisions have a moderate impact on operations within a department.
• Works under minimal supervision, uses independent judgment requiring analysis of variable factors.
• Requires little instruction on day-to-day work and general direction on more complex tasks and projects.
• Collaborates with senior professionals in the development of methods, techniques and analytical approach.
• Ability to advise management on approaches to optimize for data platform success.
• Able to effectively communicate highly technical information to numerous audiences, including management, the user community, and less-experienced staff.
• Consistently communicate on status of project deliverables.
• Consistently provide work effort estimates to management to assist in setting priorities.

Qualifications and Education:

Master level technology degree preferred

Cloud Technology certifications preferred

Bachelor's in computer engineering or equivalent field or equivalent foreign degree required