Big Data Developer Job Description Template
Our company is looking for a Big Data Developer to join our team.
Responsibilities:
- Work closely with Architecture group in delivering technical solutions;
- Design and implement product features in collaboration with business and IT stakeholders;
- Define Standard Output constructs which allow business users to extract their own data consistently across business units where needed;
- Coordinate and interact with technical and business groups of client team;
- Build scalable data pipelines on top of Hive and Spark using the Airflow scheduler/executor framework;
- Demonstrate substantial depth of knowledge and experience in a specific area of Java, Data Streaming, Big Data, Spark, Scala, Kubernetes etc;
- Create complex views and materialized views and tune for performance;
- Define data/information architecture standards, policies and procedures for the organization;
- Define Standard Integration points for data being captured into the Hadoop Data Platforms;
- To run data aggregation either directly from Kafka or from Hbase;
- Demonstrate substantial depth of knowledge and experience in a specific area of Java, Data Streaming, Big Data, Spark, Scala etc;
- Provides consultation on complex projects and is considered to be a top-level contributor;
- Apply Machine Learning models to forecast and visualize key revenue metrics and trends;
- Relies on extensive experience and judgment to plan and accomplish goals according to timelines and budgets agreed upon with business stakeholders;
- Work with data and analytics experts to strive for greater functionality in our data systems.
Requirements:
- Bachelor’s Degree in Computer Science, Computer Engineering or a closely related field;
- Scala;
- Spark;
- Hbase;
- Extensive experience architecting and engineering BigData processing applications using Spark, Hive;
- Participate in requirement gathering and create design documents;
- Certification in Hadoop preferred;
- Practical knowledge ofScala or similar language;
- Systems or equivalent experience;
- Good working knowledge of data warehouse, database design, Microsoft Office Products and Microsoft Vision;
- Familiarity with both waterfall and iterative development methodologies;
- Proficient in Kafka Streaming, PostgreSQL, Spark, Scala, Java;
- 4+ years of IT experience;
- Strong analytical and diagnostic skills;
- Participate in day to day activities involving design, development, test support, deployment, production monitoring for Order Management platform.