Data Engineer

Job Title: Data Engineer


Location: US Remote (Candidate must be in EST)


Duration: Full time with Altimetrik

Job Description:


Skills: Must be 5 to 10 years of experience


a. SQL (expert)


b. Snowflake - not a roadblock (added advantage)


c. Cloud - AWS is preferred (exp on any cloud)


d. Python - intermediate


e. Databricks - added advantage.

The Data Engineer will be part of our Global Fulfillment team, building enterprise data streams, ingesting data from multiple data sources, and provisioning data as raw data streams and canonical business events. We are working towards integrating our Fulfillment data sources into our federated enterprise data. The engineer would also be responsible for continuously identifying opportunities for automation and reduction of technical debt & manual data manipulations to create efficiencies & optimize data delivery. They should also work with cross-functional business teams, identify data hydration solutions, assess the feasibility, and provide recommendations to other developers on automation opportunities ensuring that delivered frameworks are scalable.


Qualifications:

• • BS in Computer Science or Related
  • 5 to 10 years of data engineering experience
  • SQL heavy Data Engineering minded and who has been involved performance fine tuning and building ETL pipelines using SQL transformation logic.
• Good understanding of modern data platforms such as Snowflake, Data bricks including data lakes, with good knowledge of the underlying architecture, preferably in Snowflake.
• Proven experience in assembling large, complex sets of data that meets non-functional and functional business requirements.
• Experience in identifying, designing, and implementing integration, modelling, and orchestration of complex fulfillment data and at the same time look for process improvements, optimize data delivery and automate manual processes.
  • Working experience of scripting, data engineering and analytics (SQL, Python, devops CI/CD pipelines, Airflow/ADF, Machine Learning (good to have))
  • Working experience of performance tuning and optimization, bottleneck problems analysis, and technical troubleshooting in a, sometimes, ambiguous environment.
  • Working experience of working with cloud-based systems (AWS preferred but GC Azure experience ok to have)

Desired Qualifications:

• • Experience working with cloud-based systems - AWS/Azure/GC & Snowflake data warehouses.
  • Expertise in designing data table structures, reports, and queries.
  • Working knowledge of CI/CD
  • Working knowledge of building data integrity checks as part of delivery of applications
  • Experience working with Kafka technologies.
  • Technical expertise to build code that is performant as well as secure.
  • Technical depth and vision to perform POC's and evaluate different technologies.
  • Experience with Real Time Analytics and Real Time Messaging
  • Working experience with Microservices is desirable.
  • Design, implement and monitor 'best practices' for Dev framework.
  • Experience working with large volume data; retail experience strongly desired.
  • Possesses an entrepreneurial spirit and continuously innovates to achieve great results.
  • Communicates with honesty & kindness and creates the space for others to do the same.
  • Fosters connection by putting people first and building trusting relationships.
  • Integrates fun and joy as a way of being and working, aka doesn't take themselves too seriously.
  • Preferred Tools: Data bricks, Snowflake, Apache Airflow, Microsoft Azure Data Factory, Kafka, Power BI, SSAS

A day in the life:

• • Co-ordinate with multiple data source teams in getting data published to our Finance Data Warehouse
  • Working with stakeholders including data, design, product, and executive teams, and assisting them with data related and technical issues.
  • Building Scalable Data Pipelines for generating training datasets for machine learning deliverables
  • Mentoring Junior resources and drive end to end design, implementation, and delivery of engineering components.
  • Communicating with and educate both senior and junior colleagues to further embed data science and analytics across the organization.
  • Strive for continuous improvement of code quality and development practices.
  • Willingness to adapt to and self-learn new technologies and deliver on them.
  • Translating business issues to technical terms
  • Understanding, leveraging, and applying best practices effectively. Also leads by example and comes up with coding standards and best practices for technology.
  • Collaborating with cross-functional teams - business stakeholders, engineers, program management, project management, etc. - to produce the best solutions possible.
  • Anticipating system/application challenges and proposes solutions for the same.
  • Contributing to story sizing and work estimates for implementation, validation, delivery, and documentation
  • Reviewing user stories to ensure a quality user experience, well-defined acceptance criteria and thorough test coverage.
  • Participating in design and code review to ensure quality and testability of feature code.

Please share Resumes on View email address on click.appcast.io