Principal Data Engineer

Principal Data Engineer

Analytics & Data Science | Bengaluru, India | Remote

Atlassians can choose where they work – whether in an office, from home, or a combination of the two. That way, Atlassians have more control over supporting their family, personal goals, and other priorities. We can hire people in any country where we have a legal entity.

Atlassian is looking for a Principal Data Engineer to join our Data Engineering Team and play a tech lead & architect role to build world-class data solutions and applications that power crucial business decisions throughout the organization. We are looking for an open-minded, structured thinker who is passionate about building systems at scale. You will enable a world-class engineering practice, drive the approach with which we use data, develop backend systems and data models to serve the needs of insights and play an active role in building Atlassian's data-driven culture.

Responsibilities:

• Own the technical evolution of the data engineering capabilities and be responsible for ensuring solutions are being delivered incrementally, meeting outcomes, and promptly escalating risks and issues
• Establish a deep understanding of how things work in data engineering, use this to direct and coordinate the technical aspects of work across data engineering, and systematically improve productivity across the teams.
• Maintain a high bar for operational data quality and proactively address performance, scale, complexity and security considerations.
• Drive complex decisions that can impact the work in data engineering. Set the technical direction and balance customer and business needs with long-term maintainability & scale.
• Understand and define the problem space, and architect solutions. Coordinate a team of engineers towards implementing them, unblocking them along the way if necessary.
• Lead a team of data engineers through mentoring and coaching, work closely with the engineering manager, and provide consistent feedback to help them manage and grow the team
• Work with close counterparts in other departments as part of a multi-functional team, and build this culture in your team.

On your first day, we'll expect:

• You have 12+ years of experience in a Data Engineer role as an individual contributor.
• You have at least 2 years of experience as a tech lead for a Data Engineering team.
• You are an engineer with a track record of driving and delivering large (multi-person or multi-team) and complex efforts.
• You are a great communicator and maintain many of the essential cross-team and cross-functional relationships necessary for the team's success.
• Experience with building streaming pipelines with a micro-services architecture for low-latency analytics.
• Experience working with varied forms of data infrastructure, including relational databases (e.g. SQL), Spark, and column stores (e.g. Redshift)
• Experience building scalable data pipelines using Spark using Airflow scheduler/executor framework or similar scheduling tools.
• Experience working in a technical environment with the latest technologies like AWS data services (Redshift, Athena, EMR) or similar Apache projects (Spark, Flink, Hive, or Kafka).
• Understanding of Data Engineering tools/frameworks and standards to improve the productivity and quality of output for Data Engineers across the team.
• Industry experience working with large-scale, high-performance data processing systems (batch and streaming) with a "Streaming First" mindset to drive Atlassian's business growth and improve the product experience.

We'd be super excited if you have:

• Built and designed Kappa or Lambda architecture data platforms and services.
• Experience implementing Master Data Management (MDM) solutions.
• Built pipelines using Databricks and is well-versed with their APIs.

Atlassian offers a wide range of perks and benefits designed to support you, your family and to help you engage with your local community. Our offerings include health and wellbeing resources, paid volunteer days, and so much more.

At Atlassian, we're motivated by a common goal: to unleash the potential of every team. Our software products help teams all over the planet and our solutions are designed for all types of work. Team collaboration through our tools makes what may be impossible alone, possible together.

We believe that the unique contributions of all Atlassians create our success. To ensure that our products and culture continue to incorporate everyone's perspectives and experience, we never discriminate based on race, religion, national origin, gender identity or expression, sexual orientation, age, or marital, veteran, or disability status. All your information will be kept confidential according to EEO guidelines.

To provide you the best experience, we can support with accommodations or adjustments at any stage of the recruitment process. Simply inform our Recruitment team during your conversation with them.