Data Engineer

Software EngineerEmployees in this job function are responsible for designing, developing, testing and maintaining software applications and products to meet customer needs. They are involved in the entire software development lifecycle including designing software architecture, writing code, testing for quality and deploying the software to meet customer requirements. Full-stack software engineering roles, who can develop all components of software including user interface and server side also fall within this job function.Key Responsibilities:Engage with customers to deeply understand their use-cases, pain points, and requirements, showcasing empathy and advocating for user-centric software solutionsSolve complex problems by designing, developing, and delivering using various tools, languages, frameworks, methodologies (like agile) and technologiesAssess the requirements of the software application or service and determine the most suitable technology stack, integration method, deployment strategy, etc.Create high-level software architecture designs that outline the overall structure, components, and interfaces of the applicationCollaborate with cross functional teams like product owners, designers, architects etc.Define and implement software test strategy, guidelines, policies and processes in line with organization vision, industry regulations and market best practices.Work on continuously improving performance and optimizing the application and implement new technologies to maximize development efficiencyFamiliar with programming practices, such as test-driven development, continuous integration, and continuous deliverySupport security practices to safeguard user data including encryption and anonymization.Create user-friendly and interactive interfacesDevelop and maintain back-end applications like APIs and microservices using server-side languagesEvaluate and incorporate emerging technologies and capabilities to deliver their solutions, and are monitoring and participating in solutions for new stack layers, often involving industry collaborationSkills Required:NoSQLKubernetesKAFKAGCPCloud ArchitectureData ArchitectureBig DataCloud InfrastructureApache SparkBig QueryApache HadoopJavaDockerPythonData ManagementSkills Preferred:GitHub, Machine LearningExperience Required:5+ years of progressive experience in data engineering or a related role, with a significant focus on large-scale data systems.Expert proficiency in programming languages such as Python or Scala.Advanced SQL skills for complex querying, data manipulation, and database design.Extensive experience with big data technologies (e.g., Apache Spark, Hadoop, Kafka, Flink).Proven experience with cloud platforms (AWS, Azure, GCP) and their data services (e.g., AWS Glue, S3, Redshift, Athena; Azure Data Factory, Databricks; Google BigQuery, Dataflow).Strong understanding of data warehousing concepts, data modeling (dimensional, relational), and ETL/ELT processes.Experience with version control systems (e.g., Git).Excellent problem-solving, analytical, and critical thinking skills.Strong communication and interpersonal skills, with the ability to lead discussions and present technical information clearly.Lead Data Architecture & Design: Oversee and contribute to the design, development, and implementation of complex, scalable, and resilient data architectures, including data warehouses, data lakes, and streaming platforms.Develop & Optimize Data Pipelines: Lead the creation, maintenance, and optimization of advanced ETL/ELT processes and data pipelines to ingest, transform, and deliver large volumes of data from diverse sources. Ensure data integrity, quality, and performance across all data flows.Education Required:Bachelor's DegreeAdditional Information :***HYBRID / 4 days per week in the office***EEO Employer