Data Engineer

Data EngineerOur client is a forward-thinking platform that is revolutionizing the way individuals curate and express their personal style through digital content. Like TikTok tailors to one's sense of humor, this platform personalizes the user experience by curating product recommendations and content based on individual aesthetic preferences. Users can explore and save content from Instagram directly to their profiles, organize their inspirations into collections, and create moodboards that reflect their unique taste.Position OverviewThe Data Engineer will be instrumental in constructing and optimizing data pipelines that gather, organize, and analyze vast amounts of information from various sources, including Instagram. This role demands close collaboration with the machine learning team to enhance the recommendation algorithms, ensuring they genuinely reflect each user's distinctive style preferences.Key ResponsibilitiesDesign, construct, and maintain efficient, reliable data pipelines to process extensive datasets from Instagram and additional sources.Create systems that enable users to seamlessly save Instagram content to their profiles and categorize these into collections.Work alongside machine learning specialists to perfect data gathering approaches for the personalization algorithms.Uphold strict data privacy and security standards, complying with legal and best practice frameworks.Develop the data framework supporting user moodboard creation and content archival functionalities.Liaise with different teams, including product management and UX/UI design, to ensure data engineering initiatives align with user experience objectives.Utilize analytics tools to evaluate and improve the performance of personalization algorithms.QualificationsBachelor's degree in Computer Science, Engineering, or a related field; Master's preferred.Demonstrated expertise in a Data Engineer or similar role, with an emphasis on social media data integration.Proficient in Python, Java, Scala, or similar programming languages.Skilled in using data pipeline and workflow management tools such as Apache Airflow.Expertise in SQL, relational databases, and familiarity with various database systems.Experience with big data tools (e.g., Hadoop, Spark) and real-time data processing technologies (e.g., Kafka).Familiarity with cloud services like AWS, Azure, or GCP.Competence in social media platform API integration.Knowledge of machine learning algorithms and personalization systems is a significant plus.Exceptional problem-solving abilities and meticulous attention to detail.What We OfferA competitive compensation and benefits package.An energetic, creative, and nurturing work environment.Professional development and growth opportunities.A chance to contribute to a cutting-edge personalization experience in the intersection of technology and aesthetics.Interested candidates should submit a resume and a cover letter explaining their qualifications and interest in the role.