Senior Big Data Engineer

Position: Senior Big Data Engineer Working model: Remote or hybrid Form of employment: contract of employment Join our client which makes software to help users organize data, discover the truth, and act on it. Their SaaS product manages large volumes of data and quickly identifies key issues during litigation and internal investigations. The AI-powered communication surveillance product proactively detects regulatory misconduct like insider trading, collusion, and other non-compliant behavior. They have more than 300,000 users in 49 countries serving thousands of organizations globally. Responsibilities Build our next-generation data platform tooling and services to support the ingestion, and processing of billions of documents at scale. Improve and extend our Spark-based distributed data processing pipeline. Improve and extend our Rust-based distributed query engine used to request large amounts of document data. Create tools to automate and optimize processes across disciplines. Actively participate in the on-call schedule to investigate and fix production issues related to our data processing pipeline or query engine. Participate in code reviews for projects written by your team. Focus on quality through comprehensive unit and integration testing. Qualifications 4+ years of software development experience in writing performant, commercial-grade systems, and applications. Experience with monitoring and troubleshooting production environments. Proficiency in programming languages used in high-volume data processing and applications like Java or Scala and Python. Experience building data pipelines with distributed frameworks like Hadoop, Spark, or Dask. Knowledge of Linux/Unix systems, Docker/Kubernetes, and CI/CD including scripting in Python or other scripting languages to automate build and deployment processes. Knowledge of professional software engineering practices & software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations. Leverages best practices and past experiences to mentor and improve the productivity of the team. Benefits Hybrid workplace arrangements IT & home office stipends. Unlimited time off, two company-wide breaks per year. Private medical healthcare for employees & family members in Medicover. Multisport card & other wellness perks MyBenefit. Cafeteria System. Long-term incentive program. Continuous training programs. #J-18808-Ljbffr