Data Engineer (Gen AI)

Company Overview Interactive Brokers Group, Inc. (Nasdaq: IBKR) is a global financial services company headquartered in Greenwich, CT, USA, with offices in over 15 countries. We have been at the forefront of financial innovation for over four decades, known for our cutting‑edge technology and client commitment. IBKR affiliates provide global electronic brokerage services around the clock on stocks, options, futures, currencies, bonds, and funds to clients in over 200 countries and territories. We serve individual investors and institutions, including financial advisors, hedge funds and introducing brokers. Our advanced technology, competitive pricing and global market help our clients make the most of their investments. Barron’s has recognized Interactive Brokers as the #1 online broker for six consecutive years. Hybrid Role This role is hybrid: 3 days in office / 2 days remote. About Your Team The Enterprise Architecture organization is looking for a Data Engineer to join the team and help build the next generation of data infrastructure and AI‑enabled workflows. In this role, you will design, build and maintain scalable data pipelines, data lake platforms and analytics solutions that support enterprise‑wide AI initiatives and advanced analytics capabilities. You will partner closely with internal development teams and IT leadership to architect data solutions that meet diverse use‑cases across the organization. This role offers the opportunity to work with cutting‑edge data technologies, AI knowledge bases and cloud‑native data platforms as we scale our data operations. Your work will focus on delivering robust, well‑documented data solutions and establishing best practices for data engineering across the enterprise. Responsibilities Design, build and maintain scalable data crawlers and ETL/ELT pipelines to ingest data from various sources including enterprise collaboration tools, web applications and internal databases. Develop and manage data lake platform infrastructure including S3‑based storage, Iceberg tables, AWS Glue data catalog and cloud data warehouse solutions for analytics workloads. Build and optimise real‑time streaming data pipelines using Kafka for event‑driven analytics and data processing. Create and maintain data transformation pipelines to clean, curate and prepare data for analytics and AI/ML applications, supporting multiple data formats including structured, semi‑structured and unstructured data (text, images, audio, video). Develop Python‑based applications for data ingestion, processing and integration supporting Gen AI RAG workflows and knowledge base systems. Collaborate with internal development teams, data scientists and stakeholders to understand requirements, architect appropriate data solutions and create comprehensive technical documentation. Monitor, troubleshoot and optimise data pipelines for performance, reliability and data quality. Write clean, maintainable, well‑tested code following software engineering and data engineering best practices. Create comprehensive technical documentation for data pipelines, architectures and platform capabilities. Required Skills 6+ years of hands‑on data engineering experience with modern data stack technologies. Strong experience with AWS cloud services, particularly S3, AWS Glue, Athena, EMR and Lambda. Proficiency in Python for data processing, ETL and application development. Experience with PySpark on EMR for large‑scale data processing. Strong SQL skills for data analysis and transformation. Experience building and maintaining ETL/ELT pipelines at scale. Experience with Kafka for streaming data pipelines and real‑time data processing. Knowledge of data lake architectures and modern table formats (e.g., Iceberg). Experience with CI/CD practices using Git, version control systems and containerisation (Docker). Understanding of data modelling, data warehousing concepts and analytics best practices. Exceptional problem‑solving and analytical skills. Excellent collaboration and communication (verbal and written) skills. Self‑motivated with ability to work independently and manage multiple priorities. Willingness and enthusiasm to learn AI/ML technologies and stay current with emerging data engineering trends. Success Criteria Self‑motivated and able to handle tasks with minimal supervision. Superb analytical and problem‑solving skills. Excellent collaboration and communication (verbal and written) skills. Outstanding organisational and time‑management skills. Company Benefits & Perks Competitive salary, annual performance‑based bonus and stock grant. Retirement plan 401(k) with competitive company match. Excellent health and wellness benefits, including medical, dental and vision benefits; company‑paid medical healthcare premium. Wellness screenings and assessments, health coaches and counselling services through an Employee Assistance Program (EAP). Paid time off and a generous parental leave policy. Daily company lunch allowance and a fully stocked kitchen with healthy options for breakfast and snack. Corporate events including team outings, dinners, volunteer activities and company sports teams. Education reimbursement and learning opportunities. Modern offices with multi‑monitor setups. Compensation The anticipated base salary range for this role is $150,000 to $200,000 per year, based on skills, experience and location. The offered salary is just part of the total compensation package. In addition to a competitive salary, the company offers a discretionary cash bonus and a stock award, as well as a wide range of benefits, including healthcare and tuition reimbursement. #J-18808-Ljbffr Interactive Brokers