Data Engineer

Senior Data Engineer

Location: Westlake, TX Hybrid / Onsite (Connect weeks required)

Duration: Long

Interview: Video

Technical Screening via ROPES Test (Test Name: FID - DevOps/Python - Asset Management)

This pivotal role focuses on driving critical data migration activities, designing robust API integrations, and leveraging cutting-edge AI technologies to optimize our data workflows.

The ideal candidate will bridge the gap between traditional data engineering and modern AI development, boasting deep expertise in API management, LLM tuning, and database optimization. As a key collaborator, you will work closely with both onshore and offshore teams to deliver high-quality, scalable, and thoroughly tested data solutions.

Required Skills & Qualifications

• Experience: 5+ years of hands-on data engineering experience (Mid-to-Senior level).
• API Expertise: Proven, deep experience in API development, integration, and API management tools.
• AI/LLM Skills: Direct experience working with, tuning, and optimizing Large Language Models (LLMs).
• Database Mastery: Strong database design, modeling, and query optimization skills (SQL/NoSQL).
• Testing Rigor: Demonstrated experience writing and executing unit tests and regression tests for data pipelines.
• Communication: Exceptional verbal and written communication skills, with a proven track record of successfully collaborating with offshore teams.

Nice-to-Have / Advantages

• Experience creating business-facing reports and dashboards in Power BI.
• Familiarity with cloud data platforms (Snowflake, AWS, Azure, or GCP).

Key Responsibilities

API Management & Integration (High Priority)

• Design, develop, and manage robust, secure, and scalable APIs to facilitate seamless data integration and migration.
• Implement best practices in API gateway management, rate limiting, and security protocols.
• Collaborate with application teams to ensure high-performing data pipelines via API endpoints.

AI & LLM Optimization

• Actively leverage AI technologies to automate and enhance data engineering pipelines.
• Hands-on tuning, prompt engineering, and optimization of Large Language Models (LLMs) to align with business logic and performance requirements.

Data Engineering & Migration

• Lead hands-on data migration activities from legacy systems to modern cloud data architectures.
• Write complex, highly optimized SQL queries and manage database performance tuning.
• Note: Deep Snowflake development is not required, as a dedicated Snowflake team is already in place; however, a strong conceptual understanding of cloud data warehousing is a plus.

Testing, Quality Assurance & Reporting

• Own the end-to-end testing lifecycle for data pipelines, including rigorous unit testing and regression testing to ensure data integrity.
• Develop and maintain operational reports and dashboards using Power BI to provide actionable insights to stakeholders.

Collaboration & Communication

• Act as a technical bridge between onshore leadership and offshore development teams.
• Communicate technical requirements, architectural designs, and feedback clearly and concisely to ensure alignment across time zones.