AI Engineer

AI Engineer

Location: Houston, Texas

Type: Direct Hire

Salary: $120,000 - $125,000

Job Description

The AI Engineer will be primary responsibility for architecting and implementing on-premises AI platform from scratch including LLM infrastructure, retrieval-augmented generation (RAG), agentic search, and related intelligent automation systems. This is a foundational role: there is no pre-existing AI infrastructure or precedent and the work performed in this position will establish the technical foundation upon which AI is built for years to come. The role works closely with Data Engineering to ensure AI systems are supported by clean, governed data pipelines, and partners directly with the VP of Software Development. Must be able to work 5 days a week on site in office.

Duties

• Architects and implements on-premises AI platform from the ground up, leveraging newly acquired on-prem GPU hardware (H200-based) to deliver enterprise AI capability with no cloud dependency.
• Designs and builds LLM-based applications, including RAG pipelines, agentic search, and vector retrieval systems, using self-hosted frameworks such as Haystack, LlamaIndex, Ollama, or comparable tools.
• Establishes foundational standards, patterns, and best practices for AI development, since none currently exist.
• Sets the technical and architectural precedent for how AI is built, deployed, secured, and governed going forward, including PII and data-governance safeguards (redaction, access controls, audit logging) sufficient standards.
• Deploys, tunes, and monitors models on on-prem GPU infrastructure, optimizing for throughput, latency, and resource utilization across shared workloads.
• Collaborates with Data Engineering to define data contracts, feature pipelines, and integration points between the MS SQL Server/DB2 data warehouse and AI systems.
• Evaluates and prototypes emerging AI tooling, both open-source and commercial, for fit within a strict on-premises, no-cloud-egress environment.
• Develops internal tools and APIs that expose AI capabilities to other departments, such as virtual agent support, document processing, and ticket triage.
• Performs other job related duties as assigned.
• Successfully architect and stand up on-premises AI platform on schedule, establishing a stable foundation for future AI initiatives.
• Design and implement AI systems that operate entirely within the on-premises environment, with no unauthorized cloud egress of sensitive data.
• Establish documented standards, patterns, and governance practices for AI development that can be adopted across future projects and team members.
• Ensure all AI deployments satisfy regulatory expectations, including auditability and PII protection.
• Collaborate effectively with Data Engineering and the VP of Software Development to align AI systems with broader data platform architecture.
• Provide informed, professional, and accurate support to internal stakeholders leveraging AI-powered tools and capabilities.
• Stay current on LLM and AI security risks (e.g., prompt injection, data leakage, model drift) and implement appropriate mitigations.
• Demonstrate sound judgment and independent decision-making when operating in a greenfield environment with limited existing precedent.
• Accept individual accountability and responsibility for the success of AI initiatives, including meeting assigned goals and project milestones.

Requirements

• 3+ years of experience building and deploying machine learning or LLM-based applications in production, including experience architecting systems rather than solely implementing within existing ones.
• Work involves regular collaboration with Data Engineering, departmental leadership, and end users across the credit union.
• Ability to clearly explain complex technical concepts to non-technical stakeholders and to operate with a high degree of independence and sound judgment is essential.
• Strong Python skills, including experience with ML/AI frameworks (PyTorch, Hugging Face Transformers, LangChain/LlamaIndex/ Haystack, or similar).
• Experience with vector databases and embedding-based retrieval systems.
• Hands-on experience with containerized deployment (Docker) on Linux (Debian/Ubuntu) servers.
• Demonstrated ability to operate independently in greenfield environments, building from zero with limited existing infrastructure or precedent.
• Understanding of data privacy and PII handling practices.
• Experience operating self-hosted LLMs (Ollama, vLLM, text-generation-inference) on GPU hardware, including GPU resource planning and capacity management, preferred.
• Must have good communication skills.
• Ability to maintain a high level of confidentiality at all times.

Education

Equivalent to a college degree, in the field of Computer Science (BS or BA in a relevant field), or related professional work experience.