Software Engineer - Python | Remote

Overview Position: Senior Software Engineer – Python (LLM Evaluation & Repository Validation) Type: Contractor Assignment (3 months) Compensation: $19-$20/hr Location: Remote Commitment: 40 hrs/week with some overlap with PST Responsibilities Evaluate and analyze GitHub issues across widely used open-source repositories. Set up and configure development environments for repositories, including Docker setup. Modify and run codebases locally to evaluate LLM performance in debugging and bug-fixing scenarios. Evaluate unit test coverage and assess the quality of existing tests. Work with real-world Python repositories to identify engineering tasks suitable for LLM evaluation. Collaborate with researchers to expand datasets used for training and evaluating AI models. Identify repositories and issues that present challenging real-world scenarios for LLMs. Support the creation of realistic software engineering tasks based on repository histories. Requirements Strong experience in software development. Strong experience with Python programming. Proficiency in Git, Docker, and development environment setup. Experience working with well-maintained public GitHub repositories. Ability to understand, modify, and test complex codebases. Strong analytical thinking and problem-solving skills. Excellent written English and ability to document findings clearly. Ability to work independently in a remote environment. Desktop or laptop with a stable internet connection. Preferred Qualifications Experience contributing to open-source projects. Exposure to LLM evaluation, AI research, or machine learning projects. Experience with developer tools, automation agents, or CI/CD pipelines. Application Process Application Form ICF + Profile Review Assessment Submit #J-18808-Ljbffr Crossing Hurdles