Senior Software Engineer – LLM Evaluation

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Software Engineer – LLM Evaluation based in Canada.

This part-time consulting opportunity is designed for experienced software engineers interested in shaping the evaluation of advanced large language models.

You will curate high-quality code, develop software solutions, and assess AI-generated implementations against professional engineering standards.

The role spans multiple programming languages and covers the full software-development lifecycle, from architecture and prototyping to deployment, monitoring, and maintenance.

You will also help create automated verification mechanisms and benchmarks that make model evaluation more rigorous and reproducible.

Working alongside research and technical teams, you will help identify model strengths, weaknesses, and recurring coding errors.

Your engineering judgment will directly contribute to improving coding-focused AI evaluation systems and research.

The engagement is fully remote and flexible, with a minimum commitment of 10 hours per week and the potential to work up to 40 hours weekly.