Software Engineer (Open Source + LLM Evaluation)

Posted on June 18, 2025

Apply Now

Job Description

  • Senior Software Engineer (Open Source + LLM Evaluation)
  • Experience: 3 Years
  • Location: Remote
  • Immediate Joiners Preferred
  • ? Must-Have Requirements:
  • 3+ years of software engineering experience.
  • Mandatory: Proven contributions to GitHub repositories with 5,000+ stars.
  • Hands-on experience with at least one of the following languages:
  • Python, JavaScript, Java, Go, Rust, C, C++, C#, Ruby.
  • **About the Role:**
  • We�re seeking experienced Software Engineers (Tech Lead Level) passionate about open-source software and LLM research. This role involves hands-on engineering in high-quality, public GitHub repositories. You�ll triage issues, assess code quality, and evaluate LLM capabilities in real-world coding scenarios.
  • ** Day-to-Day Responsibilities:**
  • Analyze and triage GitHub issues in popular open-source repositories.
  • Set up development environments (e.g., Docker, tooling, repo setup).
  • Evaluate unit test coverage, execution, and overall code quality.
  • Modify and run open-source projects locally to test LLM-based bug-fixing performance.
  • Collaborate with research teams to identify challenging repositories for LLMs.
  • Option to lead junior engineers and collaborate on large-scale LLM research projects.
  • **Skills:**
  • Strong hands-on experience with any of the following:
  • JavaScript, Java, Go, Rust, C/C++, C#, Ruby.
  • Solid knowledge of Git, Docker, and setting up dev pipelines.
  • Comfortable navigating and working within complex open-source codebases.
  • Ability to run, test, and modify production-grade projects locally.
  • Past contributions or familiarity with open-source projects is a plus.
  • **Nice to Have:**
  • Experience participating in LLM evaluation or AI/ML research.
  • Worked on developer tools or automation agents.
  • Open to mentoring or leading a small engineering team.

Required Skills

hands-on experience with at least one of the following languages: python javascript java go rust c c++ c# ruby.