Join AI for Good as we introduce the TelcoRetrieve Challenge, an open competition inviting participants to improve retrieval-augmented generation systems that ground large language model (LLM) answers directly in telecommunication standards with citations.
In collaboration with the International Telecommunication Union (ITU), AMD, IEEE/ETSI, Huawei, University College London (UCL), University of Leeds (UoL), the University of Vale do Rio dos Sinos (Unisinos) and GSMA’s Open Telco AI, this new challenge offers a range of cash prizes for winning solutions.
The challenge is built around TelcoRetrieve, an open, vendor-neutral RAG pipeline spanning 3GPP, ITU-T, TM Forum, and many more specifications. Participants will learn about TelcoRetrieve-Chunks, the large-scale, pre-vectorized dataset the pipeline generate, and can compete in two challenge tracks.
In this webinar, we introduce the challenge, walk through the two competition tracks, and explain how submissions are evaluated, from the public leaderboard through to final, held-out judging.
Participants will compete for a total prize pool of $15,000 ($5,000 for the winner and $2,500 for the runner-up in each track) and finalists will be invited to present their solutions at MWC Barcelona from 1–4 March 2027.
About the challenge:
Telecom standards are fragmented across multiple standards bodies, span millions of pages, and are revised continuously across releases. Answering a question correctly requires more than surface-level keyword matching. It requires retrieving the right passage from the right release, and generating an answer that is faithful to it rather than hallucinated from a model’s pretraining knowledge. In a domain where a wrong interpretation of a specification can break interoperability, this is a trust problem, not just an accuracy problem.
The TelcoRetrieve Challenge puts this to the test across two tracks:
- Pipeline Optimization — improving chunking, retrieval, and reranking strategies while the underlying models stay fixed
- Model Improvement — fine-tuning the models inside the pipeline, including the Gemma generator and the telecom-domain OTel embedder
Every submission moves through two evaluation phases: a public phase, scored on accuracy against a published question set, and a private phase, where top performers from each track are re-evaluated by the organizers against a held-out dataset on accuracy, cost, and latency together. This two-phase structure ensures that results reflect genuine generalization, not overfitting to a known test set.
Learning objectives:
By the end of the webinar, attendees will be able to:
- Describe how TelcoRetrieve grounds LLM answers in telecom standards with citations
- Identify the two challenge tracks and what participants can and cannot change in each
- Explain how the public and private evaluation phases work, and why both exist
- Understand how submissions are scored on accuracy, and on cost and latency in the final phase
Recommended prerequisites: This challenge is recommended for participants with intermediate to advanced knowledge of RAG systems and/or LLM fine-tuning. A basic understanding of telecom standards (3GPP) is helpful but not required.
