IIT Madras–backed research initiative AI4Bharat has launched the Indic LLM Arena, a crowd-sourced evaluation platform that aims to set the standard for assessing global large language models (LLMs) tailored to Indian users. Designed to fill the void left by English-centric global leaderboards, the new platform benchmarks how effectively AI systems understand, respond, and behave across India’s diverse languages and cultural contexts.
The Indic LLM Arena evaluates AI models across three crucial dimensions—language, context, and safety—to ensure fair and culturally aligned AI performance. It measures how well language models handle code-mixed languages such as Hinglish and Tanglish, respond with awareness to local nuances, and adhere to India’s evolving social and ethical norms. The platform employs a human-in-the-loop system, allowing users to type or speak prompts, compare model outputs, and vote for the better response. These inputs are then aggregated into statistically validated rankings that reflect real-world linguistic diversity and user expectations.
Calling the initiative a “public utility” for India’s AI ecosystem, AI4Bharat highlighted that the Indic LLM Arena is open source, enabling collaboration among developers, enterprises, and end users to collectively shape standards for trustworthy and inclusive AI. The project’s open approach ensures that Indian voices directly contribute to defining what “good AI” looks like in the country’s multilingual and multicultural context.
Supported by Google Cloud in its initial rollout, the Indic LLM Arena is already drawing strong industry attention. Adithya S. K., AI researcher and founder of CognitiveLabs, praised the platform’s usability, noting, “The UX is spot on and I had the best Kannada typing experience. These are the sort of efforts labs in India should be doing more across domains.”
By creating a transparent, participatory benchmarking system, AI4Bharat’s Indic LLM Arena is expected to play a transformative role in shaping the next generation of AI systems—ones that not only perform technically well but also reflect India’s linguistic richness, cultural nuances, and social values. The initiative represents a major step toward building a globally competitive yet locally relevant AI ecosystem rooted in inclusivity and collaboration.
