Meta’s Standard Llama 4 Maverick AI Underperforms in LM Arena Benchmark After Controversy

Meta recently came under scrutiny after it was revealed that the company used an experimental version of its Llama 4 Maverick model to boost its ranking on LM Arena, a popular crowdsourced AI benchmark. The version used, named ‘Llama-4-Maverick-03-26-Experimental,’ had been specifically optimized for conversational ability, which likely influenced its performance favorably in human-rated comparisons. Once the benchmark’s maintainers learned of this, they issued an apology, revised their evaluation policies, and re-ranked Meta’s standard version of the model — ‘Llama-4-Maverick-17B-128E-Instruct.’ This unmodified version performed poorly, falling to 32nd place, well behind older models like OpenAI’s GPT-4o, Anthropic’s Claude 3.5 Sonnet, and Google’s Gemini 1.5 Pro. Meta responded by stating that it regularly experiments with custom variants to enhance performance in specific use cases. They emphasized that the now-public open-source version of Llama 4 allows developers to customize it according to their needs. The incident reignites concerns about how AI models are benchmarked and the transparency of companies participating in these public evaluations, particularly when tailored optimizations may skew real-world expectations.