Zhipu AI, a Chinese AI startup founded in 2019, recently released its free AI agent to the general public. Known as AutoGLM Rumination, the new solution, which has already secured millions of dollars in government-backed funding, is making headlines across the industry.
What is AutoGLM Rumination?
AutoGLM Rumination is one of the newest AI agents to hit the consumer market. It’s capable of performing basic web searches as well as more advanced research tasks. Current uses for AutoGLM Rumination include technical writing and travel planning.
The AI agent is powered by two of Zhipu AI’s proprietary large language models (LLMs): GLM-4-Air-0414 and GLM-Z1-Air. To assess how these LLMs measure up against competing models, it’s essential to examine available benchmark data.
Sizing up the competition through benchmarks
Though only in operation for a few years, Zhipu AI developers have made ambitious claims regarding the performance of their generative AI tools. Developers claim that GLM-Z1-Air performs eight times faster than DeepSeek-R1; it reportedly does so while only using a fraction of the computational power.
A research paper published in June 2024 shows that Zhipu AI’s most recent LLM, GLM-4, does surpass OpenAI’s GPT-4 across numerous benchmarks. The paper’s authors stated that GLM-4 “closely rivals or outperforms GPT-4 in terms of general metrics such as MMLU, GSM8K, MATH, BBH, GPQA, and HumanEval.”
However, it falls short when compared to other types of AI models, such as Claude 2, Claude 3 Opus, Gemini 1.5 Pro, and GPT-4 Turbo, in certain areas. In Python and Java programming, for example, GLM-4 is the only Zhipu AI model that scores high enough to provide any real competition — and even that lags behind the top models on NaturalCodeBench.
While GLM-4 is leading many of the benchmarks on LongBench-Chat, GLM-4-Air and GLM-4-9B-Chat didn’t perform quite as well. They both struggled to keep pace with the competition in the English language benchmarks, but all three performed well with the Chinese language tests.
Contributing to the open-source community
Zhipu AI has made numerous contributions to the open-source AI community, and they’ve accumulated more than 10 million downloads for their past releases. These include:
- ChatGLM-6B
- GLM-4-9B
- GLM-4V-9B
- WebGLM
- CodeGeeX
Despite a slew of AI startups entering the market with their own solutions, developers with Zhipu AI are already making their presence known. AutoGLM Rumination is only the latest in a line of AI-driven products, and we’ll likely hear more from Zhipu AI in the near future.