Team Glm Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao et al.
ChatGLM-4 is a family of LLMs specialized in Chinese and English, achieving competitive performance with GPT-4 and equipped with tool-use capabilities.
Existing LLMs are biased towards specific languages or domains, or lack integration with external tools for complex tasks. Balancing efficiency and performance during large-scale model training and alignment is also challenging.
GLM-4 is pre-trained on 10 trillion tokens (mostly Chinese and English, with a small portion from 24 languages) and then aligned via multi-stage post-training involving supervised fine-tuning and learning from human feedback. GLM-4 All Tools is further aligned to understand user intent and autonomously choose and use tools such as web browser, Python interpreter, and text-to-image model.
GLM-4 achieves similar or better performance than GPT-4 on MMLU, GSM8K, MATH, BBH, GPQA, and HumanEval, competes with GPT-4 Turbo and Claude 3 on long context tasks, and outperforms GPT-4 in Chinese alignment. GLM-4 All Tools surpasses GPT-4 All Tools in tasks like web browsing and math problem solving. The open-source models have attracted over 10 million downloads on Hugging Face in 2023 alone.