ChatGLM: the open-weight chat model lineage

ChatGLM: origin and significance

How ChatGLM arrived, why its initial release was notable, and what made it different from competing open-weight chat models at the time.

ChatGLM arrived at a moment when the open-weight chat model landscape was dominated by models that handled English well but struggled to hold a coherent conversation in any other language. The original ChatGLM-6B was a 6-billion-parameter bilingual model that produced natural Chinese and English responses with no additional fine-tuning, shipped with weights publicly available on Hugging Face, and ran on a single consumer GPU with 8 GB of VRAM. That combination — conversational quality, bilingual coverage, local-inference feasibility — made it the most-downloaded Chinese-origin model in the first months after release.

The architectural decision behind ChatGLM was the same span-masking approach that characterised the broader GLM research line: the model was trained to predict entire masked spans autoregressively, giving it both the contextual understanding of a bidirectional encoder and the fluency of a generative model. That pretraining objective turned out to transfer well to instruction tuning, which is why the ChatGLM variants responded well to prompt engineering from the start, even before the field had developed a consensus on system-prompt design patterns.

The significance for Z.ai is direct: every inference product in the current Z.ai portfolio builds on the architectural and training decisions first validated at scale in ChatGLM. The current GLM-4 and GLM-4.5+ flagship generations are the mature expressions of the same design lineage. Understanding ChatGLM history therefore gives a developer a concrete mental model for why the current-generation Z.ai models behave the way they do under prompting pressure.

ChatGLM generation history

A chronological walk through each ChatGLM generation, noting the key change introduced in each release.

The original ChatGLM established the 6B parameter size as the primary open-weight target. It used a 2K context window — adequate for single-turn conversations but limiting for longer tasks — and was instruction-tuned on Chinese and English data with a relatively small supervised fine-tuning dataset by today's standards. Despite those constraints, it outperformed alternatives at its parameter class on Chinese-language evaluations and set the template that later generations refined.

ChatGLM2-6B extended the context window to 32K tokens using a sliding-window position embedding approach that extrapolated well beyond the training context. It also added a multi-query attention variant that reduced the KV-cache memory footprint during inference, making longer conversations more tractable on the same hardware. The instruction tuning dataset grew significantly, and the feedback from the first generation's public release was incorporated into the alignment process.

ChatGLM3 introduced a tool-calling schema that aligned with the emerging function-calling standards being adopted across the industry. It shipped at multiple parameter sizes simultaneously for the first time, providing a 6B local variant and a larger option accessible via API. The context window reached 128K in the long-context variant, and code capabilities improved markedly compared to the generalist training of the first two generations.

ChatGLM4 — the most recent build in the original ChatGLM naming convention — bridges into the GLM-4 family while maintaining the open-weight distribution model. The 9B ChatGLM4 build is competitive with models twice its size on standard benchmarks, reflecting the maturation of Zhipu AI's alignment pipeline over the four-generation arc. Practitioners who download ChatGLM4 and run it locally will find it substantially more capable than the original ChatGLM on every quality dimension, while requiring only marginally more hardware than the original 6B build when quantised.

Local inference workflow

How to load and run ChatGLM weights on local hardware, covering the main frameworks and quantisation options for consumer-grade setups.

Loading ChatGLM weights locally starts with the Hugging Face Transformers library. The model card for each generation lists the exact repository path and the recommended loading snippet. For the 6B and 9B builds, the standard float16 load requires 12–16 GB of VRAM on a single GPU; the int4 quantised build fits into 8 GB, and the int8 build sits between those two points with a modest quality trade-off. The Transformers library handles the quantisation transparently through the BitsAndBytes integration, so no separate quantisation step is required before inference.

The vLLM framework offers a higher-throughput option for teams running ChatGLM as a local API server. It implements continuous batching and paged attention, which increases requests-per-second by a meaningful margin compared to a naive Transformers inference loop. For a small team running ChatGLM as a shared internal service, a vLLM deployment on a single A100 or H100 instance handles the kind of concurrent-user load typical in a development environment without the operational complexity of a multi-node setup.

Practitioners comfortable with llama.cpp can also run the GGUF-format ChatGLM builds that the community has produced for the smaller parameter sizes. These builds run on CPU with optional GPU offloading, which makes them the most accessible path for developers who do not have a dedicated GPU available. Quality degrades relative to the full-precision builds, but for evaluation and prototyping purposes the GGUF path is often the fastest way to get a ChatGLM response into a development workflow. The download reference on this site covers the community GGUF mirror locations in more detail.

ChatGLM generation comparison across the public lineage

Generation	Release class	Notable change	Context window	Parameter size(s)
ChatGLM	Original	First bilingual open-weight chat model; 6B on consumer GPU	2K	6B
ChatGLM2	Second generation	32K context via sliding-window PE; reduced KV-cache footprint	32K	6B
ChatGLM3	Third generation	Tool-calling schema; multi-size release; 128K long-context variant	32K–128K	6B, 32B+
ChatGLM4 (6B)	Fourth generation	Bridges into GLM-4 family; competitive with larger models on benchmarks	128K	6B
ChatGLM4 (9B)	Fourth generation	9B open-weight build; improved alignment; tool-calling refinement	128K	9B

ChatGLM versus GLM-4: where the lineages diverge

The practical difference between downloading a ChatGLM build and accessing the GLM-4 flagship through the BigModel API.

ChatGLM and GLM-4 share the same architectural heritage and the same Zhipu AI training pipeline. The divergence is primarily one of scale, distribution method, and intended use case. ChatGLM is the open-weight download lineage: weights are public, inference runs locally, and the cost of use is hardware time rather than per-token billing. GLM-4 is the hosted flagship lineage: the largest parameter classes are available only through the BigModel API, the context window reaches 128K, and the alignment quality at the top end clearly separates from what a locally-hosted 9B build can achieve.

For a developer starting a new project, the right starting point is usually the ChatGLM4-9B local build for prototyping and the GLM-4.5+ API for production. The local build allows rapid iteration without API costs, and because the underlying architecture is essentially the same, prompts that work well locally generally transfer to the hosted version without significant rework. That migration path is one of the structural advantages of the Z.ai ecosystem over competing families where the open-weight and hosted variants have more divergent training histories.

Research guidance from the MIT Center for Research on Equitable and Open AI notes that local-inference models provide stronger privacy guarantees for sensitive workloads, which is a legitimate reason to stay on the ChatGLM download path rather than migrating to the hosted API even when scale requirements would otherwise justify the shift. Teams handling sensitive data should assess that trade-off explicitly before committing to a deployment architecture.