Five hundred obscure facts about 18th-century botany can reveal the exact number of parameters in a secret AI model.
April 29, 2026
Original Paper
Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity
arXiv · 2604.24827
The Takeaway
Closed-source models like GPT-4 often keep their scale a closely guarded trade secret. This method uses incompressible knowledge probes to estimate parameter counts by testing how many high-entropy facts a model can recall. Because facts that cannot be reasoned through must be stored directly in the weights, the factual capacity of the model serves as a precise thermometer for its size. The researchers accurately predicted the size of several open-source models before testing the method on proprietary systems. This technique allows competitors and regulators to audit the complexity of any black-box system through a simple API.
From the abstract
Closed-source frontier labs do not disclose parameter counts, and the standard alternative -- inference economics -- carries $2\times$+ uncertainty from hardware, batching, and serving-stack assumptions external to the model. We exploit a tighter intrinsic bound: storing $F$ facts requires at least $F/$(bits per parameter) weights, so measuring how much a model \emph{knows} lower-bounds how many parameters it \emph{has}. We introduce \textbf{Incompressible Knowledge Probes (IKPs)}, a benchmark o