Amazon researchers removed half of the “experts” inside an open AI model, and the model kept its score on a coding test. Their method, HOPE, presented on November 9, 2026, cuts the memory such models need, and memory decides what hardware can run them.
HOPE appeared on the blog of Hugging Face, a platform for sharing AI models. It targets “mixture of experts” models, split into many specialized sub-networks called experts, and deletes some of them for good, without retraining. The researchers removed half of the experts in Qwen3.5-35B-A3B, an open model from Alibaba’s Qwen team: anyone can download and run it on their own hardware.
The slimmed-down model scored 42.3 out of 100 on SWE-Bench Pro, a test of real coding tasks, versus 41.8 for the full model. The research paper, first posted online on September 16, 2026, also shows that result’s limits. It depends on the data used to pick the experts to cut, and on the choice of test.
Most experts sit idle but still take up memory
The researchers tested Qwen3.5 models and GLM-4.5-Air, another open model. All are built as stacks of layers, each holding 128 or 256 experts, depending on the model. For each token, a chunk of text often shorter than a word, a router switches on only a few of them.
According to its Hugging Face page, Qwen3.5-35B-A3B has 35 billion parameters, the values it learned in training, but uses only 3 billion per token. Each of its 40 layers holds 256 experts, 8 of them active at a time, plus one shared expert that is always on. It also uses Gated DeltaNet layers, which rely on linear attention, a lighter way to track earlier text that also cuts working memory.
Amazon’s post cites the larger Qwen3.5-122B, where experts make up 95% of roughly 250 GB of parameters. Yet only 8 of the 256 experts in each layer work on any given token. According to the paper, removing half of them would bring the model down to 125 GB. Size decides what hardware it takes to run a model locally, on a company’s own servers.
Removing experts this way is called expert pruning. The authors distinguish it from quantization, which stores each parameter with less precision, in fewer bits: every expert gets lighter, but all stay in memory. Pruning removes some experts outright. The two can be combined.
HOPE looks at experts in pairs
Choosing which experts to remove is the hard part. Existing methods feed the model a sample of prompts, called a calibration dataset, and score each expert on its own. They drop the lowest scorers, layer by layer. The usual point of comparison is REAP, a method published in October 2025 by researchers at the chipmaker Cerebras.
HOPE’s five authors, from the Agentic AI team at Amazon Web Services (AWS), noticed that experts work in stable groups that switch on together. Removing an expert that looks unimportant can leave an important one without its usual partner. So HOPE, short for Higher-Order Pruning of Experts, also scores each pair of experts. An optimization step then picks the experts whose removal changes the model’s output the least.
What separates HOPE from REAP is the pair scoring. According to the authors, HOPE’s one-by-one scores rank the experts almost exactly as REAP does, with a correlation of 0.988, where 1 means identical.
The optimization step adds only one to two seconds per layer. On the 35-billion-parameter model, calibration took 1.27 hours per 1,000 prompts, compared with 1.20 hours for REAP, on hardware the paper does not name.
The code is on GitHub, a code-sharing site, under the open-source Apache 2.0 license. It runs in three commands (calibrate, solve, prune) and outputs a smaller model, on disk and in memory, in Hugging Face’s standard format. According to the post, that model loads unchanged in vLLM and SGLang, two programs that run models on servers. Qwen models have been altered before: in early November, OpenAI’s GPT-6 Astra had already edited the parameters of a Qwen model without training.
Amazon’s tests put HOPE first on average
The researchers compared five pruning methods on three models: Qwen3.5-122B-A10B, Qwen3.5-35B-A3B and GLM-4.5-Air. They removed between 10% and 50% of the experts, with two calibration datasets, and scored every test out of 100.
The Tulu 3 test suite covers math, instruction following, general knowledge and code. SWE-Bench Pro has 731 tasks from enterprise software projects. In it, the model acts as an agent, fixing code on its own, step by step.
Across all settings, HOPE has the best average rank of the five methods: 2.07, where 1 would mean first every time. It comes first in 39% of cases. With half of the experts removed, its average rank improves to 1.58, compared with 2.42 for REAP, the runner-up.
The headline 42.3 comes from Qwen3.5-35B-A3B with half its experts removed, calibrated on Evol-CodeAlpaca, a dataset of coding prompts. In the same setup, REAP’s version scores 36.2.
HOPE’s half-point edge over the full model, 42.3 versus 41.8, is smaller than the 1.2-point variation the paper reports across three runs. The two are effectively tied. The authors note that pruned models sometimes beat the original, as seen with other methods. Amazon claims gains of “up to +6.1%” over the second-ranked method on these agentic coding tasks.
The paper’s own tables show the limits
Amazon’s 6.1% figure is the gap in points between HOPE and REAP, the runner-up when half the experts go: 42.3 versus 36.2. In the same code-calibrated test, EAN, another of the five methods, scores 40.6. So HOPE’s lead over its closest rival there is under two points. On SWE-Bench Pro, the paper puts that lead at 2.0 points at most.
The 42.3 score also depends on the calibration data. Calibrated on recorded agent sessions from SWE-Bench Verified, another coding test, the HOPE version scores 40.8, a point below the full model. REAP scores 40.2 and MAN, another of the five methods, 40.6: a near tie.
On the larger Qwen3.5-122B-A10B, HOPE does not always keep the full model’s score. Calibrated on code and stripped of half its experts, the HOPE version drops to 43.8 on SWE-Bench Pro, from 50.2 for the full model. EAN scores 48.3 there, and even the simplest method, which removes the least-used experts, scores 45.7, both ahead of HOPE. With the other calibration dataset, HOPE holds at 50.6, but EAN does better, at 51.8.
On two of the three models, scores on general tests drop as well, and HOPE does not always beat REAP on them. On the Tulu 3 suite, Qwen3.5-35B-A3B’s average falls from 48.9 to 42.9 when HOPE removes half its experts with code calibration. REAP scores 43.0 under the same conditions. On GLM-4.5-Air, REAP beats HOPE here with both calibration datasets: 44.7 versus 43.3, and 43.4 versus 40.9.
Amazon’s post says code calibration did not “severely” degrade the non-coding tests. On Qwen3.5-35B-A3B, though, the score on MMLU, a multiple-choice general-knowledge test, more than halves under those conditions, from 65.2 to 29.8.
The authors list three limits themselves. HOPE’s advantage shrinks when only 10% to 20% of the experts are removed. The method treats each layer separately. And it scores pairs of experts but stops short of groups of three or more.
No pruned model comes with the code. Users must run the calibration themselves, with the full model loaded in Transformers, Hugging Face’s AI software library. Neither the post nor the GitHub page mentions llama.cpp or its GGUF file format, which are used to run models on a workstation.
On October 24, developer Filip Lebiecki published a test on GitHub that tempers the case for pruning on a 16 GB graphics card. He tested REAP, so HOPE was not part of it. Keeping 128 of the 256 experts in Qwen3.5-35B-A3B, he got a 19-billion-parameter model.
At the same file size, about 10.6 GiB, the full model compressed to 2.6 bits per parameter beat the pruned model compressed to 4.9 bits. The full model averaged 91.4 across three tests of math, code and knowledge, versus 82.7 for the pruned one. Compressing harder beat pruning.
What it means
HOPE builds on REAP’s pruning approach and changes how the experts to remove are chosen, by accounting for those that work together. The gain shows up mostly when half of the experts go, and on long agentic tasks. That setting matters to anyone trying to fit a large model on a smaller machine.
For teams that run open models on their own servers, as the digital bank Revolut does with a self-hosted GLM model, memory sets the hardware bill. According to Filip Lebiecki’s GitHub page, Qwen3.5-35B-A3B weighs 67 GiB in its original 16-bit format, over four times a 16 GB card’s memory. Qwen3.5-35B-A3B and the HOPE code are both under the Apache 2.0 license, which allows modified versions to be distributed.
The headline result rests on one model, one calibration dataset and one test. Other methods sometimes do better, and general-test scores drop on two of the three models. On LiveCodeBench, the paper’s other coding test, the code-calibrated pruned model drops to 48.0, from 64.3 for the full model.
Several questions remain open: how HOPE-pruned models hold up once converted to GGUF and compressed, and how they handle long tasks outside coding. No published test covers French. Results for a variant that spreads pruning across layers are still preliminary, the authors say. The first HOPE-pruned models on Hugging Face and independent tests will show whether the coding score holds outside Amazon’s labs.
Sources
- Your Experts Have Partners. There’s HOPE for Keeping Them Together, Amazon (Hugging Face blog), November 9, 2026
- Higher-order pruning of experts in mixture-of-experts language models, Tseng et al. (AWS), arXiv, September 29, 2026
- awslabs/HOPE, Amazon Web Services (GitHub), September 30, 2026
- Qwen3.5-35B-A3B, Qwen (Hugging Face), February 24, 2026
- REAP: One-Shot Pruning for Trillion-Parameter Mixture-of-Experts Models, Cerebras, October 16, 2025
- Shrinking Qwen3.5-35B-A3B: MoE expert pruning, distillation, quantization, Filip Lebiecki (GitHub), October 24, 2026
Header image: MACCUS chart, data from Tseng et al. (AWS), arXiv 2609.18916, Table S4
