Skip to content
Türkçe

Alibaba · Models · Chips and hardware

Janas runs a 48 GB AI model on a laptop with 32 GB of RAM

Published: 16 sourcesTürkçe

Janas, an open-source engine written in C by developer Maurizio Cammalleri, runs the 48 GB Qwen3-Next-80B model on a laptop with 32 GB of RAM. The model is bigger than the computer’s memory. The project reached the Hacker News front page on October 13.

The model uses a “mixture of experts” design: it has 80 billion parameters, but only about 3 billion of them work on each word. Janas keeps the experts it uses most in memory and reads the rest from the SSD when it needs them, instead of leaving those choices to the operating system.

The measurements were made on a laptop with no separate graphics card: an Intel Core Ultra 9 185H processor with its built-in Arc GPU, 32 GB of RAM and a 1 TB NVMe SSD that reads about 5.7 GB per second. The developer describes the project’s aim as “to lower the hardware that private AI needs.”

llama.cpp is a widely used open-source tool for running models on personal computers. On the developer’s laptop, Janas writes 24 tokens per second where llama.cpp manages 2.8. With the model’s own draft block, writing speed rises to 37 tokens per second. When reading, Janas handles 62 tokens per second against 16 for llama.cpp.

In the developer’s table, Janas is faster than llama.cpp on all 14 models.

The only independent test was run in late September by a user on another laptop with 64 GB of RAM, using an older version of Janas. When the model fit in memory, llama.cpp was faster or tied. On a 142 GB, 235-billion-parameter model bigger than memory, Janas was about twice as fast: 3 tokens per second against 1.6. The developer says this is the case Janas was built for.

But the headline numbers come from the developer’s single laptop, and no test has yet confirmed the current version on another machine. On Hacker News, commenters pointed out that llama.cpp has also been able to read models from disk for years. Janas only runs on Linux and Windows for now, and its own developer calls it “an experiment, not a product.”

Sources

  1. GitHub (prabanta-dev/janas), “Janas”, Project page: headline figures, llama.cpp comparison table, "Read this first" notice, test machine, license, (github.com)
  2. Hugging Face Forums, “Janas-LLM 37 tok/s vs llama.cpp 2.8 tok/s on Qwen3-Next-80B, on an Intel Core Ultra 9 185H with 32 GB RAM”, The developer's post: 37 with the model's own draft block, 24 vs 2.8 like for like, (discuss.huggingface.co)
  3. Hacker News, “Janas: Run LLMs larger than RAM by streaming experts from SSD”, HN thread; criticism: llama.cpp has read from disk for years, (news.ycombinator.com)
  4. Hacker News, October 13 front-page archive: Janas at No. 7, (news.ycombinator.com)
  5. DEV Community, “Run a 48 GB AI Model on a 32 GB Laptop: Janas Streams Model Experts From Your SSD at 37 tok/s”, Summary of the project and the Hacker News discussion, (dev.to)
  6. daily.dev, “Janas: Run LLMs larger than RAM by streaming experts from SSD”, Summary sourced from Hacker News, (daily.dev)
  7. GitHub (prabanta-dev/janas), “janas/ChangeLog.md at main · prabanta-dev/janas”, Release notes: October 11 measurements, October 12 Windows build, (github.com)
  8. GitHub (prabanta-dev/janas), “Test report: Intel(R) Core(TM) i9-14900HX, full · Issue #13 · prabanta-dev/janas”, The only independent test (older version, another 64 GB laptop): llama.cpp ahead on 30B, tied on 80B, Janas about 2x faster on the larger-than-memory 235B (table on slide 6), (github.com)
  9. GitHub (prabanta-dev/janas), “janas/AUTHORS at main · prabanta-dev/janas”, The project's aim: "to lower the hardware that private AI needs" (github.com)
  10. GitHub (prabanta-dev/janas), “janas/docs/hardware.md at main · prabanta-dev/janas”, Supported model families; discrete GPUs not yet tried (github.com)
  11. GitHub (prabanta-dev/janas), “Release Janas v2026.10.13 · prabanta-dev/janas · GitHub”, Prebuilt packages: Linux and Windows 10/11 (x86-64), (github.com)
  12. Hugging Face, “prabanta-dev”, Models converted to the Janas format, (huggingface.co)
  13. Hugging Face Forums, “Call for testers: Janas, a CPU-only engine that runs MoE models larger than RAM — one command to test your machine - Community Calls - Hugging Face Forums”, The developer's call for testers: "built for ordinary machines: no GPU needed", (discuss.huggingface.co)
  14. GitHub (ggml-org/llama.cpp), “GitHub - ggml-org/llama.cpp: LLM inference in C/C++ · GitHub”, The llama.cpp in the comparison: an open-source C/C++ tool for running language models locally (github.com)
  15. Hugging Face (Qwen), “Qwen/Qwen3-Next-80B-A3B-Instruct”, Model card: 80 billion total, 3 billion activated parameters (huggingface.co)
  16. Hugging Face (Qwen), “Qwen/Qwen3-Next-80B-A3B-Instruct-GGUF”, Q4_K_M file, 48.4 GB (huggingface.co)

About this story

Spotted an error in this story? [email protected] · Instagram

Short link: thejarrus.com/en/janas-big-model

This story in Turkish: Janas, 48 GB’lık bir yapay zeka modelini 32 GB RAM’li laptopta çalıştırıyor

On the same topic