Jan nano AWQ
by warshanks
Open source · 118k downloads · 3 likes
0.8
(3 reviews)ChatAPI & LocalAbout
Jan nano AWQ is a compact 4-billion-parameter language model specifically designed for in-depth research tasks. Optimized to run on MCP servers, it stands out for its native integration with various tools and data sources, enhancing its efficiency in augmented research environments. Though marketed as a "non-thinking" model, it delivers strong performance on benchmarks like SimpleQA, largely due to its ability to leverage external tools via the MCP protocol. Its primary use cases include data analysis, information synthesis, and research assistance—all with a focus on precision and autonomy. What sets it apart is its balance between lightweight design and high performance, offering a robust solution for local and private applications.
Documentation
Jan-Nano: An Agentic Model
Note: Jan-Nano is a non-thinking model.
Authors: Alan Dao, Bach Vu Dinh
Overview
Jan-Nano is a compact 4-billion parameter language model specifically designed and trained for deep research tasks. This model has been optimized to work seamlessly with Model Context Protocol (MCP) servers, enabling efficient integration with various research tools and data sources.
Evaluation
Jan-Nano has been evaluated on the SimpleQA benchmark using our MCP-based benchmark methodology, demonstrating strong performance for its model size:
The evaluation was conducted using our MCP-based benchmark approach, which assesses the model's performance on SimpleQA tasks while leveraging its native MCP server integration capabilities. This methodology better reflects Jan-Nano's real-world performance as a tool-augmented research model, validating both its factual accuracy and its effectiveness in MCP-enabled environments.
How to Run Locally
Jan-Nano is currently supported by Jan, an open-source ChatGPT alternative that runs entirely on your computer. Jan provides a user-friendly interface for running local AI models with full privacy and control.
For non-jan app or tutorials there are guidance inside community section, please check those out! Discussion
VLLM
Here is an example command you can use to run vllm with Jan-nano
CSS
vllm serve Menlo/Jan-nano --host 0.0.0.0 --port 1234 --enable-auto-tool-choice --tool-call-parser hermes --chat-template ./qwen3_nonthinking.jinja
Chat-template is already included in tokenizer so chat-template is optional, but in case it has issue you can download the template here Non-think chat template
Recommended Sampling Parameters
- Temperature: 0.7
- Top-p: 0.8
- Top-k: 20
- Min-p: 0
📄 Citation
Bibtex
@misc{dao2025jannanotechnicalreport,
title={Jan-nano Technical Report},
author={Alan Dao and Dinh Bach Vu},
year={2025},
eprint={2506.22760},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2506.22760},
}
Documentation
Capabilities & Tags
transformerssafetensorsqwen3text-generationconversationaltext-generation-inferenceendpoints_compatiblecompressed-tensors
Links & Resources