Artificial Intelligence1 min read

Kimi K3.1 Pro model weights are out

By · Published by Everything Blog

In short

The architecture features an extensive 1,048,576-token context window designed to assist with complex tasks including 3D creation and general agentic workflows. Under the hood, it incorporates a specialized 401-million-parameter MoonViT-V2 vision encoder alongside an attention structure composed of 69 KDA layers and 24 Gated DeepSeek Sparse Attention layers. Developers can immediately deploy and run the system across multiple popular platforms, including Hugging Face Transformers, vLLM, and SGLang.

Key points

  • Moonshot AI's latest multimodal agentic model supports both text and image inputs for div…: Moonshot AI's latest multimodal agentic model supports both text and image inputs for diverse downstream tasks.
  • High computational efficiency is achieved through 896 experts and sparse attention layers…: High computational efficiency is achieved through 896 experts and sparse attention layers paired with advanced quantization formats.
  • Broad open accessibility is enabled through pre-configured support for Docker containers,…: Broad open accessibility is enabled through pre-configured support for Docker containers, Google Colab, Kaggle, and HuggingChat.

Sources checked by our analysis pipeline: 1

Instructions to use moonshotai/Kimi-K3.1-Pro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

- Libraries

- Transformers

How to use moonshotai/Kimi-K3.1-Pro with Transformers:

# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="moonshotai/Kimi-K3.1-Pro", trust_remote_code=True)

# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("moonshotai/Kimi-K3.1-Pro", trust_remote_code=True, device_map="auto")

- Inference

- HuggingChat

- Notebooks

- Google Colab

- Kaggle

- Local Apps Settings

- vLLM

How to use moonshotai/Kimi-K3.1-Pro with vLLM:

Install from pip and serve model

# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "moonshotai/Kimi-K3.1-Pro" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "moonshotai/Kimi-K3.1-Pro", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'

Use Docker

docker model run hf.co/moonshotai/Kimi-K3.1-Pro

- SGLang

How to use moonshotai/Kimi-K3.1-Pro with SGLang:

Install from pip and serve…

Artificial Intelligence

More in Artificial Intelligence