Skip to main content

Ctrl+K

Get Started

Installation
Quick Start
On Other Platforms
- Get Started with Huawei Ascend (Atlas 800T A2 & Atlas 300I Duo)

Models

Supported Models

Large Language Models(LLMs) Deployment

Offline Inference Pipeline
OpenAI Compatible Server
Tools Calling
Reasoning Outputs
Serving LoRA
WebUI Demo
Request Distributor Server

Vision-Language Models(VLMs) Deployment

Offline Inference Pipeline
OpenAI Compatible Server
Vision-Language Models
- DeepSeek-VL2
- LLaVA
- InternVL
- InternLM-XComposer-2.5
- CogVLM
- MiniCPM-V
- Phi-3 Vision
- Mllama
- Qwen2-VL
- Qwen2.5-VL
- Molmo
- Gemma3

Quantization

AWQ/GPTQ
SmoothQuant
INT4/INT8 KV Cache

Benchmark

Benchmark
Evaluate LLMs with OpenCompass

Advanced Guide

Architecture of TurboMind
Architecture of lmdeploy.pytorch
lmdeploy.pytorch New Model Support
Context length extrapolation
Customized chat template
How to debug Turbomind
Structured output
PyTorchEngine Multi-Node Deployment Guide

API Reference

inference pipeline

Repository
Show source
Suggest edit
Open issue

.rst

On Other Platforms

On Other Platforms#

NPU(Huawei)

Get Started with Huawei Ascend (Atlas 800T A2 & Atlas 300I Duo)

previous

Quick Start

next

Get Started with Huawei Ascend (Atlas 800T A2 & Atlas 300I Duo)

By LMDeploy Authors

© Copyright 2021-2024, OpenMMLab.

Last updated on May 20, 2025.