模型服务

部署 LLaMA-2 服务

请从这里 下载 llama2 模型，参考如下命令部署服务：

7B

lmdeploy convert llama2 /path/to/llama-2-7b-chat-hf
bash workspace/service_docker_up.sh

13B

lmdeploy convert llama2 /path/to/llama-2-13b-chat-hf --tp 2
bash workspace/service_docker_up.sh

70B

lmdeploy convert llama2 /path/to/llama-2-70b-chat-hf --tp 8
bash workspace/service_docker_up.sh

部署 LLaMA 服务

请填写这张表，获取 LLaMA 模型权重

7B

lmdeploy convert llama /path/to/llama-7b llama \
    --tokenizer_path /path/to/tokenizer/model
bash workspace/service_docker_up.sh

13B

lmdeploy convert llama /path/to/llama-13b llama \
    --tokenizer_path /path/to/tokenizer/model --tp 2
bash workspace/service_docker_up.sh

30B

lmdeploy convert llama /path/to/llama-30b llama \
    --tokenizer_path /path/to/tokenizer/model --tp 4
bash workspace/service_docker_up.sh

65B

lmdeploy convert llama /path/to/llama-65b llama \
    --tokenizer_path /path/to/tokenizer/model --tp 8
bash workspace/service_docker_up.sh

部署 Vicuna 服务

7B

python3 -m pip install fschat
python3 -m fastchat.model.apply_delta \
  --base-model-path /path/to/llama-7b \
  --target-model-path /path/to/vicuna-7b \
  --delta-path lmsys/vicuna-7b-delta-v1.1

lmdeploy convert vicuna /path/to/vicuna-7b
bash workspace/service_docker_up.sh

13B

python3 -m pip install fschat
python3 -m fastchat.model.apply_delta \
  --base-model-path /path/to/llama-13b \
  --target-model-path /path/to/vicuna-13b \
  --delta-path lmsys/vicuna-13b-delta-v1.1

lmdeploy convert vicuna /path/to/vicuna-13b
bash workspace/service_docker_up.sh