How to debug Turbomind¶
Turbomind is implemented in C++, which is not as easy to debug as Python. This document provides basic methods for debugging Turbomind.
Prerequisite¶
First, complete the local compilation according to the commands in Build in localhost.
Configure Python debug environment¶
Since many large companies currently use Centos 7 for online production environments, we will use Centos 7 as an example to illustrate the process.
Obtain glibc and python3 versions¶
rpm -qa | grep glibc
rpm -qa | grep python3
The result should be similar to this:
[username@hostname workdir]# rpm -qa | grep glibc
glibc-2.17-325.el7_9.x86_64
glibc-common-2.17-325.el7_9.x86_64
glibc-headers-2.17-325.el7_9.x86_64
glibc-devel-2.17-325.el7_9.x86_64
[username@hostname workdir]# rpm -qa | grep python3
python3-pip-9.0.3-8.el7.noarch
python3-rpm-macros-3-34.el7.noarch
python3-rpm-generators-6-2.el7.noarch
python3-setuptools-39.2.0-10.el7.noarch
python3-3.6.8-21.el7_9.x86_64
python3-devel-3.6.8-21.el7_9.x86_64
python3.6.4-sre-1.el6.x86_64
Based on the information above, we can see that the version of glibc is 2.17-325.el7_9.x86_64 and the version of python3 is 3.6.8-21.el7_9.x86_64.
Download and install debuginfo library¶
Download glibc-debuginfo-common-2.17-325.el7.x86_64.rpm, glibc-debuginfo-2.17-325.el7.x86_64.rpm, and python3-debuginfo-3.6.8-21.el7.x86_64.rpm from http://debuginfo.centos.org/7/x86_64.
rpm -ivh glibc-debuginfo-common-2.17-325.el7.x86_64.rpm
rpm -ivh glibc-debuginfo-2.17-325.el7.x86_64.rpm
rpm -ivh python3-debuginfo-3.6.8-21.el7.x86_64.rpm
Upgrade GDB¶
sudo yum install devtoolset-10 -y
echo "source scl_source enable devtoolset-10" >> ~/.bashrc
source ~/.bashrc
Verification¶
gdb python3
The output should be similar to this:
[username@hostname workdir]# gdb python3
GNU gdb (GDB) Red Hat Enterprise Linux 9.2-10.el7
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python3...
(gdb)
If it shows Reading symbols from python3, the configuration has been successful.
For other operating systems, please refer to DebuggingWithGdb.
Set up symbolic links¶
After setting up symbolic links, there is no need to install it locally with pip every time.
# Change directory to lmdeploy, e.g.
cd /workdir/lmdeploy
# Since it has been built in the build directory
# Link the lib directory
cd lmdeploy && ln -s ../build/lib . && cd ..
# (Optional) Link compile_commands.json for clangd index
ln -s build/compile_commands.json .
Start debugging¶
# Use gdb to start the API server with Llama-2-13b-chat-hf, e.g.
gdb --args python3 -m lmdeploy serve api_server /workdir/Llama-2-13b-chat-hf
# Set directories in gdb
Reading symbols from python3...
(gdb) set directories /workdir/lmdeploy
# Set a breakpoint using the relative path, e.g.
(gdb) b src/turbomind/models/llama/BlockManager.cc:104
# When it shows
# ```
# No source file named src/turbomind/models/llama/BlockManager.cc.
# Make breakpoint pending on future shared library load? (y or [n])
# ```
# Just type `y` and press enter
# Run
(gdb) r
# (Optional) Use https://github.com/InternLM/lmdeploy/blob/main/benchmark/profile_restful_api.py to send a request
python3 profile_restful_api.py --server_addr 127.0.0.1:23333 --tokenizer_path /workdir/Llama-2-13b-chat-hf --dataset /workdir/ShareGPT_V3_unfiltered_cleaned_split.json --concurrency 1 --num_prompts 1
Using GDB¶
Refer to GDB Execution Commands and happy debugging.