Skip to content

Commit c275890

Browse files
committed
update usage in documentation
1 parent 13bfbfa commit c275890

3 files changed

Lines changed: 32 additions & 26 deletions

File tree

build.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,6 @@
3232
import os.path
3333
import pathlib
3434
import platform
35-
import re
3635
import stat
3736
import subprocess
3837
import sys

docs/customization_guide/build.md

Lines changed: 31 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ building with Docker.
9696

9797
* In the *build* subdirectory of the server repo, generate the
9898
docker_build script, the cmake_build script and the Dockerfiles
99-
needed to build Triton. If you use the --dryrun flag, build.py will
99+
needed to build Triton. If you use the `--dryrun` flag, build.py will
100100
stop here so that you can examine these files.
101101

102102
* Run the docker_build script to perform the Docker-based build. The
@@ -105,7 +105,7 @@ building with Docker.
105105
* Build the *tritonserver_buildbase* Docker image that collects all
106106
the build dependencies needed to build Triton. The
107107
*tritonserver_buildbase* image is based on a minimal/base
108-
image. When building with GPU support (--enable-gpu), the *min*
108+
image. When building with GPU support (`--enable-feature gpu`), the *min*
109109
image is the
110110
[\<xx.yy\>-py3-min](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/tritonserver)
111111
image pulled from [NGC](https://ngc.nvidia.com) that contains the
@@ -151,15 +151,16 @@ building with Docker.
151151

152152
By default, build.py does not enable any of Triton's optional features
153153
but you can enable all features, backends, and repository agents with
154-
the --enable-all flag. The -v flag turns on verbose output.
154+
the `--enable-all` flag. The `-v` flag turns on verbose output.
155155

156156
```bash
157157
$ ./build.py -v --enable-all
158158
```
159159

160-
If you want to enable only certain Triton features, backends and
161-
repository agents, do not specify --enable-all. Instead you must
162-
specify the individual flags as documented by --help.
160+
If you want to enable only certain Triton features, backends, and
161+
repository agents, there are two options:
162+
a. do not specify `--enable-all`, and instead specify the individual flags as documented by `--help`.
163+
b. specify `--enable-all` and then disable selected features that you wish to omit using the `--disable-...` arguments, also documented by `--help`.
163164

164165
#### Building With Specific GitHub Branches and Organization
165166

@@ -170,7 +171,7 @@ other repos, but if you want to control which branch is used in these
170171
other repos you can as shown in the following example.
171172

172173
```bash
173-
$ ./build.py ... --repo-tag=common:<container tag> --repo-tag=core:<container tag> --repo-tag=backend:<container tag> --repo-tag=thirdparty:<container tag> ... --backend=tensorrt:<container tag> ... --repoagent=checksum:<container tag> ...
174+
$ ./build.py ... --component-tag common <container tag> --component-tag core <container tag> --component-tag backend <container tag> --component-tag thirdparty <container tag> ... --backend-tag tensorrt <container tag> ... --repoagent-tag checksum <container tag> ...
174175
```
175176

176177
If you are building on a release branch then `<container tag>` will
@@ -184,26 +185,26 @@ instead use the corresponding branch/tag in the build. For example, if
184185
you have a branch called "mybranch" in the
185186
[onnxruntime_backend](https://github.com/triton-inference-server/onnxruntime_backend)
186187
repo that you want to use in the build, you would specify
187-
`--backend=onnxruntime:mybranch`.
188+
`--backend-tag onnxruntime mybranch`.
188189

189-
If you want to build a backend from an alternative organization or user `<org>`, you can extend this syntax as follows:
190+
If you want to build a backend from an alternative organization or user `<org>`, you can include a similar argument:
190191
```bash
191-
$ ./build.py ... --backend=onnxruntime:mybranch:https://github.com/<org>
192+
$ ./build.py ... --backend-org onnxruntime https://github.com/<org>
192193
```
193194

194195
#### CPU-Only Build
195196

196197
If you want to build without GPU support you must specify individual
197-
feature flags and not include the `--enable-gpu` and
198-
`--enable-gpu-metrics` flags. Only the following backends are
198+
feature flags and not include the `--enable-feature gpu` and
199+
`--enable-feature gpu-metrics` flags. Only the following backends are
199200
available for a non-GPU / CPU-only build: `identity`, `repeat`, `ensemble`,
200201
`square`, `pytorch`, `onnxruntime`, `openvino`,
201202
`python` and `fil`.
202203

203204
CPU-only builds of the PyTorch backends require some CUDA stubs
204205
and runtime dependencies that are not present in the CPU-only base container.
205206
These are retrieved from a GPU base container, which can be changed with the
206-
`--image=gpu-base,nvcr.io/nvidia/tritonserver:<xx.yy>-py3-min` flag.
207+
`--image gpu-base nvcr.io/nvidia/tritonserver:<xx.yy>-py3-min` flag.
207208

208209
### Building Without Docker
209210

@@ -216,16 +217,16 @@ repo branch for the release you are interested in building (or the
216217
*main* branch to build from the development branch).
217218

218219
To determine what dependencies are required by the build, run build.py
219-
with the --dryrun flag, and then looking in the build subdirectory at
220+
with the `--dryrun` flag, and then looking in the build subdirectory at
220221
Dockerfile.buildbase.
221222

222223
```bash
223224
$ ./build.py -v --enable-all
224225
```
225226

226227
From Dockerfile.buildbase you can see what dependencies you need to
227-
install on your host system. Note that when building with --enable-gpu
228-
(or --enable-all), Dockerfile.buildbase depends on the
228+
install on your host system. Note that when building with `--enable-feature gpu`
229+
(or `--enable-all`), Dockerfile.buildbase depends on the
229230
[\<xx.yy\>-py3-min](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/tritonserver)
230231
image pulled from [NGC](https://ngc.nvidia.com). Unfortunately, a
231232
Dockerfile is not currently available for the
@@ -235,7 +236,7 @@ cuDNN](#cuda-cublas-cudnn) and [TensorRT](#tensorrt) dependencies as
235236
described below.
236237

237238
Once you have installed these dependencies on your build system you
238-
can then use build.py with the --no-container-build flag to build
239+
can then use build.py with the `--no-container-build` flag to build
239240
Triton.
240241

241242
```bash
@@ -278,8 +279,8 @@ difference is that the minimal/base image used as the base of
278279
Dockerfile.buildbase image can be built from the provided
279280
[Dockerfile.win10.min](https://github.com/triton-inference-server/server/blob/main/Dockerfile.win10.min)
280281
file as described in [Windows 10 "Min" Image](#windows-10-min-image). When running build.py
281-
use the --image flag to specify the tag that you assigned to this
282-
image. For example, --image=base,win10-py3-min.
282+
use the `--image` flag to specify the tag that you assigned to this
283+
image. For example, `--image base win10-py3-min`.
283284

284285
### Windows and Docker
285286

@@ -327,7 +328,7 @@ and so you must enable them explicitly. The following build.py
327328
invocation builds all features and backends available on windows.
328329

329330
```bash
330-
python build.py --cmake-dir=<path/to/repo>/build --build-dir=/tmp/citritonbuild --no-container-pull --image=base,win10-py3-min --enable-logging --enable-stats --enable-tracing --enable-gpu --endpoint=grpc --endpoint=http --repo-tag=common:<container tag> --repo-tag=core:<container tag> --repo-tag=backend:<container tag> --repo-tag=thirdparty:<container tag> --backend=ensemble --backend=tensorrt:<container tag> --backend=onnxruntime:<container tag> --backend=openvino:<container tag> --backend=python:<container tag>
331+
python build.py --cmake-dir <path/to/repo>/build --build-dir /tmp/citritonbuild --no-container-pull --image base win10-py3-min --enable-feature logging --enable-feature stats --enable-feature tracing --enable-feature gpu --enable-endpoint grpc --enable-endpoint http --component-tag common <container tag> --component-tag core <container tag> --component-tag backend <container tag> --component-tag thirdparty <container tag> --enable-backend ensemble --enable-backend tensorrt --backend-tag tensorrt <container tag> --enable-backend onnxruntime --backend-tag onnxruntime <container tag> --enable-backend openvino --backend-tag openvino <container tag> --enable-backend python --backend-tag python <container tag>
331332
```
332333

333334
If you are building on *main* branch then `<container tag>` will
@@ -341,7 +342,13 @@ branch/tag in the build. For example, if you have a branch called
341342
"mybranch" in the
342343
[onnxruntime_backend](https://github.com/triton-inference-server/onnxruntime_backend)
343344
repo that you want to use in the build, you would specify
344-
--backend=onnxruntime:mybranch.
345+
repo that you want to use in the build, you would specify
346+
`--backend-tag onnxruntime mybranch`.
347+
348+
If you want to build a backend from an alternative organization or user `<org>`, you can include a similar argument:
349+
```bash
350+
python build.py ... --backend-org onnxruntime https://github.com/<org>
351+
```
345352

346353
### Extract Build Artifacts
347354

@@ -401,7 +408,7 @@ and cmake_build or the equivalent commands to perform a build.
401408
depends on that package. For example, Triton supports the S3
402409
filesystem by building the aws-sdk-cpp package. If aws-sdk-cpp
403410
doesn't build for your platform then you can remove the need for
404-
that package by not specifying --filesystem=s3 when you run
411+
that package by not specifying `--enable-filesystem s3` when you run
405412
build.py. In general, you should start by running build.py with the
406413
minimal required feature set.
407414

@@ -503,7 +510,7 @@ re-running `make` (or `msbuild.exe`).
503510

504511
### Building with Debug Symbols
505512

506-
To build with Debug symbols, use the --build-type=Debug argument while
513+
To build with Debug symbols, use the `--build-type Debug` argument while
507514
launching build.py. If building directly with CMake use
508-
-DCMAKE_BUILD_TYPE=Debug. You can then launch the built server with
515+
`-DCMAKE_BUILD_TYPE=Debug`. You can then launch the built server with
509516
gdb and see the debug symbols/information in the gdb trace.

docs/user_guide/debugging_guide.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -129,7 +129,7 @@ The easiest step to start with is running perf_analyzer to get a breakdown of th
129129

130130
The next step would be to use a performance profiler. One profiler we recommend is [Nsight Systems](https://developer.nvidia.com/nsight-systems) (nsys), optionally including NVIDIA Tools Extension (NVTX) markers to profile Triton.
131131

132-
The Triton server container already has nsys installed. However, Triton does not build with the NVTX markers by default. If you want to use NVTX markers, you should build Triton with build.py, using the “--enable-nvtx” flag. This will provide details around some phases of processing a request, such as queueing, running inference, and handling outputs.
132+
The Triton server container already has nsys installed. However, Triton does not build with the NVTX markers by default. If you want to use NVTX markers, you should build Triton with build.py, using the “--enable-feature nvtx” flag. This will provide details around some phases of processing a request, such as queueing, running inference, and handling outputs.
133133

134134
You can profile Triton by running `nsys profile tritonserver --model-repository …`. The [nsys documentation](https://docs.nvidia.com/nsight-systems/UserGuide/index.html) provides more options and details for getting a thorough overview of what is going on.
135135

0 commit comments

Comments
 (0)