@@ -96,7 +96,7 @@ building with Docker.
9696
9797* In the * build* subdirectory of the server repo, generate the
9898 docker_build script, the cmake_build script and the Dockerfiles
99- needed to build Triton. If you use the --dryrun flag, build.py will
99+ needed to build Triton. If you use the ` --dryrun ` flag, build.py will
100100 stop here so that you can examine these files.
101101
102102* Run the docker_build script to perform the Docker-based build. The
@@ -105,7 +105,7 @@ building with Docker.
105105 * Build the * tritonserver_buildbase* Docker image that collects all
106106 the build dependencies needed to build Triton. The
107107 * tritonserver_buildbase* image is based on a minimal/base
108- image. When building with GPU support (--enable-gpu), the * min*
108+ image. When building with GPU support (` --enable-feature gpu ` ), the * min*
109109 image is the
110110 [ \< xx.yy\> -py3-min] ( https://catalog.ngc.nvidia.com/orgs/nvidia/containers/tritonserver )
111111 image pulled from [ NGC] ( https://ngc.nvidia.com ) that contains the
@@ -151,15 +151,16 @@ building with Docker.
151151
152152By default, build.py does not enable any of Triton's optional features
153153but you can enable all features, backends, and repository agents with
154- the --enable-all flag. The -v flag turns on verbose output.
154+ the ` --enable-all ` flag. The ` -v ` flag turns on verbose output.
155155
156156``` bash
157157$ ./build.py -v --enable-all
158158```
159159
160- If you want to enable only certain Triton features, backends and
161- repository agents, do not specify --enable-all. Instead you must
162- specify the individual flags as documented by --help.
160+ If you want to enable only certain Triton features, backends, and
161+ repository agents, there are two options:
162+ a. do not specify ` --enable-all ` , and instead specify the individual flags as documented by ` --help ` .
163+ b. specify ` --enable-all ` and then disable selected features that you wish to omit using the ` --disable-... ` arguments, also documented by ` --help ` .
163164
164165#### Building With Specific GitHub Branches and Organization
165166
@@ -170,7 +171,7 @@ other repos, but if you want to control which branch is used in these
170171other repos you can as shown in the following example.
171172
172173``` bash
173- $ ./build.py ... --repo -tag= common: < container tag> --repo -tag= core: < container tag> --repo -tag= backend: < container tag> --repo -tag= thirdparty: < container tag> ... --backend= tensorrt: < container tag> ... --repoagent= checksum: < container tag> ...
174+ $ ./build.py ... --component -tag common < container tag> --component -tag core < container tag> --component -tag backend < container tag> --component -tag thirdparty < container tag> ... --backend-tag tensorrt < container tag> ... --repoagent-tag checksum < container tag> ...
174175```
175176
176177If you are building on a release branch then ` <container tag> ` will
@@ -184,26 +185,26 @@ instead use the corresponding branch/tag in the build. For example, if
184185you have a branch called "mybranch" in the
185186[ onnxruntime_backend] ( https://github.com/triton-inference-server/onnxruntime_backend )
186187repo that you want to use in the build, you would specify
187- ` --backend= onnxruntime: mybranch ` .
188+ ` --backend-tag onnxruntime mybranch ` .
188189
189- If you want to build a backend from an alternative organization or user ` <org> ` , you can extend this syntax as follows :
190+ If you want to build a backend from an alternative organization or user ` <org> ` , you can include a similar argument :
190191``` bash
191- $ ./build.py ... --backend= onnxruntime:mybranch: https://github.com/< org>
192+ $ ./build.py ... --backend-org onnxruntime https://github.com/< org>
192193```
193194
194195#### CPU-Only Build
195196
196197If you want to build without GPU support you must specify individual
197- feature flags and not include the ` --enable-gpu ` and
198- ` --enable-gpu-metrics ` flags. Only the following backends are
198+ feature flags and not include the ` --enable-feature gpu ` and
199+ ` --enable-feature gpu-metrics ` flags. Only the following backends are
199200available for a non-GPU / CPU-only build: ` identity ` , ` repeat ` , ` ensemble ` ,
200201` square ` , ` pytorch ` , ` onnxruntime ` , ` openvino ` ,
201202` python ` and ` fil ` .
202203
203204CPU-only builds of the PyTorch backends require some CUDA stubs
204205and runtime dependencies that are not present in the CPU-only base container.
205206These are retrieved from a GPU base container, which can be changed with the
206- ` --image= gpu-base, nvcr.io/nvidia/tritonserver:<xx.yy>-py3-min ` flag.
207+ ` --image gpu-base nvcr.io/nvidia/tritonserver:<xx.yy>-py3-min ` flag.
207208
208209### Building Without Docker
209210
@@ -216,16 +217,16 @@ repo branch for the release you are interested in building (or the
216217* main* branch to build from the development branch).
217218
218219To determine what dependencies are required by the build, run build.py
219- with the --dryrun flag, and then looking in the build subdirectory at
220+ with the ` --dryrun ` flag, and then looking in the build subdirectory at
220221Dockerfile.buildbase.
221222
222223``` bash
223224$ ./build.py -v --enable-all
224225```
225226
226227From Dockerfile.buildbase you can see what dependencies you need to
227- install on your host system. Note that when building with --enable-gpu
228- (or --enable-all), Dockerfile.buildbase depends on the
228+ install on your host system. Note that when building with ` --enable-feature gpu `
229+ (or ` --enable-all ` ), Dockerfile.buildbase depends on the
229230[ \< xx.yy\> -py3-min] ( https://catalog.ngc.nvidia.com/orgs/nvidia/containers/tritonserver )
230231image pulled from [ NGC] ( https://ngc.nvidia.com ) . Unfortunately, a
231232Dockerfile is not currently available for the
@@ -235,7 +236,7 @@ cuDNN](#cuda-cublas-cudnn) and [TensorRT](#tensorrt) dependencies as
235236described below.
236237
237238Once you have installed these dependencies on your build system you
238- can then use build.py with the --no-container-build flag to build
239+ can then use build.py with the ` --no-container-build ` flag to build
239240Triton.
240241
241242``` bash
@@ -278,8 +279,8 @@ difference is that the minimal/base image used as the base of
278279Dockerfile.buildbase image can be built from the provided
279280[ Dockerfile.win10.min] ( https://github.com/triton-inference-server/server/blob/main/Dockerfile.win10.min )
280281file as described in [ Windows 10 "Min" Image] ( #windows-10-min-image ) . When running build.py
281- use the --image flag to specify the tag that you assigned to this
282- image. For example, --image= base, win10-py3-min.
282+ use the ` --image ` flag to specify the tag that you assigned to this
283+ image. For example, ` --image base win10-py3-min ` .
283284
284285### Windows and Docker
285286
@@ -327,7 +328,7 @@ and so you must enable them explicitly. The following build.py
327328invocation builds all features and backends available on windows.
328329
329330``` bash
330- python build.py --cmake-dir= < path/to/repo> /build --build-dir= /tmp/citritonbuild --no-container-pull --image= base, win10-py3-min --enable-logging --enable-stats --enable-tracing --enable-gpu --endpoint= grpc --endpoint= http --repo -tag= common: < container tag> --repo -tag= core: < container tag> --repo -tag= backend: < container tag> --repo -tag= thirdparty: < container tag> --backend= ensemble --backend= tensorrt: < container tag> --backend= onnxruntime: < container tag> --backend= openvino: < container tag> --backend= python: < container tag>
331+ python build.py --cmake-dir < path/to/repo> /build --build-dir /tmp/citritonbuild --no-container-pull --image base win10-py3-min --enable-feature logging --enable-feature stats --enable-feature tracing --enable-feature gpu --enable- endpoint grpc --enable- endpoint http --component -tag common < container tag> --component -tag core < container tag> --component -tag backend < container tag> --component -tag thirdparty < container tag> --enable- backend ensemble --enable- backend tensorrt --backend-tag tensorrt < container tag> --enable- backend onnxruntime --backend-tag onnxruntime < container tag> --enable- backend openvino --backend-tag openvino < container tag> --enable- backend python --backend-tag python < container tag>
331332```
332333
333334If you are building on * main* branch then ` <container tag> ` will
@@ -341,7 +342,13 @@ branch/tag in the build. For example, if you have a branch called
341342"mybranch" in the
342343[ onnxruntime_backend] ( https://github.com/triton-inference-server/onnxruntime_backend )
343344repo that you want to use in the build, you would specify
344- --backend=onnxruntime: mybranch .
345+ repo that you want to use in the build, you would specify
346+ ` --backend-tag onnxruntime mybranch ` .
347+
348+ If you want to build a backend from an alternative organization or user ` <org> ` , you can include a similar argument:
349+ ``` bash
350+ python build.py ... --backend-org onnxruntime https://github.com/< org>
351+ ```
345352
346353### Extract Build Artifacts
347354
@@ -401,7 +408,7 @@ and cmake_build or the equivalent commands to perform a build.
401408 depends on that package. For example, Triton supports the S3
402409 filesystem by building the aws-sdk-cpp package. If aws-sdk-cpp
403410 doesn't build for your platform then you can remove the need for
404- that package by not specifying -- filesystem=s3 when you run
411+ that package by not specifying ` --enable- filesystem s3 ` when you run
405412 build.py. In general, you should start by running build.py with the
406413 minimal required feature set.
407414
@@ -503,7 +510,7 @@ re-running `make` (or `msbuild.exe`).
503510
504511### Building with Debug Symbols
505512
506- To build with Debug symbols, use the --build-type= Debug argument while
513+ To build with Debug symbols, use the ` --build-type Debug ` argument while
507514launching build.py. If building directly with CMake use
508- -DCMAKE_BUILD_TYPE=Debug. You can then launch the built server with
515+ ` -DCMAKE_BUILD_TYPE=Debug ` . You can then launch the built server with
509516gdb and see the debug symbols/information in the gdb trace.
0 commit comments