Technology Document Guide of TensorRT

Abstract

本示例支持指南概述了GitHub和产品包中包含的所有受支持的TensorRT 7.2.1示例。TensorRT示例在推荐程序、机器翻译、字符识别、图像分类和对象检测等领域有特殊帮助。有关TensorRT开发文档，请参阅TensorRT归档文件。

Introduction

下面的示例展示了如何在许多用例中使用TensorRT，同时突出显示接口的不同功能。
Technology Document Guide of TensorRT

1.1. Getting Started With C++ Samples

You can find the C++ samples in the
/usr/src/tensorrt/samples package directory as well as on GitHub. The following C++ samples are shipped with TensorRT.

“Hello World” For TensorRT Building A Simple MNIST Network Layer By Layer Importing The TensorFlow Model And Running Inference “Hello World” For TensorRT From ONNX Building And Running GoogleNet In TensorRT
Building An RNN Network Layer By Layer Performing Inference In INT8 Using Custom Calibration Performing Inference In INT8 Precision Adding A Custom Layer To Your Network In TensorRT Object Detection With Faster R-CNN
Object Detection With A TensorFlow SSD Network Movie Recommendation Using Neural Collaborative Filter (NCF)
Movie Recommendation Using MPS (Multi-Process Service)
Object Detection With SSD
“Hello World” For Multilayer Perceptron (MLP)
Specifying I/O Formats Using The Reformat Free I/O APIs Adding A Custom Layer That Supports INT8 I/O To Your Network In TensorRT Digit Recognition With Dynamic Shapes In TensorRT Neural Machine Translation (NMT) Using A Sequence To Sequence (seq2seq) Model Object Detection And Instance Segmentation With A TensorFlow Mask R-CNN Network Object Detection With A TensorFlow Faster R-CNN NetworkAlgorithm Selection API Usage Example Based On sampleMNIST In TensorRT1

Getting Started With C++ Samples

每个C++样本包括一个GitHub中的README.md文件，该文件提供有关示例如何工作的详细信息、示例代码以及有关如何运行和验证其输出的分步说明。

Running C++ Samples on Linux

如果使用Debian文件安装TensorRT，在构建C++示例之前，首先复制/usr/src/tensorrt到新目录。如果使用tar文件安装了TensorRT，那么示例位于{TAR_EXTRACT_PATH}/samples中。要生成所有示例，然后运行其中一个示例，请使用以下命令：

$ cd <samples_dir>

$ make -j4

$ cd …/bin

$ ./<sample_bin>

Running C++ Samples on Windows

Windows上的所有C++样本都作为VisualStudio解决方案文件提供。若要生成示例，请打开其相应的VisualStudio解决方案文件并生成解决方案。输出可执行文件将在（ZIP_EXTRACT_PATH）\bin中生成。然后可以直接或通过visual studio运行可执行文件。

1.2. Getting Started With Python Samples

可以在 /usr/src/tensorrt/samples/python包目录中找到Python示例。以下Python示例随TensorRT一起提供。

Introduction To Importing Caffe, TensorFlow And ONNX Models Into TensorRT Using Python “Hello World” For TensorRT Using TensorFlow And Python “Hello World” For TensorRT Using PyTorch And Python Adding A Custom Layer To Your TensorFlow Network In TensorRT In Python Object Detection With The ONNX TensorRT Backend In Python Object Detection With SSD In Python INT8 Calibration In Python Refitting An Engine In Python TensorRT Inference Of ONNX Models With Custom Layers In Python

Getting Started With Python Samples

每个Python示例都包含README.md文件。请参阅

/usr/src/tensorrt/samples/python//README.md文件获取有关示例如何工作的详细信息、示例代码以及有关如何运行和验证其输出的分步说明。

Running Python Samples

要运行其中一个Python示例，该过程通常包括两个步骤：

Install the sample requirements:

python -m pip install -r requirements.txt

where python is either python2 or python3.

Run the sample code with the data directory provided if the TensorRT sample data is not in the default location. For example:

python sample.py [-d DATA_DIR]

For more information on running samples, see the README.md file included with the sample.