Installation Guide TensorRT with pip on Ubuntu

TensorRT is an inference optimization library from NVIDIA for accelerating deep learning models (ONNX, PyTorch, TensorFlow) on NVIDIA GPUs. The pip install method is suitable for Python-based inference, not C++ development.

This tutorial focuses on TensorRT via pip, not via the .deb or full SDK.

1. Mandatory Prerequisites

Ensure all of the following components are available:

NVIDIA GPU

CUDA Toolkit installed

NVIDIA Docker not required, but recommended

Check GPU:

nvidia-smi

Check CUDA:

nvcc --version

If CUDA is missing, TensorRT will not work even if pip install is successful.

2. Check the CUDA Version Used

TensorRT is very sensitive to CUDA version. Note the CUDA version from:

nvidia-smi

Example:

CUDA Version: 12.2

This is important for selecting a compatible version of TensorRT.

3. Upgrade pip and base dependencies

python3 -m pip install --upgrade pip setuptools wheel

Using a virtual environment is recommended:

python3 -m venv trt-env
source trt-env/bin/activate

4. Install TensorRT Using pip

TensorRT in pip is not available on the public PyPI, but is in the NVIDIA repository.

Step 1: Install NVIDIA PyPI index

pip install nvidia-pyindex

Step 2: Install TensorRT

pip install nvidia-tensorrt

If you want a specific version:

pip install nvidia-tensorrt==9.2.0

5. Verify TensorRT Installation

Enter Python:

python

Then run:

import tensorrt as trt
print(trt.version)

If the TensorRT version appears without errors, the installation was successful.

6. Verify CUDA Access from TensorRT

import tensorrt as trt

logger = trt.Logger(trt.Logger.WARNING)
print("TensorRT OK, CUDA ready")

If no libnvinfer.so or libcuda.so errors appear, the environment is correct.

7. Common Errors and Solutions

Error: libnvinfer.so not found

Cause:

NVIDIA driver mismatch

export LDLIBRARYPATH=/usr/local/cuda/lib64:$LDLIBRARYPATH

echo 'export LDLIBRARYPATH=/usr/local/cuda/lib64:$LDLIBRARYPATH' >> ~/.bashrc
source ~/.bashrc

NVIDIA driver version is lower than TensorRT requirements

Solution:

Use an older version of TensorRT and Compatible

8. Important Notes (Must Read)

Not suitable for:

Compiling the TensorRT engine via C++

NVIDIA Docker + TensorRT image

• Or install TensorRT via .deb

9. Brief Summary

NVIDIA + CUDA Drivers
↓
pip install nvidia-pyindex
↓
pip install nvidia-tensorrt
↓
import tensorrt
↓
TensorRT is ready for inference