Access Embedl RF-DETR Base

To access this model, please review and accept the terms below. Your contact information is collected solely to manage access and, with your explicit consent, to notify you about updated or new optimized models from Embedl.

By requesting access you agree to the Embedl Models Community Licence and the upstream RF-DETR License

Optimized by Embedl

Need to fine-tune, hit performance targets, or deploy on specific hardware?

We've got you covered.

Learn more Get in touch →

Embedl RF-DETR Base (Quantized for TensorRT)

RF-DETR Base is a real-time object detector: for each object in an image it returns a bounding box, a class label and a confidence score.

Deployable INT8-quantized version of Roboflow/rf-detr-base, optimized with embedl-deploy for low-latency NVIDIA TensorRT inference on edge GPUs.

Upstream Model

Highlights

Mixed-precision INT8/FP16 quantization with hardware-aware optimizations from embedl-deploy: INT8 backbone, FP16 projector and decoder.
Drop-in replacement for Roboflow/rf-detr-base in TensorRT pipelines: same image input (560×560), same output semantics.
Measured on the Jetson with TensorRT: within 0.87 mAP points of the FP16 TensorRT baseline on all 5,000 COCO val2017 images, and 1.11× to 1.36× faster (see the tables below).
Saved at ONNX opset 19, so TensorRT 10.x (JetPack 6 and 7) builds it as intended; see TensorRT 10 compatibility.
Faster than trtexec --best on supported NVIDIA hardware (earlier revision, Orin Nano Super; see Performance below).
Includes both ONNX (for TensorRT) and PT2 (torch.export-loadable) artifacts plus runnable inference scripts.

Quick Start

pip install huggingface_hub onnxruntime-gpu pillow numpy
python -c "from huggingface_hub import snapshot_download; snapshot_download('embedl/rf-detr-base-quantized-tensorrt', local_dir='.')"
python infer_trt.py --image path/to/image.jpg   # TensorRT
# or
python infer_pt2.py --image path/to/image.jpg   # pure PyTorch via torch.export

Files

File	Purpose
`embedl_rf-detr-base_int8.onnx`	INT8/FP16 ONNX with Q/DQ nodes, ready for TensorRT (opset 19).
`embedl_rf-detr-base_int8.pt2`	The same model for PyTorch (`torch.export` ExportedProgram): INT8 backbone, projector and decoder unquantized.
`infer_trt.py`	Build a TRT engine from the ONNX and run sample inference.
`infer_pt2.py`	Load the `.pt2` with `torch.export.load` and run sample inference.

Accuracy

COCO val2017, all 5,000 images, pycocotools bbox mAP. Each row is a TensorRT 10.16.2.10 engine built and scored on the Jetson itself (JetPack 7.2): the image resized to a square 560×560, ImageNet normalization, top-300 over query × class. The baseline is upstream's own ONNX export (rfdetr 1.8.3) built with --fp16.

Device	TensorRT FP16 mAP@50-95	Embedl INT8 mAP@50-95	Difference	TensorRT FP16 mAP@50	Embedl INT8 mAP@50
Jetson AGX Orin	53.29	52.42	−0.87	72.14	71.60
Jetson Orin NX	53.30	52.43	−0.87	72.21	71.55
Jetson Orin Nano Super	53.28	52.43	−0.85	72.19	71.56
Jetson AGX Thor	53.29	52.43	−0.87	72.20	71.44

In PyTorch (the .pt2 model on the CPU: INT8 backbone, projector and decoder unquantized), COCO val2017, all 5,000 images, the same preprocessing and top-300 post-processing:

Model	mAP@50-95	mAP@50
`Roboflow/rf-detr-base` FP32 (ours)	53.30%	72.17%
Embedl RF-DETR Base INT8	52.35%	71.46%

In the same run, the earlier .pt2 (INT8 throughout, revision 5492ae0e) scores 52.15% / 71.24%.

Performance

GPU compute time per image, median: trtexec, CUDA Graph, spin wait, no data transfers (--noDataTransfers), 2 s warm-up and at least 1,000 timed runs, after a build with --builderOptimizationLevel=3 (--fp16, or --fp16 --int8 for Embedl INT8). TensorRT 10.16.2.10 on JetPack 7.2, batch 1, GPU clock locked at its maximum. Both builds were timed in the same job on the same board.

Device	TensorRT FP16	Embedl INT8	Speedup vs FP16
Jetson AGX Orin	5.99 ms	5.22 ms	1.15×
Jetson Orin NX	12.26 ms	10.61 ms	1.16×
Jetson Orin Nano Super	16.39 ms	12.02 ms	1.36×
Jetson AGX Thor	2.23 ms	2.01 ms	1.11×

Earlier measurement: revision 5492ae0e on the Jetson Orin Nano Super

Latency measured with TensorRT + trtexec, GPU compute time only (--noDataTransfers), CUDA Graph + Spin Wait enabled, clocks locked (max-performance nvpmodel mode + jetson_clocks on Jetson).

NVIDIA Jetson Orin Nano Super

Configuration	Mean Latency	Speedup vs FP16
TensorRT FP16	14.83 ms	1.00x
TensorRT --best (unconstrained)	14.87 ms	1.00x
Embedl Deploy INT8	11.77 ms	1.26x

Example Detections

Rendered from the earlier revision (5492ae0e).

Both panels in every figure below are decoded from the actual TensorRT engines running on the Jetson Orin Nano Super: inputs fed with trtexec --loadInputs, raw dets/labels outputs exported and drawn offline. Small per-detection score differences between the two precisions are expected for a two-stage DETR (proposal selection reorders under precision changes); the detection sets match.

RF-DETR Base FP16 vs Embedl INT8 detections - street scene

RF-DETR Base FP16 vs Embedl INT8 detections - living room

RF-DETR Base FP16 vs Embedl INT8 detections - COCO cats

Confidence-threshold sweep from 0.80 down to 0.30, both engines in sync; detections appear at nearly identical thresholds:

Synced confidence-threshold sweep, TensorRT FP16 vs Embedl INT8

TensorRT 10 compatibility

Revision of November 2026. The earlier ONNX (revision 5492ae0e) was exported at ONNX opset 20, where the GridSample operator spells its bilinear mode "linear". TensorRT's ONNX parser before 11.2 knows only the older spelling "bilinear" and builds any other mode as nearest-neighbour without a warning (onnx-tensorrt 10.16, 11.2). RF-DETR's decoder samples its image features with GridSample, so with TensorRT 10.16 (JetPack 7.2) that file lost about 4.5 mAP points (48.8 on the Jetson AGX Orin); TensorRT 10.3 (JetPack 6) has the same parser code.

This revision is the same model saved at opset 19, where GridSample is spelled "bilinear" and Gelu is written as x · 0.5 · (1 + erf(x / √2)). The opset change alone leaves the arithmetic as it was: built without its Q/DQ nodes in FP16, the graph scores 53.29 mAP on the Jetson AGX Orin, the same as upstream's export. Two further changes bring the INT8 build within 1 mAP point of FP16:

the projector and the decoder (including the detection heads) run in FP16 rather than INT8;
the backbone's activation scales were re-calibrated (minimum squared error on 128 COCO train2017 images).

The .pt2 file carries the same model for PyTorch: the same weights and re-calibrated backbone scales, with the projector and decoder unquantized.

Creating Your Own Optimized Models

This artifact was produced with embedl-deploy, Embedl's open-source PyTorch-to-TensorRT deployment library. You can apply the same workflow to your own models - see the documentation for installation and usage.

License

Component	License
Optimized model artifacts (this repo)	Embedl Models Community Licence v1.0 (no redistribution as a hosted service)
Upstream architecture and weights	RF-DETR License

Contact

We offer engineering support for on-prem/edge deployments and partner co-marketing opportunities. Reach out at contact@embedl.com, or open an issue on GitHub.

Community & support

Need help with this model? Chat with the Embedl team and other engineers on Discord.

Quantization gotchas, hardware questions, fine-tuning tips — bring them all.

Join our Discord →

Downloads last month: 35

Model tree for embedl/rf-detr-base-quantized-tensorrt

Base model

Roboflow/rf-detr-base

Quantized

(1)

this model