下载:Detectron2 and torchscript support, attention and panoptic notebooks, code improvementsDetectron2-0.2 ↓0.22020-06-29
+0
日增长
15,344
stars
2,678
forks
3
watchers
89
prs
526
issues
1
releases
README
DE⫶TR: End-to-End Object Detection with Transformers
========
PyTorch training code and pretrained models for DETR (DEtection TRansformer).
We replace the full complex hand-crafted object detection pipeline with a Transformer, and match Faster R-CNN with a ResNet-50, obtaining 42 AP on COCO using half the computation power (FLOPs) and the same number of parameters. Inference in 50 lines of PyTorch.
!DETR
What it is. Unlike traditional computer vision techniques, DETR approaches object detection as a direct set prediction problem. It consists of a set-based global loss, which forces unique predictions via bipartite matching, and a Transformer encoder-decoder architecture.
Given a fixed small set of learned object queries, DETR reasons about the relations of the objects and the global image context to directly output the final set of predictions in parallel. Due to this parallel nature, DETR is very fast and efficient.
About the code. We believe that object detection should not be more difficult than classification,
and should not require complex libraries for training and inference.
DETR is very simple to implement and experiment with, and we provide a
standalone Colab Notebook
showing how to do inference with DETR in only a few lines of PyTorch code.
Training code follows this idea - it is not a library,
but simply a main.py importing model and criterion
definitions with standard training loops.
Additionnally, we provide a Detectron2 wrapper in the d2/ folder. See the readme there for more information.
For details see End-to-End Object Detection with Transformers by Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.
See our blog post to learn more about end to end object detection with transformers.
Model Zoo
We provide baseline DETR and DETR-DC5 models, and plan to include more in future.
AP is computed on COCO 2017 val5k, and inference time is over the first 100 val5k COCO images,
with torchscript transformer.
name
backbone
schedule
inf_time
box AP
url
size
0
DETR
R50
500
0.036
42.0
model | logs
159Mb
1
DETR-DC5
R50
500
0.083
43.3
model | logs
159Mb
2
DETR
R101
500
0.050
43.5
model | logs
232Mb
3
DETR-DC5
R101
500
0.097
44.9
model | logs
232Mb
COCO val5k evaluation results can be found in this gist.
The models are also available via torch hub,
to load DETR R50 with pretrained weights simply do:
python
model = torch.hub.load('facebookresearch/detr:main', 'detr_resnet50', pretrained=True)
COCO panoptic val5k models:
name
backbone
box AP
segm AP
PQ
url
size
Top issues
#9Recommendations for training Detr on custom dataset?205 comments · 2020-05-28#101'assert (boxes1[:, 2:] >= boxes1[:, :2]).all()' happened when training26 comments · 2020-06-24#428DETR tutorials to use it on custom data :) 21 comments · 2021-08-25#211onnx export error and can the onnx model support tensorRT?19 comments · 2020-08-24#498model.train() and model.eval() gives different results17 comments · 2022-04-10#156How to run detr with only single class?17 comments · 2020-07-21#125Failing to converge on small datasets (Getting zeros on small custom data)16 comments · 2020-06-30#29Suggestion - change to ResNeST50 backbone (new split attention arch)14 comments · 2020-06-01
Latest release
Detectron2 and torchscript support, attention and panoptic notebooks, code improvements v0.2 2020-06-29This release contains several improvements compared to the initial release of DETR.
Highlights
Detectron2 support
DETR now can be used with Detectron2 in a seamless way. It reuses the original abstractions of DETR, while making a compatibility layer so that it fits Detectron2's API. More information can be found in the README under the d2/ folder
TorchScript support for all models
DETR models can now be natively exported to torchscript. This enables DETR to be run in C++ via libtorch. Converting a model to torchscript is easy:
python
model = torch.hub.load('facebookresearch/detr'