facebookresearch /mmf
A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)
★ 5,633⑂ 941👁 108⑃ 676● 659▤ 2
+0
日增长
5,633
stars
941
forks
108
watchers
676
prs
659
issues
2
releases
README
---
MMF is a modular framework for vision and language multimodal research from Facebook AI Research. MMF contains reference implementations of state-of-the-art vision and language models and has powered multiple research projects at Facebook AI Research. See full list of project inside or built on MMF here.
MMF is powered by PyTorch, allows distributed training and is un-opinionated, scalable and fast. Use MMF to _bootstrap_ for your next vision and language multimodal research project by following the installation instructions. Take a look at list of MMF features here.
MMF also acts as starter codebase for challenges around vision and
language datasets (The Hateful Memes, TextVQA, TextCaps and VQA challenges). MMF was formerly known as Pythia. The next video shows an overview of how datasets and models work inside MMF. Checkout MMF's video overview.
Installation
Follow installation instructions in the documentation.
Documentation
Learn more about MMF here.
Citation
If you use MMF in your work or use any models published in MMF, please cite:
bibtex
@misc{singh2020mmf,
author = {Singh, Amanpreet and Goswami, Vedanuj and Natarajan, Vivek and Jiang, Yu and Chen, Xinlei and Shah, Meet and
Rohrbach, Marcus and Batra, Dhruv and Parikh, Devi},
title = {MMF: A multimodal framework for vision and language research},
howpublished = {\url{https://github.com/facebookresearch/mmf}},
year = {2020}
}
License
MMF is licensed under BSD license available in LICENSE fileTop issues
#1037TextVQA7 comments · 2021-07-31#1010Is there a clear cut demo for using the pre-trained M4C Captioner model available on sample images?7 comments · 2021-07-10#946errors while running pytests .\test7 comments · 2021-05-16#969model_name of pre-trained Pythia model for VQA 6 comments · 2021-06-14#1221mmf hateful memes challenge dataset5 comments · 2022-03-27#999Update VizWiz annotations/features5 comments · 2021-07-01#980errors while running krisp code5 comments · 2021-06-22#1219FLAVA code4 comments · 2022-03-25