tensorflow /tensor2tensor
Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.
★ 17,471⑂ 3,829👁 9⑃ 671● 1,245▤ 75
+0
日增长
17,471
stars
3,829
forks
9
watchers
671
prs
1,245
issues
75
releases
README
Tensor2Tensor Tensor2Tensor, or T2T for short, is a library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research. T2T was developed by researchers and engineers in the Google Brain team and a community of users. It is now deprecated — we keep it running and welcome bug-fixes, but encourage users to use the successor library Trax. Quick Start This iPython notebook explains T2T and runs in your browser using a free VM from Google, no installation needed. Alternatively, here is a one-command version that installs T2T, downloads MNIST, trains a model and evaluates it: pip install tensor2tensor && t2t-trainer \ --generate_data \ --data_dir=~/t2t_data \ --output_dir=~/t2t_train/mnist \ --problem=image_mnist \ --model=shake_shake \ --hparams_set=shake_shake_quick \ --train_steps=1000 \ --eval_steps=100 Contents Suggested Datasets and Models Mathematical Language Understanding Story, Question and Answer Image Classification Image Generation Language Modeling Sentiment Analysis Speech Recognition Summarization Translation Basics Walkthrough Installation Features T2T Overview Datasets Problems and Modalities Models Hyperparameter Sets Trainer Adding your own components Adding a dataset Papers Run on FloydHub Suggested Datasets and Models Below we list a number of tasks that can be solved with T2T when you train the appropriate model on the appropriate problem. We give the problem and model below and we suggest a setting of hyperparameters that we know works well in our setup. We usually run either on Cloud TPUs or on 8-GPU machines; you might need to modify the hyperparameters if you run on a different setup. Mathematical Language Understanding For evaluating mathematical expressions at the character level involving addition, subtraction and multiplication of both positive and negative decimal numbers with variable digits assigned to symbolic variables, use the MLU data-set: --problem=algorithmic_math_two_variables You can try solving the problem with different transformer models and hyperparameters as described in the paper: Standard transformer: --model=transformer --hparams_set=transformer_tiny Universal transformer: --model=universal_transformer --hparams_set=universal_transformer_tiny Adaptive universal transformer: --model=universal_transformer --hparams_set=adaptive_universal_transformer_tiny Story, Question and Answer For answering questions based on a story, use the bAbi data-set: --problem=babi_qa_concat_task1_1k You can choose the bAbi task from the range [1,20] and the subset from 1k or 10k. To combine test data from all tasks into a single test set, use --problem=babi_qa_concat_all_tasks_10k Image Classification For image classification, we have a number of standard data-sets: ImageNet (a large data-set): --problem=image_imagenet, or one of the re-scaled versio
Top issues
#444[Tuning] Results are GPU-number and batch-size dependent81 comments · 2017-11-27#317Unable to reproduce WMT En2De results52 comments · 2017-09-21#1643Stuck after printing 'Successfully opened dynamic library libcublas.so.10.0'45 comments · 2019-07-24#975distributed training fail29 comments · 2018-08-07#529Bug introduced in v1.3.0 causing training divergence27 comments · 2018-01-19#621Inference speed for Transformer Vs GNMT. Transformer is slower23 comments · 2018-02-27#1221[Feature Request] Mixed Precision22 comments · 2018-11-13#415How to measure the number of training epochs22 comments · 2017-11-13
Latest release
v1.15.7 v1.15.7 2020-06-17Multistep Adam Optimizer many thanks to @AgoloCuongHoang for contributing in 1773 ! Residual Shuffle-Exchange Network thanks to @EmilsOzolins in 1805 ! Not pinning the gym version.