Wyniki wyszukiwania - BazTech

Ograniczanie wyników

1 Bulletin of the Polish Academy of Sciences. Technical Sciences

1 2018

Znaleziono wyników: 1

Liczba wyników na stronie

Wyniki wyszukiwania

Wyszukiwano:
w słowach kluczowych: adaptive computation time

Sortuj według:

Ogranicz wyniki do:

Probabilistic adaptive computation time

Figurnov M., Sobolev A., Vetrov D.

Bulletin of the Polish Academy of Sciences. Technical Sciences

2018

Vol. 66, nr 6

811--820

We present a probabilistic model with discrete latent variables that control the computation time in deep learning models such as ResNets and LSTMs. A prior on the latent variables expresses the preference for faster computation. The amount of computation for an input is determined via amortized maximum a posteriori (MAP) inference. MAP inference is performed using a novel stochastic variational optimization method. The recently proposed adaptive computation time mechanism can be seen as an ad-hoc relaxation of this model. We demonstrate training using the general-purpose concrete relaxation of discrete variables. Evaluation on ResNet shows that our method matches the speed-accuracy trade-off of adaptive computation time, while allowing for evaluation with a simple deterministic procedure that has a lower memory footprint.