倒排索引/搜索引擎>> Pytorch_mixture-of-experts>> 返回
项目作者： YeonwooSung

项目描述 ：
PyTorch implementation of moe, which stands for mixture of experts
高级语言： Python
项目地址: git://github.com/YeonwooSung/Pytorch_mixture-of-experts.git
创建时间： 2021-02-11T03:39:20Z
项目社区：https://github.com/YeonwooSung/Pytorch_mixture-of-experts
开源协议：
下载


The Sparsely Gated Mixture of Experts Layer for PyTorch

source: https://techburst.io/outrageously-large-neural-network-gated-mixture-of-experts-billions-of-parameter-same-d3e901f2fe05

This repository contains the PyTorch re-implementation of the MoE layer described in the paper Outrageously Large Neural Networks for PyTorch.

Requirements

This example was tested using torch v1.0.0 and Python v3.6.1 on CPU.

To install the requirements run:

pip install -r requirements.txt

Example

The file test.py contains an example illustrating how to train and evaluate the MoE layer with dummy inputs and targets. To run the example:

python test.py

Acknowledgements

The code is based on the TensorFlow implementation that can be found here.