This is the class project in New York University Machine Learning For Language Understanding class during 2022 Spring. All rights reserved.
We are grateful to Professor Samuel R. Bowman's inspiration and suggestions.
The project implements and reproduces the LRA Benchmark (its paper can be accessed at: https://arxiv.org/abs/2011.04006) based on the code coming from here: https://github.com/mlpen/Nystromformer.
In addition, the project implements the newly released efficient transformer, Perceiver (its paper can be access at: https://arxiv.org/abs/2103.03206) using HuggingFace interface for Perceiver. Our goal is to compare different efficient transformers for their abilities on handling long sequence data, and our paper can be accessed at: click here.
People who contribute to this project: Gavin Yang ([email protected]), Stephen Zhang ([email protected]), David Guo ([email protected])