CMU’s Unlimiformer Augments Transformers to Enable Unbounded Input Lengths | Synced

In the new paper Unlimiformer: Long-Range Transformers With Unlimited Length Input, a Carnegie Mellon University research team presents a general approach for improving model performance by augmenting pretrained encoder-decoder transformers with an external datastore to permit inputs of unbounded length.