A polynomial autoencoder beats PCA on transformer embeddings

Keyword: Online Learning

Publisher: Ivanpleshkov.dev

Published: May 5, 2026

The most direct way to compress an embedding (other than quantization) is to fit PCA on the corpus and keep the top-d eigenvectors. It works, but PCA is a linear projection, and neural-network embe… [+21763 chars]

Read Full Story ↗