A new method for neural network compression learns functional subspaces to select which low-rank components to remove from each transformer weight matrix, according to the paper "Learning Functional Subspaces for Neural Network Compression." The approach targets the memory and compute demands of modern transformers by keeping the compressed matrices dense, and therefore efficient on standard hardware. Existing low-rank weight factorization methods, by contrast, choose the subspace to remove from each weight matrix with limited regard for the function it serves.
Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keeping the matrices dense, and thus efficient on standard hardware. Existing methods, however, choose the subspace to remove from each weight matrix with lo
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.