2023-1-8 arXiv roundup: Language models creating their own data, Hinton vs backprop, Practical pruning + quantization for LLMs
dblalock.substack.com
This newsletter made possible by MosaicML. Feels like we have some Massive Language Models Can Be Accurately Pruned in One-Shot They prune 50% of the weights in huge OPT models with little or no loss of accuracy and no finetuning. The basic approach here is to go layer-by-layer and iteratively:
2023-1-8 arXiv roundup: Language models creating their own data, Hinton vs backprop, Practical pruning + quantization for LLMs
2023-1-8 arXiv roundup: Language models…
2023-1-8 arXiv roundup: Language models creating their own data, Hinton vs backprop, Practical pruning + quantization for LLMs
This newsletter made possible by MosaicML. Feels like we have some Massive Language Models Can Be Accurately Pruned in One-Shot They prune 50% of the weights in huge OPT models with little or no loss of accuracy and no finetuning. The basic approach here is to go layer-by-layer and iteratively: