2023-6-4 arXiv: 1063 papers, small models making their own data, Way simpler RLHF, Adam accumulation
dblalock.substack.com
This newsletter made possible by MosaicML. We beat last week’s record paper count, so once again we’re going to have more breadth and less depth than usual. Also a heads up that I’m more likely to make mistakes. Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing
2023-6-4 arXiv: 1063 papers, small models making their own data, Way simpler RLHF, Adam accumulation
2023-6-4 arXiv: 1063 papers, small models…
2023-6-4 arXiv: 1063 papers, small models making their own data, Way simpler RLHF, Adam accumulation
This newsletter made possible by MosaicML. We beat last week’s record paper count, so once again we’re going to have more breadth and less depth than usual. Also a heads up that I’m more likely to make mistakes. Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing