Davis Summarizes Papers

Home
AI Implications
Archive
About
New
Top
Discussion
2023-5-28 arXiv roundup: 994 papers, beating RLHF with 1000 good examples, LoRA with half the RAM, Multi-epoch scaling
This newsletter made possible by MosaicML. We got a bunch of interesting stuff this week, thanks in part to getting the most submissions of all time…
 • 
Davis Blalock
2
2023-5-21 arXiv roundup: Parallel transformers, Optimized data mixtures, Don't trust LLM chains-of-thought
This newsletter made possible by MosaicML. Accelerating Transformer Inference for Translation via Parallel Decoding They speed up inference in…
 • 
Davis Blalock
2023-5-14 arXiv roundup: FrugalGPT, Inverse CLIP scaling, Embedding all the modalities
This newsletter made possible by MosaicML. An Inverse Scaling Law for CLIP Training They find that larger CLIP models let you get away with training on…
 • 
Davis Blalock
2023-5-7 arXiv roundup: Easy loss spike fix, LLongboi, H100s, stable diffusion for $50k
This newsletter made possible by MosaicML. Thanks to Haoli Yin and Trevor Gale for the Twitter shoutouts this week! Introducing MPT-7B: A New Standard…
 • 
Davis Blalock
3
April 2023
2023-4-23 arXiv roundup: Adam instability, better hypernetworks, More Branch-Train-Merge
This newsletter made possible by MosaicML. Also: is anyone looking for a podcast guest? I may or may not have a podcast in the works and want to get…
 • 
Davis Blalock
2023-4-16 arXiv roundup: Segment Anything, Everything, and Everything everywhere all at once
Yes, in a two-week period we actually got papers named: Segment Anything SegGPT: Segmenting Everything In Context Segment Everything Everywhere All at…
 • 
Davis Blalock
1
2023-4-2 arXiv roundup: LLMs improving LLM output, BloombergGPT, LLM opinions differ from humans
This newsletter made possible by MosaicML. Thanks to Charlie Blake for the Twitter shoutout this week! Training Language Models with Language Feedback…
 • 
Davis Blalock
1
March 2023
2023-3-26 arXiv roundup: Unit scaling, Origins of power laws, Removing text watermarks
This newsletter made possible by MosaicML. Unit Scaling: Out-of-the-Box Low-Precision Training They make all the weights, activations, and gradients…
 • 
Davis Blalock
2023-3-19 arXiv roundup: GPT-4, Data deduplication, MoE optimizations
This newsletter made possible by MosaicML. GPT-4 Technical Report This is a 98-page document, so we’re just gonna go through some highlights. First…
 • 
Davis Blalock
2
2023-3-12 arxiv roundup: Pretraining BERT for $20, GigaGAN, Multimodal LLMs
This newsletter made possible by MosaicML. Pretraining BERT from Scratch for $20 We trained an optimized BERT model to match the results from the…
 • 
Davis Blalock
2
2022-3-5 arXiv roundup: 5-bit training, What pretraining data to use, Expanding training sets via ML
This newsletter made possible by MosaicML (look at our shiny new website). Full Stack Optimization of Transformer Inference: a Survey Besides having a…
 • 
Davis Blalock
2023-2-26 arXiv roundup: RLHF for diffusion, Multimodal chain of thought, Practical data poisoning
This newsletter made possible by MosaicML. Poisoning Web-Scale Training Datasets is Practical You can poison public datasets by buying domains …
 • 
Davis Blalock
1
© 2023 Davis Blalock
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing