2022-10-2 arXiv roundup: GPT-3 for $500k + Do we even need pretrained models?
dblalock.substack.com
Mosaic LLMs (Part 2): GPT-3 quality for <$500k We showed that, based on the Chinchilla scaling laws, you can train a GPT-3-quality model for less than half a million dollars. This is around 10x cheaper than the original price, depending on whose estimates you believe.
2022-10-2 arXiv roundup: GPT-3 for $500k + Do we even need pretrained models?
2022-10-2 arXiv roundup: GPT-3 for $500k + Do…
2022-10-2 arXiv roundup: GPT-3 for $500k + Do we even need pretrained models?
Mosaic LLMs (Part 2): GPT-3 quality for <$500k We showed that, based on the Chinchilla scaling laws, you can train a GPT-3-quality model for less than half a million dollars. This is around 10x cheaper than the original price, depending on whose estimates you believe.