Bunch of interesting stuff this week. Before we jump in, one quick clarification from last week: I mentioned how it was an interesting marketing lesson from DBRX development how we spent a bunch of time adding to MegaBlocks but then people ended up associating it with Mistral because they released an MoE first. A couple people said this part was “super spicy” (maybe because of the phrasing of the tweet I screenshotted?) so just to be explicit: I see this purely as a case study in marketing + open source, and no one at Databricks has anything against Mistral at all. We have a ton of respect for them and their models, and I hope they get to succeed as a startup too.
I like the depth, but I also like the breadth of previous posts. You could do a few in-depth summaries and then just list some other good papers, maybe with a one-sentence reaction?
I like the depth, but I also like the breadth of previous posts. You could do a few in-depth summaries and then just list some other good papers, maybe with a one-sentence reaction?