2022-4-17: Neighborhood attention, 830k TPU-hours, Revenge of the ViT
dblalock.substack.com
deep-significance - Easy and Meaningful Statistical Significance Testing in the Age of Neural Networks Proposes significance testing based on overlap of CDFs of outcomes. Doesn’t address the main problem, which is that no one reports multiple runs in the first place, but it’s an interesting statistic to look at. Might be an informative way to monitor distributions of weights, gradients, etc, changing over time or across runs.
2022-4-17: Neighborhood attention, 830k TPU-hours, Revenge of the ViT
2022-4-17: Neighborhood attention, 830k…
2022-4-17: Neighborhood attention, 830k TPU-hours, Revenge of the ViT
deep-significance - Easy and Meaningful Statistical Significance Testing in the Age of Neural Networks Proposes significance testing based on overlap of CDFs of outcomes. Doesn’t address the main problem, which is that no one reports multiple runs in the first place, but it’s an interesting statistic to look at. Might be an informative way to monitor distributions of weights, gradients, etc, changing over time or across runs.