A Static Analyzer for Detecting Tensor Shape Errors in Deep Neural Network Training Code Exactly what it sounds like. Might make for a nice IDE plugin or linting tool. AdaViT: Adaptive Tokens for Efficient Vision Transformer Intelligent token dropping that apparently "improves the throughput of DeiT-Tiny by 62% and DeiT-Small by 38% with only 0.3% accuracy drop", which might actually be a decent win.
2021-12-18: Data-free Knowledge Distillation, MagNets
2021-12-18: Data-free Knowledge Distillation…
2021-12-18: Data-free Knowledge Distillation, MagNets
A Static Analyzer for Detecting Tensor Shape Errors in Deep Neural Network Training Code Exactly what it sounds like. Might make for a nice IDE plugin or linting tool. AdaViT: Adaptive Tokens for Efficient Vision Transformer Intelligent token dropping that apparently "improves the throughput of DeiT-Tiny by 62% and DeiT-Small by 38% with only 0.3% accuracy drop", which might actually be a decent win.