mirrors/tantivy
2
0
Fork 1
mirror of https://github.com/quickwit-oss/tantivy.git synced 2026-06-19 09:16:45 +00:00
Code Issues Projects Releases Packages Wiki Activity
tantivy/tokenizer-api
History
PSeitz e56addc63e
 enable tokenizer on json fields (#2053) 
* enable tokenizer on json fields

enable tokenizer on json fields for type text

* Avoid making the tokenizer within the TextAnalyzer pub(crate)

* Moving BoxableTokenizer to tantivy.

---------

Co-authored-by: Paul Masurel <paul@quickwit.io>
2023-05-24 10:47:39 +02:00
..
src enable tokenizer on json fields (#2053) 2023-05-24 10:47:39 +02:00
Cargo.toml Add license to tokenizer-api crate (#1778) 2023-01-11 05:26:41 +01:00
README.md move tokenizer API to seperate crate (#1767) 2023-01-09 06:37:38 +01:00

README.md

#Tokenizer-API

An API to interface a tokenizer with tantivy.

The API will be kept stable in order to not break support for existing tokenizers.