I am currently building a small in-house search engine based on Apache Lucene. Its purpose is simple - based on some keywords, it will suggest some articles written internally within our company. I am ...
Let's say I have a big corpus (for example in english or an arbitrary language), and I want to perform some semantic search on it. For example I have the query: "Be careful: [art] armada of [sg] is ...
We are looking for a way to tokenize some text in the same or similar way as a search engine would do it. The reason we are doing this is so that we can run some statistical analysis on the tokens. ...