Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Have you ever seen, met, or worked with a smart/intellectual search library/framework/algorithm during your career? I need a very nice mechanism to search among text, I don't need a "Personal Google", just thought that programmers' community have met with such problem and know/can provide a good solution.

Is it possible to search by "meaning" and could you advice a way/technique/thoughts according this?

Thank you.

share|improve this question
1  
Information Retrieval is a vast and active topic see here en.wikipedia.org/wiki/Natural_language_processing (Natural Language processing ) –  Darknight Jul 6 '11 at 16:51
add comment

2 Answers

Meaning is a very slippery concept - there is a lot of ongoing debate among philosophers as to what it actually is. One relatively uncontroversial point that is raised often is that to understand anything requires consciousness, so to have a search engine that could actually "search by meaning" you would be looking at something based on some serious Artificial Intelligence. I don't know how far we have got in that direction as yet, but I don't think we have any artificial consciousnesses so for the time being meaning-based search is probably out.

The classic example of why there are problems with this is the difference between two similar phrases: Time flies like an arrow. and Fruit flies like a banana. Both are very similar from the perspective of language, yet someone who understands both will have used a lot of contextual knowledge to distinguish between them.

The avenues that are moving in the right direction are text analytics and data mining techniques, but they are still very much work in progress. You might want to take a look at Apache Mahout as a starting point for learning more about this.

share|improve this answer
add comment

I've heard quite a bit about Apache Lucene but never used it. It is a Java library, but I believe there are bindings for other libraries. There is also Apache Solr which is a HTTP search server built on top of Lucene. These might be a bit too heavyweight for your needs though.

share|improve this answer
    
Thanks, the kind of library/framework/etc doesn't matter. It may not web-based, may be on any language. –  Dehumanizer Jul 6 '11 at 16:54
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.