Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Is there such thing?

I designed a while back a rudimentary form based app for my users.

We receive from our suppliers hardware manufacturing data in XML files: file name is made of eleven fields separated by tildes, with each field having its own meaning.

R&D guys wanted to be able to search each field of the file names so I used regex() with decent results.

Problem is that we have now in the upwards of 2.5 million files. And my app can't hack it anymore.

I looked at Apache Lucene & Solr. Though it seemed like the best solution to my problem, the fields in the filenames are not peers to the file content. Big no-no with Solr.

What is the best way to implement a PHP app with indexing and search capability with such large number of files?

Do I have to buy Zend and use Zend_Search? Is it the only way?

Thanks for your input.

share|improve this question
1  
I'm assuming something like MySQL's FULLTEXT search is not an option? –  barrycarter Feb 27 '11 at 15:28
    
Thanks for your reply. I am a newbie and did not know a darn thing about MySQL's FULLTEXT capability. This is actually what I am working on at the moment. –  Chris Feb 28 '11 at 18:32
    
Does this not belong on stackoverflow? –  Chris Mar 21 '11 at 21:59

2 Answers 2

up vote 2 down vote accepted

When you receive xml files from suppliers, put the information in a database. Thats what they are designed for. You'll have essentially instantaneous searches. If the fields are consistent, a regular old SQL database will do the trick. If each is different, perhaps look at a NoSQL solution.

share|improve this answer

Well, there's always Xapian, or more specifically Omega. You can integrate it through cgi, and then be able to have lightning-fast indexing and searching of different web server documents.

In this case, you might want to have a look at their guide for how to add support for XML files.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.