Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am running into problems in NetLogo with large *.csv / *.txt files.

The documents can consist of about 1 million data sets and I need to read them (to eventually create a diagram based on the data).

With the most straightforward source code, my program needs about 2 minutes to process these files.

How should I approach reading such large data files faster in NetLogo? Is NetLogo even suitable for such tasks (as it seems to be designed more for teaching and learning)?

share|improve this question
1  
Could you add the relevant parts of your existing source code to the (edited) question? –  Joris Timmermans Sep 19 '12 at 9:51
    
Consider processing the files outside NetLogo and then create a summary file, then process the summary file in NetLogo. –  Emmad Kareem Oct 17 '12 at 0:18

1 Answer 1

The obvious solution is to extend NetLogo in Java, and take advantage of one of the highly optimised CSV readers or TXT parsers readily available. Take a look at http://commons.apache.org/proper/commons-csv/.

If that doesn't do the trick, you might need to pre-process the data into something that can be read faster. Without knowing the data, it's hard to speculate what.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.