Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

What buffer size should I choose for read/write files via POSIX fread/fwrite functions?

share|improve this question

closed as off topic by World Engineer, Walter, GlenH7, MainMa, pdr Oct 25 '12 at 14:37

Questions on Programmers Stack Exchange are expected to relate to software development within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

    
This question is too localized. It only pertains to whatever particular program you are coding at the moment. –  World Engineer Oct 25 '12 at 7:04
    
I mean general case. Many programs needs to read/write whole files with very unknown size (like grep, cp) –  Corvus Oct 25 '12 at 7:09
    
Do you mean POSIX read/write functions or ANSI/ISO/IEC fread/fwrite functions? –  Jan Hudec Oct 25 '12 at 8:19
add comment

1 Answer

The ANSI/ISO fread/fwrite functions are buffered. The buffer is usually 8 KiB and that gives the granularity independent of what you use in your code. It might make sense to increase the buffer a bit, perhaps to the value below. For bulk transfer they will always be a tiny bit slower though due to the extra copies.

For the POSIX read/write functions it depends on operating system and device and a lot of other things, but practical experience is that you don't get any performance improvement by increasing the buffer beyond tens of KiB, so 32 or 64 KiB is about right.

On some systems the dependency is larger than on others. On Linux the difference is usually minimal above 8 KiB (so the default buffer is fine), e.g. on Windows CE (using native API; they don't have POSIX) even bigger than 64 KiB still help. It might also depend on the device.

share|improve this answer
add comment

Not the answer you're looking for? Browse other questions tagged or ask your own question.