Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

(Almost-)POSIX-compliant operating systems and Windows are known to distinguish between 'binary mode' and 'text mode' file I/O. While the former mode doesn't transform any data between the actual file or stream and the application, the latter 'translates' the contents to some standard format in a platform-specific manner: line endings are transparently translated to '\n' in C, and some platforms (CP/M, DOS and Windows) cut off a file when a byte with value 0x1A is found.

These transformations seem a little useless to me.

People share files between computers with different operating systems. Text mode would cause some data to be handled differently across some platforms, so when this matters, one would probably use binary mode instead.

As an example: while Windows uses the sequence CR LF to end a line in text mode, UNIX text mode will not treat CR as part of the line ending sequence. Applications would have to filter that noise themselves. Older Mac versions only use CR in text mode as line endings, so neither UNIX nor Windows would understand its files. If this matters, a portable application would probably implement the parsing by itself instead of using text mode.

Implementing newline interpretation in the parser might also remove some overhead of using text mode, as buffers would need to be rewritten (and possibly resized) before returning to the application, while this may be less efficient than when it would happen in the application instead.

So, my question is: is there any good reason to still rely on the host OS to translate line endings and file truncation?

share|improve this question
1  
-1 - With the zero context you provided, it is impossible to even approach answering this question. –  Oded Nov 19 '11 at 17:05
1  
How is your question related to software development? Can you provide some context? Thanks. –  Anna Lear Nov 19 '11 at 17:11
    
I get easily confused between SO and programmers. My apologies. If this is out of scope for p.SE, can it be moved to SO instead? My reason for posting it here, is that it's more about general guidelines than specific programming questions. I'll update the post for context as well. –  Rhymoid Nov 19 '11 at 17:18

3 Answers 3

up vote 4 down vote accepted

Without the translation, every Unix text-processing program would recognize just '\n' as an end-of-line marker, and every Windows text-processing program would recognize '\r' followed by '\n'. (And pre-OSX Mac programs would recognize '\r'.) And any program that writes text would have to explicitly write the local end-of-line marker, which means it would have to be aware of what OS it's running on.

And that just covers the relatively simple cases where an end-of-line is indicated by a sequence of characters. Other schemes are common (though less so these days); see IBM mainframes and VMS, for example.

With the translation, programs can just treat text as text, and we don't need three or more different "hello, world" programs.

This does sometimes cause problems when you need to process a foreign file (a Windows text file that's been copied to a Unix system, or vice versa). Cygwin, a Unix-like environment running under Windows, is a rich source of such issues. But usually the best solution is to translate the file before processing it. And most of the time, programs deal with text files that were created on the same OS anyway.

It's better to write one program that can translate between formats, than to require every program to deal with all different formats. And inevitably someone would write a tool that understand Unix and Windows formats, but breaks when confronted with an old Mac text file, and someone else would get the interpretation just a little bit wrong because the wheel they reinvented wasn't perfectly round.

share|improve this answer

While Unix (MacOS-X is Unix for the purpose of this) and Windows consider files as a stream of bytes and differs only in some little details about what is a text file and what isn't, when other OS come into play you have more variety. Three examples:

  • in some OS, text file are a set of fixed width lines and lines are padded with spaces up to the fixed width (and you know understand why C and C++ don't guarantee that spaces at the end of a line are conserved)

  • others OS have a notion of permanent line numbers and the C runtime has to skip in,

  • the system call (to use a Unix terminology) used to open a text file may be different than the one used to open a binary one.

You could probably imagine a way not to specify it, you'd have to drop the interoperability for your C or C++ programs with those abiding the OS convention.

share|improve this answer
2  
That sounds arcane. Mainframe-like arcane. Could you name the operating systems in the first two cases? –  Rhymoid Nov 19 '11 at 18:50
    
For the first one, I'm just an indirect source, and I don't remember the name if I ever knew it. VMS could be in the case (its file handling is quite complex, IIRC it has even indexed file as an OS feature, but I've never programmed it at that level) and mainframes are obviously candidates. For the second one, TOPS-20 (a PDP-10 OS) was in that case. –  AProgrammer Nov 20 '11 at 9:06

With reading in text mode, you have something like 'characters', binary knows only bytes.

At least in Ruby 1.9 you can define an encoding, when you read a file in text mode.

Examples:

File.open('sample.txt', 'r') # Read in default encoding
File.open('sample.txt', 'r:utf-8') # Read a utf-8 file
File.open('sample.txt', 'r:utf-8:cp1251') # Read a utf-8 file and convert it to cp1252

Doing this, you have a possibility to make differences between bytes and characters. When you read binary, you have only bytes.

share|improve this answer
1  
Interesting (it would be much more useful), but very language-specific. For most operating systems, this conversion step is probably implemented in Ruby itself, while newline conversion can be done by 'deeper' parts of the host OS. –  Rhymoid Nov 19 '11 at 21:12

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.