34

Literate programming has good ideals. Why do you think that this isn't mainstream? It is because it has failed to deliver?

Improve this question
6
  • 2
    Because the tools that were developed for it are still pretty weak. Microsoft probably has a chance of leading in this regard.
    – Job
    Mar 7, 2011 at 19:12
  • 3
    When approaching a new problem, I often use my own 'Literate Programming' shorthand using pencil and paper. It allows me to ignore language semantics and mix in human language to describe those things that will be called functions, etc.
    – oosterwal
    Mar 7, 2011 at 23:06
  • 1
    Even Knuth is no longer conviced of this concept: "And we’re abandoning the old notion of “literate programming” that I used when developing TEX82, because documentation has proved to be too much of a pain." tug.org/TUGboat/tb31-2/tb98knut.pdf.
    – h0b0
    Nov 30, 2011 at 8:18
  • 7
    For those unfamiliar with TeX and its philosophy it should be mentioned that the Knuth quote is most likely meant ironically.
    – user1249
    Nov 30, 2011 at 10:39
  • 5
    @h0b0 & user1249: The whole article by Knuth is irony, as you can find out just by skim-reading it. He also mocks Steve Jobs, the web, Agile, refactoring, OOP, AOP, and many other things. It's a joke!
    – Andres F.
    Oct 11, 2012 at 14:42

12 Answers 12

Reset to default
35

I first saw it in a book of Knuth's writings, and thought it looked neat. Then I tried to use the literary programming display to comprehend what was going on in the program, and found it harder than it looked. It may have been that I was too used to going through program listings, but it seemed confusing.

Then I looked at the source code, and that turned me off then and there. I'd have to learn to write programs in an entirely new way, with less correspondence between the program text and what the compiler saw, and saw no corresponding benefit.

In addition, people can write long and convincing arguments that the code is doing X when it's actually doing Y, and I've run into my share of misleading comments. I developed a fondness for reading the code to see what it's doing fairly early. Literate programming is the antithesis of that.

Improve this answer
6
  • 6
    Literate programming, as well as comments in general, is not about what your code is doing. You can read that from the code itself. It is all about why and how, and this essential information is almost always missing without a proper literate programming. Needless to mention that the "why?" part quite often would involve elaborate and complicated mathematics, sometimes plots and tables, sometimes diagrams. Literate programming tools are necessary to maintain such comments in a readable way.
    – SK-logic
    Feb 28, 2012 at 7:59
  • 1
    @SK-logic fair, but the point that David Thornley is making is that even the WHY can turn out to be a misleading lie (one that is actually even harder to comprehend).
    – MrFox
    Oct 11, 2012 at 16:57
  • 1
    +1 Knuth was writing back in the (thematic) wild west days of programming when working in an "advanced" language meant writing "C" almost on top of the metal instead using machine code. Memory was so tight variables and other names were usually just single letters, often reused from scope to scope. The vast majority of programs where turn-key one shots written and maintained by one person each with his own eccentric style. Having to take over a code base was more decrypting than reading. There was no source control etc to help out.
    – TechZen
    Jun 30, 2016 at 2:13
  • 1
    Knuth was looking down the road, 30 years ago today. He knew programs would get bigger, more complicated, be written by teams with shifting members, would run for years or decades and require input, assessment and eventually acceptance from non-programmers. Literate programming was an idea for addressing all that. He was roughing out what we today call business logic and BDD. The core idea being that programmers would know what to do and non-programmers could follow along. As noted, the idea failed because no mechanism exist to enforce the linkage between the "literate" text and the code.
    – TechZen
    Jun 30, 2016 at 2:18
  • BTW: This is why I like "self-documenting" languages like Objective-C. At first the code looks way to cluttered with absurdly long method names, but even a programmer who doesn't know the language or the API can quickly puzzle out what the code is doing. Best of all, change the code and the "comments" change in sync automatically. Of course, that's why Objective-C was written with autocomplete built in. Without it, writing Objective-C is pretty hellish.
    – TechZen
    Jun 30, 2016 at 2:23
14

I was fascinated by the concept of Literate Programming in the late 90'es while studying, and I am still intrigued with Knuths approach to programming, and typesetting. Nothing but the best will do.

The Literate Programming system that Knuth designed did much, much more than immediately meets the eye, namely it overcome many shortcomings in the underlying programming language that the code generation tool generated from Knuths source document, namely standard Pascal.

For those fortunate enough not to have tried Standard Pascal, here are some of the highlights.

  • To make it easier to have a single-pass compiler, the language specification said that all declarations had to come in a certain order. From the wikipedia page: "Each procedure or function can have its own declarations of goto labels, constants, types, variables, and other procedures and functions, which must all be in that order." This meant you could not group your things logically together in the source file.
  • String handling was more tedious than in plain C.
  • Identifiers could not have arbitrary length.
  • Many more things I cannot remember anymore.

All these things basically meant that Knuth needed a better programming language (so he invented one) and it used Pascal as its assembly language.

Most modern languages can do these things without much effort, therefore removing a LARGE part of the work that Literate Programming were to solve.

Also modern languages are more expressive allowing more thought to be put in the code itself.

So, what is left? The ability to generate a typeset form of documentation from the source code, and THAT exists today.

Just think JavaDoc - the Java runtime API is perhaps the largest piece of Literate Programming available today (except that the code isn't actually presented, but it COULD have been if Java was open sourced from the start). See for instance the presentation of the collections framework on http://download.oracle.com/javase/6/docs/api/java/util/Collection.html

I believe similar systems exist for .NET and other mainstream programs.

Improve this answer
4
  • To make it possible to have a single-pass compiler, all declarations had to come in a certain order. A declaration order like that certainly simplifies compiler design, but it doesn't enable/prevent single-pass compilation. Delphi, for example, doesn't have that order restriction, but it's still a strictly single-pass Pascal compiler.
    – Mason Wheeler
    Nov 29, 2011 at 23:38
  • Agreed. Turbo Pascal didn't have this restriction either. Note, however, that this restriction was in the definition of Pascal from the beginning.
    – user1249
    Nov 29, 2011 at 23:58
  • 2
    Nope, Knuth switched to CWEB long ago, it's not about fixing Pascal deficiencies. Nope, JavaDoc has nothing to do with Knuth's "literate programming" -- he's talking about fundamentally changing how he creates code, and claiming it allows him to tackle complexity that he asserts would otherwise not be possible for him or anyone else to take on.
    – Ron Burk
    Jun 26, 2017 at 6:34
  • @RonBurk CWEB just compiles to a better "assembly language". This does not invalidate the original design decisions.
    – Thorbjørn Ravn Andersen
    Feb 8, 2018 at 10:14
13

I would blame the network effect. For other people to edit your code and documentation, they must be able to understand it.

This pushes people away from something like cweb/noweb, because using them would require you to learn TeX and the program-specific syntax on top of the programming language you are using for the project. This can be seen as a huge waste of time, especially if they don't need any of the math typesetting that is such a big draw for TeX in the first place. (And for many applications programmers, they really won't need it.) Instead they prefer something like Visual Studio's XML comments, because that's already popular and well-established.

The places I have seen literate programming take off are in scientific/statistical computing, where most of the programmers have significant training (aka PhDs) in math, CS, or statistics, and thus are already famliiar with LaTeX. The documentation they write is more likely to include a lot of complicated formulae that are best written in TeX, and they are more likely to be programming in R. The proportion of R programmers who know about SWeave is definitely much higher than, say, the proportion of C programmers who know about cweb.

Improve this answer
4
  • 2
    This answer seems to assume that all literate programming tools are using LaTeX. Is this true? There doesn't seem to be anything about the concept which requires it.
    – AShelly
    Sep 14, 2010 at 6:09
  • @AShelly: It's not required - I know that noweb, at least, lets you use HTML. But in practice, the people who write HTML documentation will use javadoc and the like instead of literate programming tools.
    – Larry Wang
    Sep 14, 2010 at 7:51
  • 1
    @AShelly, for literate programming to work, you need to be able to generate the document to be printed. This is much, much easier when the format is text-based, and to my knowledge the most powerful text-based document formatter is TeX, and the easiest way to work with TeX is using LaTeX.
    – user1249
    Nov 30, 2011 at 14:22
  • 1
    @AShelly you may want to take a look at org-mode's support for literate programming. It's quite handy, and I find it much easier to comprehend (not to mention manage) than WEB or NOWEB alone. An important aspect of code is readability, and this is readable. (cf github.com/vermiculus/stack-mode)
    – Sean Allred
    Mar 2, 2013 at 22:25
5

One thing I discovered when I had my fling with literate programming in the 90s was that it attracted very passionate people who wanted to do Exactly The Right Thing - and that involved writing their own literate programming system because no existing one was good enough for them. noweb was a good attempt to cut that off by providing a good-enough least common denominator for everyone, although even then, I spent most of my LP time developing a pretty-printer for it...

Another issue is that it is really anti-agile. In some ways, being slowed down is good because it forces you to think more up-front and get things right the first time. On the other hand, meticulously documenting as you go means that there's a large barrier to refactoring your code. And if you wait until your code is hardened before you LP-ify it, you end up with a multi-day documentation task, which can really stop you in your tracks.

Improve this answer
1
  • After experimenting I have found that the sweet spot of LP for the rest of us, might be in documenting design decisions and architecture details right next to the actual code. I agree with LP being harder to refactor. It is my understanding that Knuth did the initial design on paper and only when satisfied began actual implementation. This is most likely the same situation that I find works for me.
    – Thorbjørn Ravn Andersen
    May 11, 2018 at 12:44
3

In my humble opinions, many companies have a culture who is the opposite to the objectives of Literate Programming: they want faster results (they only cry about quality when the app is in production). In my own experience, my bosses had refuse to understand that faster results doesn't mean "a program runnable the day after I asking for." For them, if a developer isn't busy typing over his keyboard, he isn't working, is "wasting his time in design-non-sense". Yes, I know, my boss is an arsehole.

Improve this answer
2
  • Then with Literate Programming they might think you're busy to write Sci-Fi Book instead of yet-another-software! :D
    – Mahdi
    Oct 11, 2012 at 11:46
  • Companies like that do not understand that good software live very long and the better documentation the more worth the source is.
    – Thorbjørn Ravn Andersen
    May 11, 2018 at 12:45
2

Coders write code not English.

Coders don't like writing documentation because it doesn't help the code run.

Coders aren't good at writing documentation because its a poor medium to express their ideas.

Literate programming seems to be the idea of taking documentation to the next level where the code is more of an after-thought. Maybe it would work, but to most coder it looks like obnoxious documentation.

Improve this answer
5
  • 31
    Coders who adhere to the points you describe aren't coders I want working with me.
    – Paul Nathan
    Sep 22, 2010 at 17:28
  • 1
    @Paul, granted. But thats whats really out there. But it seems to me that more documentation is not necessarily better.
    – Winston Ewert
    Sep 23, 2010 at 2:54
  • 1
    enough is possibly best
    – mlvljr
    Oct 27, 2010 at 20:20
  • 6
    experienced programmers know they NEED to write documentation because that is where the "WHY did I do it like that" goes.
    – user1249
    Mar 8, 2011 at 20:02
  • 1
    @Thorbjørn Ravn Andersen, yes that's true. But literate programming, (as I understand it) suggests that you write code with your documentation instead of documentation with your code. Is that much documentation really helpful?
    – Winston Ewert
    Mar 8, 2011 at 20:21
2

Principally because people are VERY STUPID. Obvious testimony to which is an endless stream of guesses and misunderstandings expressed by young people about the nature of this simple technique.

People take LP to be: (a) a method of documentation (b) a method of writing some polished essays that necessitates some special skills or talents (c) simply have no clue - as the creator of Leo programming editor, by his own admission etc. etc. etc.

LP however is simply: (1) writing programs in a mixture of code and phrases in a (=any) human language, where the latter stand for other chunks of code and/or included phrases. This is precisely what authors of innumerous programming textbooks do.. and (2) it is a simple preprocessor that expands those phrases in human (which became as if names of included subroutines) to unravel the result IN THE ORDER REQUIRED BY THE COMPILER (or interpreter). Otherwise one can expand the written text with another small utility to include formatting symbols to turn the "literate source" into a nice well-formatted readable text.

Young people never try this extremely simple idea - and either fantasize or imagine fake reasons why they'll never try or do it.

Basically the main idea of programming "in pseudocode" written in a human language and then expanding it with a simple preprocessor utility HELPS MANAGE ATTENTION (limited, a main difficulty for any longish program), pretty much like code folding or division of your program flow into functions/subroutines, needed for you not to lose yourself in the details, but wholly unnecessary for the machine execution.

Improve this answer
1
  • 3
    You're missing one important bit: (3) a way to re-order a code in any language into the most readable and natural sequence, which is not necessarily the same order a compiler have to deal with. It includes hiding the implementation details in footnotes or wherever else far from the code outline.
    – SK-logic
    Mar 8, 2011 at 15:18
1

There are 2 aspects of literate programming that I do wish were incorporated into mainstream programming -- embedded imagery (e.g., design diagrams) and pointers to previous and alternate attempts (e.g., "The reason it's like this is because I tried this other way and it didn't work because ..."). Both of those aspects can be handled with doc-comments and URIs.

Improve this answer
1

Because the logic of programs does not work the same as we speak. A program has a well specified flow, and conditions, and loops.

After having coding at lot, I THINK in these terms. My brain transforms problems into the target domain of executable code. And it is much more efficient for me to write this down in a usually programming language, than having to do the extra transform step to make my programs literate.

In fact, I believe my programs already are literate... speaking identifiers, good function names, comments where I did some hackery which i wouldn't comprehend immediately myself after a few months.

To conclude: My Java code is more literate by itself as every "literate" programming wants to be.

Improve this answer
1
  • 2
    A Java code can't be literate. Your "speaking identifiers" won't ever explain why you choose this particular algorithm over another, what are the limits, what was your performance profile expectation, etc. My literate programs are made mostly of formulas, diagrams and graphs, and not that much english text. But all of that can't be expressed in a code and look ugly inside simple comments.
    – SK-logic
    Mar 8, 2011 at 15:21
1

I came to literate programming the other way around - I dreamed having the code organized as it fits my mind, not as the compiler requires it. I found Leo almost ideal for this purpose. It also supports keeping track of files changed outside. These files do not have to contain any special markup, so I can use Leo for myself without any need for others in the team to know. This feature - "@shadow trees" - is very promising, although still bit buggy, needs more eyeballs. And it also fixes the "oh no, everything in one big file" problem both by organizing everything into tree outline and by support for external files.

For me, contrary to the name, the "literate programming" is not about documentation at all. I don't have more documentation than before. It is about having structure that helps me to not become lost. I swear by it especially when managing behemoth JSP files (and that despite Leo was originally intended primarily for Python and it does not have support for JSP language - I have to split the file to Leo tree manually !).

Improve this answer
0

I see it as a valuable teaching tool, where a dissertation on code can be written, and then snippets of working code interleaved in it to instruct readers on the hows, whats, and whys of the code.

Outside a purely educational environment, I think only Knuth really understands how best to use it.

Improve this answer
-4

It's the worst of all worlds - you have to write a highly correct, highly specific computer program in a very non-specific language = english. So you have to carefully write it using exactly the correct phrases - so you might as well just write code.

Improve this answer
4
  • 3
    You should not repeat your code in english. Comments must explain the reason why the code is there, not what it is doing. I often stuff graphs, diagrams and plots into my literate comments, and it really helps to understand the code.
    – SK-logic
    Mar 7, 2011 at 17:05
  • If the comments don't say what the code is doing then how is it literate programming - it's just regular programming with comments. I thought the whole point of literate programming was to describe the program in the docs and have the system generate code from the documentation?
    – Martin Beckett
    Mar 7, 2011 at 18:25
  • 3
    try to read "TeX, the program". Code is never repeated in comments there. Comments explains why the code is written that way, and explains the architecture.
    – SK-logic
    Mar 7, 2011 at 19:06
  • 3
    @MartinBeckett What you describe is not LP.
    – Andres F.
    Oct 11, 2012 at 15:09

Not the answer you're looking for? Browse other questions tagged or ask your own question.