Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I love programming in languages that seem geared towards hardcore programmers. (My favorites are Python and D.) MATLAB is geared towards engineers and R is geared towards statisticians, and it seems like these languages were designed for people who aren't hardcore programmers and don't think like hardcore programmers. I always find them somewhat awkward to use, and to some extent I can't put my finger on why. Here are some issues I have managed to identify:

  • (Both): The extreme emphasis on vectors and matrices to the extent that there are no true primitives.
  • (Both): The difficulty of basic string manipulation.
  • (Both): Lack of or awkwardness in support for basic data structures like hash tables and "real", i.e. type-parametric and nestable, arrays.
  • (Both): They're really, really slow even by interpreted language standards, unless you bend over backwards to vectorize your code.
  • (Both): They seem to not be designed to interact with the outside world. For example, both are fairly bulky programs that take a while to launch and seem to not be designed to make simple text filter programs easy to write. Furthermore, the lack of good string processing makes file I/O in anything but very standard forms near impossible.
  • (Both): Object orientation seems to have a very bolted-on feel. Yes, you can do it, but it doesn't feel much more idiomatic than OO in C.
  • (Both): No obvious, simple way to get a reference type. No pointers or class references. For example, I have no idea how you roll your own linked list in either of these languages.
  • (MATLAB): You can't put multiple top level functions in a single file, encouraging very long functions and cut-and-paste coding.
  • (MATLAB): Integers apparently don't exist as a first class type.
  • (R): The basic builtin data structures seem way too high level and poorly documented, and never seem to do quite what I expect given my experience with similar but lower level data structures.
  • (R): The documentation is spread all over the place and virtually impossible to browse or search. Even D, which is often knocked for bad documentation and is still fairly alpha-ish, is substantially better as far as I can tell.
  • (R): At least as far as I'm aware, there's no good IDE for it. Again, even D, a fairly alpha-ish language with a small community, does better.

In general, I also feel like MATLAB and R could be easily replaced by plain old libraries in more general-purpose languages, if sufficiently comprehensive libraries existed. This is especially true in newer general purpose languages that include lots of features for library writers.

Why do R and MATLAB seem so weird to me? Are there any other major issues that you've noticed that may make these languages come off as strange to hardcore programmers? When their use is necessary, what are some good survival tips?

Edit: I'm seeing one issue from some of the answers I've gotten. I have a strong personal preference, when I analyze data, to have one script that incorporates the whole pipeline. This implies that a general purpose language needs to be used. I hate having to write a script to "clean up" the data and spit it out, then another to read it back in a completely different environment, etc. I find the friction of using MATLAB/R for some of my work and a completely different language with a completely different address space and way of thinking for the rest to be a huge source of friction. Furthermore, I know there are glue layers that exist, but they always seem to be horribly complicated and a source of friction.

share|improve this question
30  
python is written for "hardcore" programmers now? when'd that happen? –  TZHX Jan 28 '11 at 9:12
2  
@TZHX: Ok, so maybe hardcore was the wrong word. A better phrase would be "people who are programmers and think like programmers." –  dsimcha Jan 28 '11 at 15:22
9  
I really dislike the term "hardcore programmer". It sounds seems like some form of elitism, and the term "general purpose programmer" would had just sufficed to make the same point. –  blubb Jul 26 '11 at 13:24
2  
One of the most valuable lessons I was taught when being instructed on Matlab is: "Matlab is not a panacea." Most students see every problem as a Matlab-shaped nail with which to use their new hammer. It's been said more eloquently by more people before me, but the right language for the right job makes all the difference. I work in engineering, and most of us just don't know any better. To a lot of engineers any problem can be solved by Matlab or Excel VBA. I often get pushback when suggesting alternatives to the Big Two in engineering. I don't see that situation changing any time soon. –  Michael Jul 26 '11 at 13:38
3  
You're complaining that R and Matlab aren't for hardcore programmers, but your complaints seem to be that you're not hardcore enough to use R and Matlab. If you want to write in a language which isn't from the Algol family, you have to think in its terms. –  Peter Taylor Aug 24 '11 at 14:28

6 Answers 6

up vote 22 down vote accepted

It seems to me a fallacy to approach domain specific languages with the mindset required for programming at large, or for programming general programs with general purpose languages. Being domain specific, they will likely require a steeper learning curve and an uncomfortable mind set in order to be most efficiently used. I consider writing code in Matlab equivalent to writing highly optimized, domain specific code (on par with, for example, writing efficient and clean OpenGL code). I've also seen them move more and more towards becoming useful as libraries to be used in other languages - see, for example, http://www.mathworks.com/matlabcentral/fileexchange/12987-integrating-matlab-with-c

I would say, use the same process for these DSL as you would for any others:

  • Carefully select the problems which you are solving using Matlab or R, to make sure that they are exactly the kinds of problems which they are best at solving. For example, use Matlab to manipulate your vectors, and not for the rest of your work, if you can avoid it
  • Generally, mix/match the solution to restrict the portions that you program in Matlab or R to the exact subset of the problem which they are built to handle.
  • Follow the mindset of a typical user in the domain that the languages are built for, when designing and building your solution - adapt a vector-mathematical attitude towards the world before starting to work on a Matlab program, for example; possibly write up your work on paper, using standard math notation, first
  • Do the extra work required to build yourself a comfortable work environment, and obtain the tools required for doing the job, even if different from the standard for the DSL. If you're an emacs user, for example, consider using the matlab mode for emacs to do your work; make sure it works as well as the modes you've set up for other languages
  • Be ready to switch out. Especially if you have to come back to the language often, make sure to build yourself a reliable ecosystem where the work you do in the DSL is contained to the domain specific work, only, and it's as easy as possible to switch to another language for the rest of your work. Remind yourself, more often than usual, to look for ways to do the non-DSL specific work in other systems
share|improve this answer
1  
What you say makes perfect sense and is typically how I do things when I have to use Matlab or R. The frustrating part of this is the friction caused by having to integrate multiple languages with multiple ways of doing things and multiple address spaces. Usually, this involves dumping things to text files at seemingly arbitrary points and reading them back, or using some brittle, ugly, hard to configure glue layer. –  dsimcha Jan 28 '11 at 16:54

I'll preface this by noting that I'm familiar with MATLAB, but not R.

The reason that MATLAB doesn't do well with OO, string processing, or custom data structures is that it's not meant to do those things. There are lots of languages for OO, lots that do a good job with string processing, and lots more that support crazy custom data types. None of them are any good at matrix multiplication, because they weren't designed for it.

Just optimizing the vector and matrix operations that MATLAB does is hard enough without dealing with user-defined types or pointers or what not (if it wasn't hard, they wouldn't be able to charge so much for it). Adding fast vector support to existing general-purpose languages is hard to do, too---it adds a large overhead for a feature that few programmers will ever use (too few programmers understand linked lists, how are they expected to use eigenvalue decomposition?).

MATLAB is so alien to you because it was designed to let scientists and engineers to do matrix multiplication and ODE calculations very fast. MATLAB doesn't measure up to your definition of a "hardcore" language because it was never supposed to. Trying to think about MATLAB in terms of Python or D is like trying to think about LISP or Haskell in terms of C or about Verilog and VHDL in terms of JavaScript---they solve different problems and approach problem solving in radically different ways. To be fair, MATLAB made some (okay, a lot of) bizarre language-design choices that I just can't wrap my head around, even from the perspective of a domain-specific language. But there's no particular reason why an astronomer should care that celestial body X is exactly 48 AU away from celestial body Y as opposed to 48.0 AU. All scientific measurements contain some error and it's absurd to say (from the perspective of the average MATLAB customer) that a given quantity is an integer and not a real number with a tiny fractional component.

Now, thankfully, some libraries are coming onto the scene which do exactly as you suggest: good support for scientific computation in a general-purpose language. For Python, there's NumPy/Matplotlib which has some rough edges but otherwise provides reasonable MATLAB functionality inside Python. The reason there haven't been other project like this is that the libraries are incredibly difficult to write and serve a market already covered by MATLAB and FORTRAN.

If you absolutely have to use MATLAB or R, you can't approach programming in them like a "hardcore" programmer, you have to approach it like a "hardcore" scientist or engineer. For LISP, you have think in recursion. In MATLAB, you just have to think in matrices. Brush up on linear algebra (MIT's lectures on the topic are a great review). Otherwise, the only way to "survive" MATLAB is with practice to recognize when a loop can be replaced with a vector operation or when your problem reduces to finding the eigenvalues of an outer product.

share|improve this answer
    
Yes, overall I love Numpy/Scipy/Matplotlib and use them whenever the obvious alternative would be MATLAB/R. My only complaints about these are that they're not as deep as MATLAB/R and that, since they're Python, they're still somewhat slow. –  dsimcha Jan 28 '11 at 14:22
1  
@dsimcha, this is factually incorrect. In this performance study, Numpy is on a par with MATLAB, and Pyrex is well within a factor of 2 of C++. –  wvoq Jul 24 '11 at 0:08
    
@wvoq: Clarification: I meant the Python interpreter is slow, not Numpy. I know Numpy is mostly a wrapper for BLAS and LAPACK, which are fast. Of course there's still the fixed overhead of calling into this code. I'm also aware of Pyrex, Cython, etc, and they do help, but you're still mixing languages at a fine-grained level and this can still be a source of friction. –  dsimcha Jul 25 '11 at 19:59
    
@dsimcha, The overhead of calling Numpy is practically a constant. In the performance study I mentioned, you're gaining a few tenths of a second with C++. That time must be compared to time spent writing and debugging, and debugging BLAS calls at that. It might be instructive to ask why not write everything in assembly? Or even straight machine code, since the conversion from assembly to machine code adds some fixed overhead? –  wvoq Jul 26 '11 at 1:53
1  
@dsimcha Huh? You want to do everything in a single (fast) language, you like Python most and then you're complaining about Python being slow? So what's the whole point. I guess you just want MATLAB to have more general purpose features and be faster than an interpreted language? –  Christian Rau Aug 24 '11 at 12:56

Your repeated use of the term "hardcore programmer" in reference to yourself. along with your insinuation that the designers of R and MATLAB are not, strikes me as very silly and encourages people not to take your criticisms seriously.

If you would like to read some serious criticism of R, you would do well to read this piece by Ross Ihaka, one of R's designers. It seems significantly harder core to me to design R than to use D or Python.

share|improve this answer
2  
-1. I never meant to imply that the designers of MATLAB and R are anything other than very good, hardcore programmers. However, MATLAB and R are not designed for hardcore programmers. –  dsimcha Jul 25 '11 at 19:50
2  
"...it seems like these languages were designed by people who aren't hardcore programmers and don't think like hardcore programmers." –  wvoq Jul 26 '11 at 1:24
5  
You still haven't clarified what a "hardcore programmer" is. By your examples, "hardcore" sounds like it just means "most comfortable with C++", in which case R and MATLAB will not be hardcore by definition. Almost all of your examples reduce to complaints that these languages are not what you are used to, without asking why experts in these fields have seen fit to implement them that way. –  wvoq Jul 26 '11 at 1:36
1  
Also, in terms of definition, I thought it was obvious, but a "hardcore programmer" is just someone who's comfortable with basic programming concepts like references/pointers, object orientation, lambda functions, basic data structures, etc. and is used to developing in a general-purpose language. –  dsimcha Jul 26 '11 at 12:58
1  
Well firstly, "hardcore" is a pretty loaded term if intended to denote "someone conversant with basic concepts in oo/procedural/functional programming". Secondly, I can't speak for MATLAB, but R has all of these things. The only difference is that in R you are encouraged by the language to couch your problem in such a way that the elements of statistical vocabulary become your primitives. The reason for this is that statisticians and machine learning folks often work with problems easily expressed this way, which makes R a natural fit even if you're comfortable with "harder core" stuff. –  wvoq Jul 26 '11 at 20:11

The extreme emphasis on vectors and matrices to the extent that there are no true primitives.

It depends what you call a true primitive. In R, a vector is a true primitive; that is, all variables are vectors. Likewise, in MATLAB all variables are matrices.

The difficulty of basic string manipulation.

In MATLAB, string manipulation is powerful but I agree that the code is often ugly and unintuitive (at least for now). For R, there is the stringr package, which is as nice to use as tools in any other language.

Lack of or awkwardness in support for basic data structures like hash tables and "real", i.e. type-parametric and nestable, arrays.

In R, vectors have names which works like a hash. There are also the hash and filehash packages. Not sure about MATLAB implementations, but you can call JAVA or .NET versions easily if you want.

They're really, really slow even by interpreted language standards, unless you bend over backwards to vectorize your code.

Once you get the hang of vectorisation (I'm sure you will, if you're really hardcore) you'll curse having to use loops when you return to other languages. Speed of execution is a tradeoff for speed of programming.

They seem to not be designed to interact with the outside world. For example, both are fairly bulky programs that take a while to launch and seem to not be designed to make simple text filter programs easy to write. Furthermore, the lack of good string processing makes file I/O in anything but very standard forms near impossible.

They both can read and write data in pretty much any format. They can both be called from most other programming languages. Or from a command prompt. You can create GUIs with them. How is that not interacting with the outside world? If you're struggling with your text filter program, ask on stackoverflow.

Object orientation seems to have a very bolted-on feel. Yes, you can do it, but it doesn't feel much more idiomatic than OO in C.

Agreed; they're primarily procedural languages.

No obvious, simple way to get a reference type. No pointers or class references. For example, I have no idea how you roll your own linked list in either of these languages.

Agreed in R. In MATLAB, references are called handles.

You can't put multiple top level functions in a single file, encouraging very long functions and cut-and-paste coding.

Nonsense. Just create multiple files.

Integers apparently don't exist as a first class type.

They do. See int8, int16, int32 and int64.

The basic builtin data structures seem way too high level and poorly documented, and never seem to do quite what I expect given my experience with similar but lower level data structures.

They are suited to doing data analysis. Please give specific examples of unexpected behaviour.

The documentation is spread all over the place and virtually impossible to browse or search. Even D, which is often knocked for bad documentation and is still fairly alpha-ish, is substantially better as far as I can tell.

There are many kinds of documentation. Start with ?some_function, RSiteSearch('some concept'), rseek.org, and the sos package. Not to mention the manuals that come with the install. Or a good book.

At least as far as I'm aware, there's no good IDE for it. Again, even D, a fairly alpha-ish language with a small community, does better.

Try Architect or RStudio or the Revolution Analytics IDE. See "IDEs and editors for R" section of the Stack Overflow info page for links and more options.

share|improve this answer

MATLAB can integrate with Java and C/C++. You can implement all of your non-numerical workload in these languages, and invoke them from MATLAB.

their use is necessary

Is there a reason why it is necessary? Are you working on an existing MATLAB code base written by other people? Is it a work requirement? (or class requirement if you're in school) If not, you may consider using SciPy or NumPy instead.

Unfortunately, in my personal opinion, if this situation is being forced onto someone, it is not always survivable. Even in college, not every engineering student can get used to the MATLAB way of computational thinking.

share|improve this answer
1  
I'm aware of these solutions, but they seem fairly backwards. I'd like to call MATLAB from C++, Java, etc., not the other way around. I'd like anything but MATLAB to be my "driver" language. –  dsimcha Jan 28 '11 at 14:19
1  
@dsimcha you can call MATLAB and R libraries from C. See mathworks.com/help/techdoc/matlab_external/f38569.html or math.univ-montp2.fr/~pudlo/R_files/call_R.pdf –  Charles E. Grant Jan 31 '11 at 2:20

I work with MATLAB, Python and C (and sometimes C++), and I consider myself to be (primarily) a software developer, sitting opposite colleagues who tend to be data scientists, mathematicians, or other domain specialists.

Although I would be the first to admit that it is not a general purpose programming language in the sense that C or Python is, I actually enjoy writing scripts in MATLAB rather a lot, particularly for things like time-series analysis or image processing.

There are a couple of features of the language which, although generally implemented quite inefficiently, are a joy to use. For example, take logical indexing: I can create a logical vector or matrix that selects a region-of-interest, and name it "isInROI", performing a filter operation to select elements from the vector or matrix "data" in that region is then simply a matter of writing: "roiData = data(isInROI)".

It is moments like this that really make me appreciate MATLAB, and allow me to overlook its other, numerous and much-discussed sins.

share|improve this answer
    
Note that numpy has logical indexing as well : docs.scipy.org/doc/numpy/user/… –  jarondl Feb 17 at 8:21

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.