Choosing language to write general library in

Question

I'm about to start writing a library to support machine learning algorithms (decision trees, ANNs, Bayes nets, etc.), and I'm planning on making this a very general library. By this, I mean I want to be able to plug this module into as many languages as possible.

To clarify, here's why I've started writing it in C++ (Windows headers):

• I know I can import C++ libraries into C#, and I've heard you can import C++ into Python. So, obviously, the library can be used in 3 languages, possibly 4 if I don't use system-specific headers.

• Is there a language where I can cover a wider spread, weighted by popularity of languages? (For example, C# and C++ have pretty high weight to me because I use both frequently, Python less so.)

Answer 1

If you are careful, you can write a quite general library with several bindings to other languages. For example both GTK (a graphical toolkit, coded in C, LGPL licensed) and Parma Polyhedra Library (a library handling numerical inegalities and abstractions, coded in C++, GPL licensed) have interfaces to several languages. Be sure to learn how to bind code to several languages & implementations (Ocaml, C++, Python, Haskell, Java, Lua, ...). Learn about memory management and garbage collection techniques.

I also suggest that you make your library free software, it is IMHO the best (and perhaps the only) way to make it widely used.

I strongly suggest that you make your library working on several platforms (from the beginning), like Windows, Linux, MacOSX. Don't make it Windows only!

Study the competitor's libraries. There already exist several machine learning libraries.

So the language to code your library is not the most important factor. And you could also make it a server, with published protocol to access it.

Perhaps chosing the language you know well is better. Otherwise, I might recommend Ocaml, C, C++, Java, ...

Answer 2

Generally speaking, you can write in whatever language you want, as long as you expose a C binding then virtually any language can bind to it.

Answer 3

If wide compatibility and portability are your main concerns, I think there are two serious options:

• C/C++ - this will be most easily importable into most other languages, as nearly every language has some mechanism for calling C code. You can compile C/C++ code on pretty much any platform. Also if performance is a priority, it's nice to be able to code very close to the metal.
• Java - it's the most popular language overall, with definitely the widest spread and portability of any of the major VM platforms, also is a very common choice for machine learning development. I suspect the majority of your potential library users will want to use Java. An additional advantage of pure Java is that it gives you binary (bytecode) portability so you don't even need to recompile for different platforms. The downside is that calling Java from other languages is a bit more tricky (one solution to this would be to provide a REST-style or socket API)

My personal recommendation would probably be to go for Java, because that's where the majority of your potential library users will be (research, enterprise software development, cloud startups etc.). If you want to make it really easy for these people to adopt your library, then a pure Java library will be much nicer than a set of native code bindings.

P.S. I do have quite a bit of experience in this space and have researched the options quite extensively, I'm currently writing a machine learning library on the JVM using Java + Clojure.