Is it OK to live without knowing how the program you created works?

Question

I mean, there are really useful libs that can solve problems when you are stuck and do not know how to solve this or that with your knowledge of programming language you use... For example, Boost for C++ or JQuery for JavaScript or Spring for Java... They solve problems in seconds and you do not really care how they did it (despite that they are written in the very same language you are programming in)... So I wonder am I alone in using libs while not being capable to write solutions for my problems from scratch or is it standard practice?

Answer 1

Is it ok not to understand how to solve the problems yourself, and use libraries instead?

In the general, no, it's not.

A library can save you the (hard!) work of figuring out how to solve a problem, and then debugging the solution, and then, maintaining it. But, if you're going to use it, you'd better make sure you understand how it works - why the solution actually solves the problem. You don't have to know how to invent cars, and engines, and robots that build car engines, if you're working as a mechanic - but you'd better understand how the parts work, what they all do, and how they fit together!

This is the reason why you will see many people become very specialized - many times only learning how to work with a single language, a single platform, a single framework and set of libraries.

That being said, there's only so much you will have time to learn. Sometimes you've got to take shortcuts - take them, but know they are shortcuts. Maybe you only read enough about a library to know you could figure it out, if you had the time. Or maybe you only figure out the two functions that you actually need to call, and only enough to make the calls correctly. That's a shortcut, which will come at a price - usually later, when someone (maybe an older, and more experienced, you) has to fix the code.

Answer 2

Yes - and we all do it!

Let's take, for instance, a very simple bug I was fixing in some Mac-related graphics code. The code around the bug involved several steps:

1. First, an Objective C method allocates a pixel buffer using malloc(), and attaches that to its Objective C object.
2. Later, something happens, and a C routine sends a message to the Objective C object and retrieves the pixel buffer.
3. The C routine compresses the pixel buffer contents using jpeglib, and sends it out over a TCP/IP connection.

There's an awful lot going on there! Here are a few things:

• A dynamic memory allocator to implement malloc(), which assumes memory is physically contiguous and linearly addressable.
• The underlying Darwin kernel virtual memory system to map both fragmented physical RAM, and disk space (which is a different physical device from the RAM), into something that appears to the dynamic memory allocator like it's physically contiguous and linearly addressable RAM.
• Objective C's object system
• The Mac OS runtime messaging system, and the way it interacts with Objective C objects
• The jpeglib library's implementation of the JPEG lossy raster image compression standard, which uses a discrete cosine transform algorithm
• The userspace networking routine for sending out the data, which calls through the various TCP and IP protocol implementations, which in turn calls into the OS kernel. Then depending on what you've got turned on for your networking, it may call a driver for the Ethernet port, a wi-fi chip, or more unusually a USB or Firewire driver.

Do you understand all the details of how all those things are actually implemented? I sure don't! I doubt there are very many people on the planet who do - maybe even none. So I just don't worry about it.

But it's a good thing to be curious, and to learn at least a little bit about the libraries and tools you use. When I first started programming, I knew compilers and operating systems couldn't be magic, but they sure seemed that way to me. By indulging my curiosity about those things, I've learned an awful lot and had a great career so far.

Answer 3

Once computerworld.com.au asked Bjarne Stroustrup "Do you have any advice for up-and-coming programmers?"
And he answered "Know the foundations of computer science: algorithms, machine architectures, data structures, etc. Don't just blindly copy techniques from application to application. Know what you are doing, that it works, and why it works. Don't think you know what the industry will be in five years time or what you'll be doing then, so gather a portfolio of general and useful skills. Try to write better, more principled code. Work to make "programming" more of a professional activity and less of a low-level "hacking" activity (programming is also a craft, but not just a craft). Learn from the classics in the field and the better advanced textbooks; don't be satisfied with the easily digested "how to" guides and online documentation - it's shallow."
Hope it will clarify your doubts about what is required for a True Programmer and what is Necessary for anyone to be one.

Answer 4

I find the main reason we use libraries is to not "reinvent the wheel" all the time, abstracting the problems they intend to solve. You could try to solve the problems yourself, but that would take longer time.

However I do feel that we also need to know or guess how the problem is solved by the library. This is usually documented in the library´s user documentation and with open source software, you could always look at the code yourself.

Also we usually solve problems by abstracting the difficult parts anyway, so why isn't it ok?

Answer 5

The libraries are there to provide solutions to common problems. You need decide if they solve the particular problem that you are solving. They are NOT a substitute for not knowing how to solve a problem. For example, suppose your application requires a hash table, you should have enough knowledge to understand what problem a hash table is solving. You should be able evaluate the performance of the library you are using to decide whether or not it will work in your application. I believe using a library to cover for inadequate technical knowledge is not the correct use case. The decision to use a library should revolve around whether or not using library will speed up development and provide a tested and reliable solution. The decision to use a library should not revolve around the programmers inability to solve a given problem.

Answer 6

Up to you, really.

The best you understand the tools you are working with the better you can take advantage of them.

For example, I rarely use jQuery, but when I have to I know what to take advantage from it, and how can I make it coexist with other frameworks like Mootools.

Also, soon I'll adventure into gamedev with UDK, and I'm sure the more I understand about it the more I will be able to bend it to my evil will, but I could also just follow no-brainer tutorials. I choose the first, just a little extra time and brain cycles and I'll get better and easier results.

Answer 7

It's important to know your realm, and your part of the process.

For example, say you are using an image processing library. Do you really need to know all about gaussian blurs, transformations, and color spaces? No. But you do need to know why you are using the library in the first place. Or a framework's sorting function. Do you need to know the actual sorting algorithm used? In most cases, no. But you do need to know why you need the data sorted.

On the other hand, if you are writing a compiler, you darn well better know how a compiler works, since thats your part in the process.

Certain frameworks like jQuery abstract away a lot. Do you need to know how exactly they are working? No. But having a strong, fundamental understanding of what the library is doing will be very beneficial to you as you write code because you will understand better why the framework is made the way it is, and be able to use it to its full potential.

Answer 8

Upon my experience: since you can't eliminate the library-dependence, you and your team should know enough to solve the problem.

As programmers, we have little time, so we must choose the one which has highest priority. The problem must be solved, as fast and gentle as possible. Only this reason makes "learning all about things work" somewhat redundant.

The things I want to add here is "dependence". As a community, we are all dependent to others. We stands on the Giants to build our application: Java, .NET, API... And we trust the Giants about their work; because it works for so many people. If you have a problem about the framework, or API, there's a good chance that others have faced it somewhere, and there's a solution/work around.

The only problem here: maybe, somewhere, in a restricted criteria the Giants collapsed. For example, flash is not supported in some OS, and there's many things we couldn't do without it. This possibility is more than zero, but in this case we have little things that we can do. Only in these cases, the knowledge about "what is behind the hoods" proves useful, as it point out where the problem truly is and may create a big work-around; but I am not sure the time we invest really worth it.

To cope that possibility, I think there's a solution: because most of programmers can easily catch the "surface working" of a library, and only sometimes we really in need someone who is very well-understanding: let divide the team to do that. Trying to comprises a team that each one has experted about 1,2 useful libraries/tools/"skill set" that involved: one has good experience about jQuery, one has specialize with database, ... This will help much in minimizing risks.

Answer 9

Another point of view is security. When you use a library of which you don't know the exact inner workings, then you make assumptions about what's exactly happening. Each failed assumption may open an attack vector for a malicious attacker.

When calling a Quicksort, then you should be aware about the worst case behaviour. Else an attacker may be able to inject worst case data and to perform a DoS.

When calling a compression library, then you should be aware that when some data compresses to less bytes, then there must be data that "compresses" to more bytes than the original. So when your assumption is that the output buffer only needs the size of the input data because it compresses [to less bytes], then there is a buffer overflow waiting to happen.

You should know enough fundamentals about the things you're going to do, to be able to prove your assumptions true. Else the library should explicitely take care for this, e.g. throwing an exception when the provided output buffer is not large enough.

Answer 10

It's OK to not understand everything you use as long as you are sure it works. Once you are bitten by a bug in the lib then there's time to see how it works, why it works and why it doesn't. Of course you are always welcomed and encouraged to look under the hood even if you don't have to.

One of the hard things in programming is overcoming temptation to solve all the problems by yourself.

Answer 11

It is ok but it is dangerous. As general practice, one should know working of what he has developed.

Answer 12

Kinda...

It's fine as long as you get the general gist of what the library or framework is trying to do. As for the internal parts and what not then, no. Take the pragmatic approach. It works, it did what I want it to do, alright.

The point is to not get boggle down with bunch of small details and just implement your damn idea already.

I guess, the point is, you ain't going to know everything. Seriously, you have so little time to investigate everything, because it'll distract you from your main goal of creating that idea of yours. Little by little, perhaps, you can set a side some free time on the weekend to read up a chapter or so on the subject.

But don't try to figure everything out, unless you have lots of free time... Look at it this way. The reason for programming languages is the shield us from doing assembly code, the reason for assembly code is to shield us from doing 1's and 0's. I don't think you need to know every fine details of the mechanism behind it, but just know the general gist of it. Like a garbage collector, I know it going to deal with my pointers/memory, I don't care what magically neat algorithm it uses, I just know it works (for what I need) and it doesn't do any else. Maybe the con of it but meh. Unless of course you're in the field where you have to deal with it. Then you wouldn't be asking this question anyway cause it's part of your job haha.