Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am an Sr. developer/architect/Product Manager for embedded systems. The systems that I have had experience with have typically been small to medium size codebases - typically close to 25-30K LOC in C, using 8-16 and 32 bit low end microcontrollers. The systems have been entirely bootstrapped by our team - meaning right from the start-up code to the end application code has either been written by the team, or at the very least, is thoroughly understood and maintained by us.

Now, if we were to start developing more complex systems with complex peripherals, such as USB OTG et al. (think, low end cell phones), there are libraries and stacks available commercially and from chip vendors that reduce the task to just calling the right APIs and being able to use those peripherals. Now, from a habit point of view, this does not give me and the team a comfortable feeling, not being able to comprehend the entire code tree, with virtual black boxes at the lower layers.

Is it reasonable to devote, and reserve, time getting into the details of how the APIs are implemented, assuming that the same would also entail getting into details of relevant standards (again, for USB as an example)? Or, alternatively, should a thorough understanding of the top level usage of the APIs be sufficient? This of course assumes that the source codes to all libraries are available, which they are, in almost all cases.

Edit: In partial response to @Abhi Beckert, the documentation is refreshingly very comprehensive and meticulously maintained, AFAIK and been able to judge. I have not had a long experience with the same.

Edit2: Excellent advice, everyone! Let me add that the evaluation phase is being conducted by only 2 devs, including myself. Also, shims may be potentially expensive in a resource constrained embedded environment, but I will have a look.

share|improve this question
add comment

5 Answers

up vote 11 down vote accepted

It is quite understandable that you're going through this type thinking because you're used to having full control over every piece of code.

It is indeed scary when you start getting into a situation where you have to rely on the robustness of external API and assume that it's developed correctly.

I think that you have to use a balanced approach. My suggestion would be that you created a formal mechanism for accepting a new external API into your system. One good way of dealing with external code in a rigorous way is to create a shim that standardizes interface to external API functionality with the rest of your software.

For instance, if you're using an external USB library, you should create a stub library in the middle that the rest of your project links to, and defines the functionality that the project is actually looking for. Then a small team of programmers from within your group can actually vet out various solutions that deliver that functionality and link those in as the backend.

Using this thin-middle layer approach has several advantages:

  1. You're not spreading external API that is only partially understood all through your code.
  2. You can easily swap out the external dependency with something new.
  3. You can create very specific unit tests that comply with your exact requirements and ensure that both sides of the thin layer are acting the way they're supposed to.

The advantages of this are obvious: maintaining external API's separately, ensuring bug localization, etc.

Of course you can modify this approach and adjust it to your specific situation. But I believe that you can do this without creating a lot of overhead.

With some effort, for a release you might actually be able to eliminate a good chunk of the middle layer by doing interesting compile and linking manipulation, such as inlining, weak symbols, etc.

share|improve this answer
1  
+1 for adding an insulation layer which could also be used as an abstraction layer so different API's (libraries) can be plugged in as required. –  Marjan Venema Dec 1 '11 at 7:43
    
@Rohan Singh hehe thank you for the edits, that was a late late night :-) and i was using a dictation program. hugs –  Ahmed Masud Dec 4 '11 at 19:28
    
@AhmedMasud Sure thing, glad to help! –  Rohan Singh Dec 6 '11 at 19:16
add comment

I would recommend having at least one developer per product team become familiar with each third party SDK. In theory, you should be able to treat them like a black box. In practice, hardware vendors are much better at making chips than APIs. Some APIs are little more than thin wrappers around the registers, hardly abstracting anything. Often their updates significantly lag chip releases, so if you use new chips in your product you will be writing your firmware while they are still working out the kinks. It's not uncommon for us to either work around a broken API with direct register writes, or fix the SDK ourselves and send a patch to the vendor because we couldn't wait for their support cycle.

You might get lucky with unusually high-quality SDK, but I still would factor time into your schedule just in case.

share|improve this answer
    
The evaluation phase is being conducted by only 2 devs, including myself. Given this constraint and the time required for the same being done by only 2 devs, does this strategy make business sense? Point well taken, however. –  Vaibhav Garg Dec 1 '11 at 5:54
    
API do not have to abstract things away. –  user1249 Dec 1 '11 at 12:35
    
@Thorbjørn, in the embedded world you always have a default "API" of just register reads and writes. Some vendors also publish a C library on top of that. If it provides little abstraction, there's not much reason not to just use the registers directly, as the OEM is going to have to write their own abstractions anyway. –  Karl Bielefeldt Dec 1 '11 at 14:13
add comment

It is certainly reasonable to assume that you will pick up vital parts of the lower levels over time, particularly when you encounter a problem where those details are necessary to finding the solution. This is normal when working with large OSes.

That said, it really helps when I'm developing UI for Windows that I learned the basics in C++ 15 years ago...

share|improve this answer
add comment

In my experience, it depends how stable things are. If a component is rock solid and rarely ever changing, then you don't need to know how it works. But if something is new then you cannot afford to use it unless you're intimately familiar with the internals.

Knowing "the whole stack" is critical when you run into bugs or design flaws. But if something is already perfect or near-perfect you can safely ignore how it works under the hood.

Documentation is also important, you need good documentation for something you don't know well.

As a real world example, I'm happy to make use of MySQL every day without understanding much about it. But I won't touch MongoDB because it's a relatively new project and I haven't got the time to sit down and really learn how it works.

I suspect Mongo might be a better database for my projects, but it's 3 years old compared to MySQL's 16 year history.

If something is old old and proven, you can safely just learn how it's outermost layer works. But if it's fairly new you MUST understand the internals or you are better off not using it at all.

share|improve this answer
1  
It is all a matter of trust. –  user1249 Dec 1 '11 at 12:36
    
I have to agree about the trust bit and I'd also question the merit of using age as a determining factor - yes the MySQL project is 16 years old, but the current version of MySQL isn't to varying degrees (same is true of a lot of "established" tools) –  Murph Dec 1 '11 at 12:51
    
@Murph we do not use the bleeding edge latest version of MySQL on any of our production servers. We use an older version and only install minor security/bug updates. –  Abhi Beckert Dec 1 '11 at 20:41
    
@Abhi - my point is that whilst age may point to stability its by no means certain, there are a lot more factors involved (including that older may be worse because its failed to adapt to environmental changes - 16 -> 32 -> 64 bit being the most obvious example, file systems being another). Frustratingly even very popular and widely used tools can have issues - often that one is supposed to "just know" –  Murph Dec 2 '11 at 10:19
    
Being old doesn't do anything to guarantee it is going to be stable. But being brand new almost guarantees there will be issues. There are no hard and fast rules with things like this, it is a sliding scale, depends how much time you are willing to put into learning how something works, and how much benefit it will bring to you. –  Abhi Beckert Dec 2 '11 at 21:10
add comment

Great answers. I would only add, if you're using a 3rd-party SDK, it should be open-source. The reason is not so much that you will ever need to go inside it, rather you can make a snapshot of it in time, and never have to worry about its support and availability going away.

Just as an example, one product I maintain works by generating Fortran and compiling it on the fly. We have used compilers from MS, Watcom, Lahey, Digital, Compaq, and now Intel, every time with minor and major changes. What an unbelievable recurring headache. We've now switched to GNU. Should have done it a long time ago.

share|improve this answer
    
Most(almost all) of the stack comes with the source code, for all vendors in this case. Advice appreciated.Thanks –  Vaibhav Garg Dec 1 '11 at 16:21
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.