Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am about to start a new project and must choose a compiler. Because we are working in the embedded domain, one of the criteria that we are evaluating is the size of the output of the compiler.

There are benchmarks, such as Dhrystone and Whetstone for evaluating compilers in terms of how compilers handle string, integer, and floating point functions. Is there something along these lines for providing a general comparison of the ability of compilers to produce small output files?

share|improve this question
    
what is the "code size" criteria? Binary output size optimization? –  BЈовић Apr 16 '12 at 14:37
    
The comparison will be straight optimised size of the binary output. –  tehnyit Apr 16 '12 at 14:51
    
Also, what is "code code benchmarking"? What kind of source are you looking for? –  BЈовић Apr 16 '12 at 14:51
    
Long day..corrected again –  tehnyit Apr 16 '12 at 15:12
2  
I believe the new wording makes your question much more clear. –  Thomas Owens Apr 16 '12 at 19:48

2 Answers 2

up vote 3 down vote accepted

The best benchmark is one that measures the kinds of things your code is going to do. This generally means that the best benchmark is going to be YOUR OWN CODE, from previous projects/products.

In the embedded C world, processor selection is generally controlled by hardware issues: power, size, features. Once you've selected a processor, there usually won't be that many choices of compiler, and compilers for the embedded world are generally going to be about uniformly good at generating tight code.

Also, it is generally a LOT easier to generate optimal code for modern non-x86 processors than it is for the latest generation of 8088 hardware emulation engines. (Modern x86 processors are all RISC machines emulating the x86 instruction set in microcode.) The exception to this is digital signal processors, of course: getting maximum bang for the buck out of the quirkier DSP speedups usually requires a human expert assembly language programmer, who knows how and when to use the (typically) 1-instruction 2x2 butterfly operation, the bit-reversed addressing mode, and the fast multiply-accumulate instructions, plus knowing how to exploit parallelism. Modern compilers can't really be expected to recognize radix-2 and radix-4 FFT source code.

share|improve this answer

I propose using csmith. It's actually a tool to find bugs in compilers, but it can be misused (in a positive way). The idea is to generate a statistically significant number of random C programs, feed those through all compilers that are eligible and compare the results.

I did something similar a while back (but I was after compile speed and execute speed), where I had to tweak csmith a bit to produce different flavours of programs, but it's certainly a great starting point.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.