Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Is there, somewhere, a freely usable/accessible script, source file, or whatever, that is able to measure the compliance of a given C++ compiler?

For example, the Acid3 test for browsers: http://acid3.acidtests.org/

The results I dream of would be a global percentage note (or multiple notes, one for each standard, e.g. , c++98, c++11, c++14, etc.), and then detailed tests with "success" or "failure" for each of them.

Background: I had a discussion at work about boost and some challenged compilers. My interlocutor spoke about boost being an academic project, because it won't work in major C++ compilers, and me answering that mentally challenged compilers should not count. Being able to measure with code the actual conformance of a compiler would help both in evaluating the compiler, and discovering the "corner cases" that should be avoided in cross-platform code compiled with them.

Edit: 2013-06-22

Not an answer, but apparently, the C++ committee is working on the subject:

SG10, Feature Test: Clark Nelson (Intel). Investigation into whether and how to standardize a way for portable code to check whether a particular C++ product implements a feature yet, as we continue to extend the standard.

Source: http://isocpp.org/std/the-committee

share|improve this question
    
There's this: peren.com/pages/products_set.htm. Won't attempt an answer, as I haven't used it and I'm not sure of it's status in the C++ community. –  Yannis Rizos May 28 '13 at 21:52
    
What major compilers does Boost not work on? It works on Microsoft's Visual Studio Compiler, GCC, and CLANG. –  Travis Pessetto May 28 '13 at 21:56
    
@YannisRizos : Thanks for the comment, but this is a commercial product whose results are not freely accessible. I'll rephrase my question to ask for "free/open" tests. –  paercebal May 28 '13 at 21:57
4  
I nearly fell off my chair laughing at this question.... Its a very valid question, I am interested to here the answers..... It's 2013 - surely something as fundamental as our tool chains use TDD - or is it a case of "Do what I say, not what I do...... " –  mattnz May 29 '13 at 4:01
2  
Why the close votes? I believe this is a very valid and relevant question. Proof of compliance to standards is mandatory for most tools used by most professionals. –  mattnz May 29 '13 at 21:16

3 Answers 3

The closest thing I'm aware of is in the context of a specific standard version. Similarly, there are blog posts that contain like content. However, I'm not aware of a de facto or de jure body, test, or test suite that rules on such matters. Even isocpp.org doesn't appear to have conformance resources currently. The closest you can get is probably each vendor's specific conformance charts which is inconsistent at best.

e.g. Clang, GCC, MSVC, Intel

There are really only a few competitive compilers (at least in the x86/x64 space - I assume it's also the case for less popular platforms), but I agree that it would be very nice to have a reference, especially now that the standard is picking up speed.

share|improve this answer

I am unaware of any standards testing, though the following URL is the official standards website comitee http://www.open-std.org/jtc1/sc22/wg21/.

Also, many things that are found in Boost make their way into the C++ standard. According to Dr. Kenneth Sundberg of Utah State University.

I also can't verify the accredibility of it yet, I am still trying to figure out if it is credible, but the Google Group at https://groups.google.com/forum/?fromgroups#!forum/comp.std.c++ claims that no official test exist.

share|improve this answer
    
Dr. Sundberg? Like in Dr. No or Dr. Guttenberg? –  ott-- May 28 '13 at 23:18
3  
@ott No, as in a Ph.D. in Computer Science. –  Travis Pessetto May 29 '13 at 3:24

One way to test compliance of a given C++ compiler is to run a (large) testsuite, i.e. a bunch of C++ files that test all aspects of the language.

Well known testsuites are the gcc and llvm testsuites. These test all kinds of aspects of the compiler, obviously including conformance. However it is hard to particularly assess conformance specifically as (to the best of my knowledge) there are no special reports generated on conformance.

In this realm you have:

  • the GCC C-torture tests used to test the gcc compiler
  • the LLVM testsuite: a mix of directed tests for the clang compiler and llvm backend plus a set of full-fledged applications to test the compiler.
  • the LLVM libcxx test (part of the libcxx repository) to test C++11 support.

There are some extensive testsuites that to test C/C++ conformance available. Alas these are all commercial licenses in the order of 10k-40k.

  • SuperTest: The most extensive of this list. Roughly 3 million directed tests towards C99, C++, EmbeddedC, DSP-C conformance.
  • Plum Hall. C and C++, including the C++ library. Includes ANSI C conformance, random program generator, test templating, and a scripting interpretor to control the overall testing process.
  • Perennial. C and C++.
  • Nullstone. C only.
share|improve this answer

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

    
would you mind explaining more on what it does and why do you recommend it as answering the question asked? "Link-only answers" are not quite welcome at Stack Exchange –  gnat Feb 7 at 17:49
    
If I had asked this question, I would have been very happy to get this answer. –  david.pfx Feb 11 at 13:11

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.