Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

We have a system here. Recently there is a wrong calculation in one of the number in the report generated by the system. Through out our experience, we've never encounter any problem/error in this system for some years.

Because the writer of this system had already gone, we can hardly trace the programs. But we've verify the input data, the setting and they are right.

Now my question is, will a computer program suddenly go wrong without any logical reason? If I slam on the server machine, will one of the number which the computer is calculating, become another number and make the calculation wrong?

I agree my idea there is quite mad, but I just want to know, how can we know the problem is not caused by the program and the input, but some other factors?

P.S. This mad system has no log.

share|improve this question
8  
One of the RAM modules in my PC had exactly one defect bit, so a program being unfortunate enough to use that bit might deliver a wrong result. Running memtest86 on your machine might be a simple way to exclude that kind of problem. –  user281377 Oct 4 '11 at 6:45
16  
yes, by deleting it –  Steven A. Lowe Oct 4 '11 at 8:56
6  
Some pieces of hardware actually have bugs. It is a testament to the chipmakers of the day that they are so few. I would suspect the software first. –  user1249 Oct 4 '11 at 9:03
    
There is always a logical reason for a program to go wrong. A slam is a logical reason. –  mouviciel Oct 4 '11 at 9:18
2  
You can have a statistical bomb, or a malicious compiler, or bad ram, disk, or a virus that can write to your ram or modify OS, or OS bug, or a bug in a library somewhere, or the famous merge sort bug, or ... –  Job Oct 4 '11 at 12:49

13 Answers 13

up vote 8 down vote accepted

I would say no!

In theory the answer is no, we can only test for:-

  • some limited number of environments.
  • some limited number of timescales.
  • some limited number of test cases.

This is considerably less than the total possible number of environments, times and cases the program may encounter in its lifetime. Also we have little knowledge of the future should you program cope with 10,000% inflation, should your program cope with a super duper new 31 bit architecture?

The theory is supported by experience I personally have encountered:-

  • Programs breaking when moved to a different locale. Checking for "MAY" when the month was "MAI".
  • Programs failing tests on a new version of the compiler, A bug in the previous version in conjunction with a bug in the program produced the correct result.
  • Programs breaking on a new release of the OS. When Solaris increased the default number of directory entries the SMALLINT returned by ftok() always returned zero for the first file in the directory.
  • programs breaking because it was the first time they encountered a particular combination of inputs, which were both valid and unexpected and would never have been tested for -- negative interest rates on deposits, zero weight items to be shipped, items of such low value that VAT couldn't be calculated etc. etc.
share|improve this answer
    
I say yes, with a provision - If you have a multi-threading. Ever heard of "Race Condition". –  mattnz Aug 12 '12 at 21:09

It is just barely possible that the problem is caused by failing RAM, but this is relatively (very) unlikely. Run a memory test, but be ready to look through code.

share|improve this answer
    
To the downvoter - I've seen this happen. Once. –  James McLeod Aug 12 '12 at 22:57

Most (standard) computing is deterministic, I would think.

If possible, set it up to do a batch of 1000 or 10000, etc., iterations with the same input data, and verify that the results come out the same.

Make sure the current values going into the calculation would cause an over- or underflow anywhere (if it is an older system it may not have been intended to be used this long).

Y2K11 anyone?

share|improve this answer
    
To do N iterations and verify the results does not prove correctness. At best, it proves absence of error within the sample set, and even that is assuming that your test case (and the implementation of it, as well as its execution) is absolutely correct. While testing is very useful, it does not address the OP's concern. –  Michael Kjörling Oct 4 '11 at 14:18
    
@Michael Perhaps I should clarify, I'm not suggesting trying to "prove" anything with this approach, but if it goes umpteen more iterations without ever showing the error again, I'd be thinking sunspots and not integer overflow. It still gives you more insight than not, IMHO. –  jonsca Oct 4 '11 at 17:07

Yes, hitting a system could bend and/or move parts enough to cause a temporary open-circuit (or possibly short-circuit, though it's probably less likely).

share|improve this answer

Now my question is, will a computer program suddenly go wrong without any logical reason?

If you have the same computing environment exactly, then given an input X to a program will always produce the same result R. In practice, it is seldom to have a single program executing in isolation. The simplest application today runs in an operating system and shares memory with other programs that may be 'loaded' in memory at the same time. These programs may alter the memory in a way that makes a given program malfunctions. This is a famous problem with variables of type 'pointer' for example. Usually such errors cause abnormal system behaviors and not wrong calculation results.

In your case, I assume that the problem may be (and usually is) not what I have described above. The problem may be that:

  • the program used the wrong datatype(s) to calculate the result, that error only manifests itself when special values are used.
  • the program encountered an error in calculation (due to a logical condition) but did not handle the error and still produced the result. (e.g. mixing float and integer arithmetic)
  • a business rule or a logical condition was not coded correctly, the data that was entered represents this condition but the wrong calculation was used. (e.g. subtract amount from account amount before checking the amount in the account first).
  • using formulas that apply only to certain range of numbers but the data contains different range. (e.g. calculating an interest rate based on a range of values)

Because of the above and many other reasons software people spend so much resources in an attempt to create correct software, however, software errors still occur, but the errors are 'logical' and have a reason, it is just that the reason is not obvious to some without good research. So, in general tested software is predictable and does not produce random results. Due to the complexity of some programs, and other factors, even tested programs can go wrong but when that happens, errors are for a logical reason.

If I slam on the server machine, will one of the number which the computer is calculating, become another number and make the calculation wrong?

The answer is no in general, software is not fragile in that sense.

What you can do is isolate the cases where the error is occurring, find the similarity between these sets of data causing the error and find the difference between theses sets and the other sets that produce the correct result. You may be able to identify the specific set of values causing the problem. For example you may find that every time a variable has a negative value, the result is wrong.

Updated information about memory corruption errors: Please see Memory Corruption

share|improve this answer
    
was thinking about compound rounding errors myself as a source of such problems. They may not show up for a long time, until exactly the right (or wrong) combination of inputs leads to them all combined ending up with a result that's off from what it should be. –  jwenting Oct 4 '11 at 7:06
3  
Modern operating systems don't allow programs to modify (or even read) memory belonging to other programs. –  Péter Török Oct 4 '11 at 7:58
    
Yes, modern OSes do not allow anything of that nature. –  DeadMG Oct 4 '11 at 12:20
    
"f you have the same computing environment exactly, then given an input X to a program will always produce the same result R" I'm not sure this is true. What if one of the SR latches in the memory components gets two 1's because of some earlier corruption? en.wikipedia.org/wiki/… –  Yam Marcovic Oct 4 '11 at 14:01
    
@DeadMG and Péter Török thanks for your feedback, I have edited the message and added a reference to a page describing that the issue can still occur (I know as mentioned in the text, that it is highly improbable). –  Emmad Kareem Oct 4 '11 at 20:04

The first computer I owned was an Altair 8080 with 256 Bytes of memory. Input was from console switches and output was from a few blinking lights. If you disallow cosmic rays and hardware failures, I believe I could prove that some programs I ran on it would always produce the same results.

Since then, no.

share|improve this answer

In theory, if you start with identical state, the result will be identical. In reality, assuring identical initial state in "server sized" equipment is pretty much impossible.

Take uninitialized variables. Look at this code:

  short i;

  if(i==-1)
  {
        //do something special
  }
  else
  {
        i=0;
        //do something else
  }

This will produce unexpected results once in 65536 runs. And unless you assure memory will be in the same state before each run, i will be entirely random.

There are hundreds similar ways for errors to pop up following unpredictable elements of the initial state someone forgot to override, or border cases that happen rarely - race conditions in multi-threaded environment, out of bounds array access, disk IO on corrupted filesystem, and so on.

If you can prove the program is bug-free, there's only the cosmic rays that can break it. But mathematical proof of correctness of anything more complex than two nested loops is pretty much beyond scope of the biggest systems (and costs a small fortune), and for all the rest you can only hope.

share|improve this answer

Testing shows the presence, not the absence of bugs (Edsger W. Dijkstra)

If you're trying to prove that your program works correctly by testing, it won't work.

However, there some approaches in theoretical computer science where you develop a formal proof of the software you have written. Depending on the complexity of your system, this can be a tedious process though. If your system, however, works on a restricted set of commands, you could be successful with this approach.

share|improve this answer
    
Did you read the question? –  Winston Ewert Jan 8 '12 at 4:58
    
I did and I'm saying that he cannot use tests to guarantee that a program will never go wrong. That's what the title of his question says, right? –  Raku Jan 9 '12 at 8:23
    
Yes, the title seems to say that. The body clearly does not. –  Winston Ewert Jan 9 '12 at 15:07

Can you guarantee a program has no bugs and will never go wrong? No, unfortunately not.

Can you demonstrate that a program has a sufficiently small number of bugs that the cost of finding and fixing them far exceeds the benefit from that action? It sounds to me like you already have.

To paraphrase an old statistics maxim, all programs are wrong, but some programs are useful.

share|improve this answer
1  
+1 for "all programs are wrong, but some programs are useful" –  Michael Kjörling Oct 4 '11 at 11:56
    
I don't think this answer is actually relevant. It seems like he's asking whether a correct program might sometimes operate unexpectedly because of some environmental flaw. –  Yam Marcovic Oct 4 '11 at 14:03
    
My whole point is that no program is ever "correct". Everything is always a work in progress, and is only ever right until it's wrong. Computer science is a science after all. I do see what you're saying, and that may be more where the focus of his question is. However, I think that makes my answer all the more relevant, rather than less so. –  John N Oct 4 '11 at 14:51
    
@Hallainzil: I believe I have successfully written correct "Hello, World!" programs and the like. I've even written correct useful programs (although not big ones). –  David Thornley Oct 4 '11 at 20:32

I'm inclined to say no, you cannot prove that a program will never go wrong or provide an incorrect result, even if you can assume perfect input.

Raku mentioned formal proof of correctness. That's one thing to consider, but unless I'm completely mistaken, that will still have to assume a perfect execution environment. So, with some amount of time and effort, you can perhaps prove that the program is correct, but that does not necessarily prove that it will always produce the correct results, even given perfect input. The execution environment matters. And I would be wary of assuming that the input is always perfect, either.

This is why, in certain high-availability situations, multiple, completely independent implementations and execution environments are used, and the results are compared to make sure that they are within an acceptable margin of error from each other. In some situations, that margin may very well be zero. Even back in the 1960s, this was deemed important enough to include separate sets of computing hardware in spacecraft. Even if an errant static discharge, cosmic ray or whatever were to affect both computers simultaneously, the odds that both would be affected in the same way (particularly if they are both still working and producing valid-looking results) are miniscule. The odds of the same bug creeping into two completely separate implementations are also extremely small. And so on.

share|improve this answer

Hardware and software environments are in a constant state of flux. Moving parts, electricity, , temperature, dust, and OS code changes are examples.

Therefore I do not think it is even probable or expected that a computer software program will always behave the same since the environment is always changing.

Software can run for a long time as expected , but eventually either a small change to the host OS software will change which would effect the program in question or the hardware will value.

I am talking about current day computers.

share|improve this answer

Unless you can control every single bit in the machine, and every bit of electrical impulse flowing through the circuitry, then you cannot guarantee with absolute certainty that something will not go wrong with your program. Memory modules fail, CPUs can overheat and introduce errors, hard drives can scramble data, and power supplies can introduce noise into the system. The more expensive the hardware, and the more redundant the hardware, the less likely these things will occur, but at some point hardware can fail.

Then you have the operating system, with bugs that can be tickled by the most arcane means imaginable. Compilers may also have obscure bugs just waiting to deftly turn your pristine code into hard-to-trace bugs. It's a jungle out there, and your poor software is vulnerable to all of this. LOOK OUT!

And in my experience, more often than not, whenever there's a bug in a calculation, we never have to dig nearly that far to find the culprit. Generally speaking almost all bugs I've seen in the corporate world are easily found with the right debugging tools, and some elbow-grease.

In other words, while the hardware and OS might not be perfect, you'll likely never have to worry about that level of detail. Just find someone who knows the language, and is handy with a debugger, and dig in.

"simpler explanations are, other things being equal, generally better than more complex ones." -- Summarization of Occam's Razor.

share|improve this answer

Now my question is, will a computer program suddenly go wrong without any logical reason? If I slam on the server machine, will one of the number which the computer is calculating, become another number and make the calculation wrong?

The answer to that question is unknowable. It is impossible to prove that anything is always true of the universe we happen to live in. Instead we make assumptions and prove that if the assumptions hold, then some complicated property will also hold. This is what formally verified programs guarantee. Most programs are not formally verified though, they instead try to build confidence by providing tests. These tests give you assurance that provided the tests do what they were designed to do and that the assumptions that you've made hold, the program you are using will work at least some of the time.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.