Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

We are developing an application; it includes a library developed by another coder, this library communicates with server via multiple network connections, and this involves multiple threads working together. The server side code is quite complicated, and we don't have access to the source code.

Recently I've discovered a mandelbug making application crash sometimes. I could reproduce it once and got a stack trace, so I opened a bug report. The bug itself is easy to fix (uncaught web exception in one of background threads, that makes CLR terminate the program).

The problem is that developer is refusing to fix the bug, because "he is not convinced it exists". Unfortunately for me the boss is siding with him and says this bug cannot be fixed unless I make a "solid test case" to prove existence of the bug, and to make unit test verifying that it's gone. What is basically impossible due to a nature of the bug.

Any advice?

share|improve this question
12  
I'd say it is pretty simple. Create a unit test that proves what you're saying is true. –  cspray May 30 '13 at 5:25
1  
Have you saved the stacktrace in some form? E.g. do you have a screenshot of your IDE showing the stacktrace of the crash? –  Giorgio May 30 '13 at 6:15
7  
@fithu: you are a little bit too convinced that reproducing such kind of bug is impossible - it may be hard, but seldom "impossible". And how can you know that the bug is "easy to fix" when you don't have access to the source code? Just catching an exception might not really fix the problem. Or are you talking about library code you have access to, and you already pinpointed the exact line where the bug occurs? If so, why don't you suggest a fix in code? –  Doc Brown May 30 '13 at 6:23
2  
@fithu: your original title was some kind of rant against your boss. I changed it in hope it prevents the soon closing of your question, rants are not very popular on this site. If the new title don't reflect your question correctly, feel free to improve it further. –  Doc Brown May 30 '13 at 6:34
4  
@Giorgio: a stack trace is a proof that a program can crash at a specific line, it does not proof that this line is the root cause of the bug. That seem to be the fact the OP seem to have misunderstood, and the cause why I had problems to understand some question details. –  Doc Brown May 30 '13 at 8:04

5 Answers 5

up vote 35 down vote accepted

If possible, may be spend some time to check if this defect can be reproduced by putting some sleep or block in your application code. But do not spend too much time. As this issue is due to multi-theading (and also as you observed), it's occurrence will be rare.

My advice is not to sweat over this too much. Continue your work. Whenever you come across this crash, update your bug report with the stack trace, saying that this is repeat occurrence, and changing the owner to library developer. Let the management/lead decide whether to fix or not depending on it's frequency.

Also try to understand the developer's mentality. You said "uncaught web exception". The developer at this stage may not be entirely sure what will be other effects of catching this. So he/she may be is reluctant to touch the code.

share|improve this answer

So, from your more or less clarifying comments, I got it this way:

You are sure there is only a simple additional exception handling missing, and you know already which code line in the lib is problematic and how the lib could be fixed.

Why then don't you just add the few missing lines of code to the lib by yourself, ask the team kindly to test the lib with that changes? Make sure it is a low-risk change, easy to understand by the dev who is responsible for the lib. The worst thing that could happen is that someone has to revert that change in your VCS if your fix causes some new unexpected behaviour.

Most people are easier to convince when the work is already done. Also, they react better on "here is an improved solution", opposed to "this code is wrong, fix it somehow".

EDIT: when the dev still refuses to add that change, best option is trying to make the problematic code work inside an isolated test harness where you simulate the network error. Working effectively with legacy code describes a lot of techniques on how to tackle such kind of problems. For example, you could create a test version of the library, including only the problematic modules and functions, and create a "mock environment" around it where you can simulate the "network exception" under controlled conditions. That may seem to be too much effort at first glance, but once you have such an environment, you can add a lot of additional tests to it (and I guess that will make sense, since when the author of the lib refuses to add missing exception handling in one place, one should expect more bugs of that kind).

share|improve this answer
    
He refuses to merge this change, because "it's not necessary" –  fithu May 30 '13 at 8:10
    
@fithu: see my edit. –  Doc Brown May 30 '13 at 8:20
4  
@DocBrown +1 for They (people) react better on "here is an improved solution", opposed to "this code is wrong, fix it somehow". –  laika May 30 '13 at 10:12
2  
@fithu: So come up with a test case that triggers the unhandled exception to be thrown. I.e. find out paramaters that trigger it. –  wirrbel May 30 '13 at 10:17

For a bug such as this, automated fuzz testing (also called random testing) might be helpful in trying to reproduce it. This automates the process of finding the bug by randomizing a fixed set of parameters (or inputs) into the thing you're testing. Each test run, parameters are recorded to a log file, including time stamps, etc. so that when the crash occurs you can (theoretically) just replay back the test, using the same parameters, to reproduce it.

Since its automated, the test process can run many tests in a short period of time. Often it can be left to run overnight, and in the morning you can check a log file to see if it reproduced the crash.

share|improve this answer
3  
"just replay back the test, using the same parameters, to reproduce it" - not really possible for threading/networking issues. But I like the idea. –  fithu May 30 '13 at 7:08

The Devil's Advocate suggests another path.

The other developer has stated, flatly, that there is no bug there.

Can you come up with a way to stress the hell out of his allegedly-nonexistent bug, and cause it to crash a lot more frequently?

share|improve this answer

The stack trace is clear evidence the bug exists, or at least did exist in a certain build. What you don't have is evidence the bug was fixed. They are foolish to ignore it. I've had "impossible to reproduce" bugs after hundreds of thousands of automated tries on multiple systems that triggered every single time on a customer's system.

I get a couple bugs like that per year, most without even the benefit of a stack trace. In nearly every case, even though I couldn't reproduce it beforehand, I could pretty easily make an automated test for it once it was fixed.

For example, a few months ago I fixed a bug that only occurred when the user typed faster than 96 words per minute. Before I fixed it, all I knew was that the bug happened "sometimes." It would never occur to me to write a unit test for fast typing. However, after I knew the root cause, making a test for it was trivial.

Even in those rare cases where a bug can't be reproduced even after being fixed, you can close it by code inspection.

share|improve this answer
    
how do you do an automated test for stuff like that? (to avoid misunderstanding, everything else you wrote matches my own experience and beliefs) My most recent bug like that was data race for unsync'd concurrent access, both the bug and fix were very easy to prove by code inspection but I can't imagine how to reliably auto-test that. (I mostly have little problems designing tests for concurrent stuff but can't figure test code to prove absence of data race) –  gnat May 30 '13 at 21:20
1  
That might fall into my code inspection exception, but you can also trigger race conditions by introducing a delay in one of the threads. Often you can accomplish this by delaying an external stimulus, or less ideally, you can put the delay directly into the code during testing. –  Karl Bielefeldt May 30 '13 at 21:28
    
I see, thanks. Sounds interesting, I need to give it some thought... –  gnat May 30 '13 at 21:32

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.