Testing thread safety

Question

Say you have a function f that is or contains a critical section. How would you unit test that only one thread can run it at once, that it doesn't have race conditions, and that it doesn't cause a deadlock, etc...?

Answer 1

You can't. Testing can only prove that you didn't find any bugs, not that none exist, and in the presence of multi-threaded code, I would say that it's almost pointless. You cannot test for how your function behaves when the OS changes it out or pauses execution or any of a thousand external events which can affect when the thread is scheduled and it's behaviour.

You need to, at least informally, prove the function correct.

Answer 2

You cannot do a "strong" test, but you can mitigate some risk by doing a "weak" test.

Parameterize f to take some additional code if passed in. Then in the test, pass code for a thread counter, a sleep and an assertion that the thread counter never got above 1. When running the test I would use 5-10 threads. I would also restructure a function without a critical section and make sure the thread counter got above 1. The idea would be to make sure the serialization was a result of the critical section only.

Answer 3

Add sleeps around operations within f and add a parameter to let you control their length. You can then unit test as normal, but with different scheduling patterns.

Answer 4

TDD is useless in multi-threaded scenarios. You don't test thread safety, you prove it, or it counter-proves you, painfully...