So you've made some code changes that should hopefully speed up some part of an application. But there is just one problem - you don't know how it will perform in live.
Different networks, different machines, different memory, different configuration, different times of day, different users, different server loads etc etc etc
(a) Do you ask them if that bit of the application has got faster?
(b) Do you ask them to monitor that part of the application?
(c) Do you just ask them to see if they notice any difference in the application in general? After all, you can't be 100% sure you haven't affected something else.
Each of these approaches would seem to have problems (I know this as I've tried them all):
(a) You're asking a leading question. Before such a fix goes life, there is an emperor's new clothes scenario. The developer believes the code should be faster so there is some kind of crusade to get everyone believing it before it goes live. So...it goes live. Some people say it is faster, some slightly slower and others about the same. What then?
(b) Again: Some people say it is faster, some slightly slower and others about the same. What then?
(c) You're no longer asking a leading question, but you start getting hit with a whole torrent of issues that people have known about for months (maybe years) but you've now opened Pandora's box. How can you be sure which if these (if any) are new issues?
Or maybe I should be asking different questions entirely. Any thoughts?
No doubt there are processes leading up to this point that could potentially be improved. But the nub of the question is this:
The change is tested and has been put live. You have a myriad of stats to prove the changes should improve performance. Your boss has given the stats a once over but now wants to make sure the users are happy before signing off the release. What question(s) should you be asking the users?