How to handle Non Reproducible Software Bugs/issues

This task has been a challenging one for both developers and testers.  Once the tester observes an issue, he/she will try the same steps again to see the issue again, however  it might so happen that  the issue can’t be seen when tested second time or after subsequent trials. However tester logs the issue as non reproducible and tester is on safer side.  In order for non reproducible issues to be fixed, it requires a lot of time. The developer ends up spending time to reproduce the issue and to get the exact steps.  When I had a talk with a tester who logged a non reproducible issue, he said “I observed the issue once and the  exact steps are not known”.  After spending some more time reproducing it, he said “Out of 10 times, 3 times the issue was seen”.  These kind of responses will never help developers to debug the issue. If issue is there, definitely its there, the only problem is that we don't know the exact steps to capture. 

So, what did go wrong here? Why is the software behaving differently at different times? I think it does not make sense to keep wondering about these but to think several parameters involved while testing. There are certainly some things that we need to take care while testing. Basing on my experiences in dealing with non reproducible bugs/issues in a Software Product, I would like to share some good  practices that we including testing  team should consider before arriving at the decision that a particular bug/issue is not reproducible.

Some Guidelines dealing with non reproducible Issues

• If a certain bug/issue is non-reproducible on one system and the same is reproducible on another system, then the possibility of going wrong is in Test Bed or the test environment where the software is being tested.
• People can't capture the exact steps that they performed while testing because they may have some assumptions. Avoid all the assumptions while testing the software. Every parameter while testing the software matters such as Test bed, the operating system, what was the CPU and memory usage while testing the software etc
• Capture the logs, logs are very important inputs to developers to debug the issue and find out the possible cause for it. I believe every software will have logs enabled for debugging purposes, if not its highly recommended that you enable the logs in your software. Sometimes the logs help developers to find the exact root cause of the bug with no time.
• Think out of the box: I remember one issue was because the Windows Firewall was turned off while testing the software but if the firewall is turned the issue wasn't there. There could be lot of such parameters to consider that no one will usually think. So all the initial computer settings may also matter. So think out of the box. Test the software in multiple computers before concluding the bug is non reproducible.
• Keeping the test bed (the test environment) as one thing, if you are not confident of finding out the exact steps to perform to reproduce the issue, its highly recommended to use the desktop recording software. Start the desktop recording software before you start testing, this helps a lot. In fact it helped our team to find out the exact steps to reproduce the issue that was observed once. There are many free software available which can record your entire testing activities. The one we used was WebEx Recorder. Its normal tendency of human beings that they tend to forget to capture certain steps that they actually performed while testing. The recording software can be used to resolve these issues. Turn on recording software while testing all the time. You always have the option of deleting the recording if you do not need it. In case of non reproducible bugs or issues, you always have the option of going back and watching what are the steps you were performing when the bug was seen.
• It may so happen that the test bed being used itself is wrong while testing the software, so it does make sense to define the test bed first before even start testing the software.
•  Developer has to talk to the testing team to get more details of the issue. It may so happen that the testers will not capture each and every details of the issues that they raise though its not recommended.  
• Discuss sufficiently: Especially for non reproducible issues, we need to discuss a lot with Software product testing team or the testing team has to discuss a lot with developers. This might give some insights as to what is going wrong.

I have seen many people mentioning in the bugs/issue that they raise that the issue is 'Sometimes' observed. This is simply not acceptable. I remember if you see a bug being raised as 'Sometimes'  as frequency of the bug/issue I encourage you to simply reject the issue, do not even accept it, because its not going to lead us anywhere. Its expected that the person who is testing the software has to provide all the possible details.

Conclusion

It is true that if the so called non reproducible issues are not much analyzed and fixed, its certain that today or tomorrow the same bug/issue comes from our customers. Its also equally important to understand the software deployment scenarios. In-fact this should go as part of the software requirements. Its very much essential to understand in what conditions the customers are going to use the product. Testers must always think from this perspective and prepare their test bed according to the end user scenarios.
If you've faced such issues, you can share with me and we can learn from each other.