I'm looking into Lab Management right now for our test environment for manual and automated testing. I'm just getting started on the process of putting together various servers to host/support it and then I'll be making some dummy projects and working through the process.
Currently, we are using System Center Virtual Machine Manager (SCVMM) to create VMs and we build systems on them. Our workflow for manual testing is more-or-less like this:
- Build VMs with supported platform (OS, Pre-req software, etc.)
- Install latest build, or any build noted in a bug that we're trying to repro
- Snapshot the VM
- Test or Repro
- Snapshot when bug is seen
- Create bug in TFS, gets assigned to Dev
- Dev usually asks to see the VM, they login and do what they need
- While the Dev is working on a fix we write automation to perform the repro (using internal tools - this is a bit of a weak point) or move on to a different bug retest or something
- Doc anything important that the team might need to know in the future in our wiki
- When we get a fix, we basically return to the snapshot created in step 5 and run the automation (if it was completed - sometimes they can take time that isn't available)
I was wondering if anyone else has used lab management or is currently using it in a production environment. I would like to ask those people:
- What tangible benefits have you seen from using lab management? What's easier? What's harder? (compared to what you were doing before)
- Considering what we're using now, where can you see weak points being that lab management will help significantly?
- How does writing automation compare to any other methods you've used?
- Do the bugs created using lab management provide everything you need? The marketing for lab management shows lots of info auto-generated and being able to hop into a snapshot seems to help, but I can't see how you would be able to easily step through anything in a debugger without having pulled down the code base and having a dev environment setup on the VM (not likely in our case, codebase is quite large).