A software development company I work with has a web based product they are successfully selling to clients. All of these clients have a support agreement with us that generally requires us to access their servers, databases and application data. As the number of clients grows, (and the number of different people supporting those clients grows), we are seeing a need to better manage in particular passwords, but more generally all the information one would need to fully support client systems ie. URLs, application login details, IP addresses, any client-specific weird stuff etc etc. This data is changing on an increasingly rapid basis, as systems get moved around to different machines, IP addresses. The worst problem is superuser passwords for the applications. These are constantly changing and occasionally we find ourselves in a situation where nobody seems to know who made the most recent change.
This is clearly an unsatisfactory situation. There are procedures in place which say that password changes etc need to be documented in a central secure place. But of course, people being humans, sometimes forget to document these things. The danger is that when people get so fed up of password chaos, that you can end up with a situation where everyone's password is "password123" as they can't be bothered any more.
Apart from enforcing the procedures, what other approaches could we take? One suggestion I heard was to have a "hidden" application superuser whose password is easily remembered/derived, which only support personnel would know about it. Personally I think that solution is a scary security hole, especially when you have multiple clients using your software.
So how would you handle this situation if you were supporting multiple clients? More strict procedure enforcement, or a technical approach to make life easier for everyone?