You may use the version control system to keep tracking of code ownership. I'm not sure if you can do this with all VCSes or not, but for the git/Gerrit pair, you can put the name and contact details of the code owner in the description field. For other systems you can consider creating a text file (let's call it OWNER.txt) within the module that provides the same information; so that, if the developer can checkout the code, she already has the contact details of the module owner. If you create a template, you can also script it, so that you get an up to date list available whenever you need it (even create a web page that displays the code owners).
Developers leaving the project happens all the time and I don't see a direct link to this problem, but with my proposed solution, you can simply change the module description (or the OWNER.txt file for that matter).