In fact, I'd probably have three repositories, one for the Client and corresponding client-only libraries, one for the Server (and corresponding libraries), and one for the shared libraries (incorporating the API interfaces that expose the functionality between the two, plus any other shared code). I think that's really the key, the shared code should go into a separate repository of its own. That way you can make sure that the interoperability between your client and server is both always at the same version AND is isolated from the design of each of its consumers.
Obviously, this is not always possible, depending on the particular communication framework you're using, but there's likely to be shared code that dictates the format of the data-transfer objects or the handshake steps in your custom protocol (or some other example).
Assuming you have a fairly decent Continuous Integration and QA setup (a fairly large assumption, in my experience, but one I'm going to make nonetheless. If you don't have a QA department you should at least get some CI) you shouldn't need to use the single-repo pattern as a defense against possible code mis-matches, either your CI server will flag up library-interoperability or your QA team will catch runtime errors (or, even better, your Unit Tests will).
The benefits of split repositories lie in the ability to separately version separate parts of a system. Want to take a copy of last week's Server and run it with this week's Client, to try and lock down the root of a performance issue? No worries.