As an addition to the other answers:
While a DVCS fits this problem very well, you could technically also use Subversion, if you feel more comfortable with it. Subversion can use a local directory instead of a central server. You could just put that onto a thumb drive, and use it.
The disadvantage, compared to a DVCS, would be that you can only work with Subversion (i.e. commit, view logs etc.) while the thumb drive is plugged in. Also, it must always be the same thumb drive (or at least an up-to-date copy), because with Subversion you should not use more than one repository (that's the non-distributed part). So if you ever forget your thumb drive, you can't use it, unlike with Git or Mercurial.
As explained above, and in the comments, a DVCS is really a better fit for your problem. I only mentioned Subversion for completeness sake, and in case you have some special reason to use Subversion.