In short: SVN is one repo and lots of clients, while GIT is a repo with lots of client repos, each with a user. Git is a decentralized to a point where people can track their own edits locally without having to push things to an external server.
SVN is designed to be more central where GIT is based on each user having their own GIT repo and those repos push changes back up into a central one. For that reason, GIT gives individuals better local version control.
GIT is really great on OS systems like Unix, BSD, Linux, Solaris and their derivatives. There is a little GUI that ships with GIT that works under a POSIX layer, but it's hardly as intuitive or powerful as TortoiseSVN. Thus, Git has very powerful command line that you would need to get used too.
In addition, Git is simply more versatile and allows offline development in a way subversion never really could. Its available on almost every platform imaginable and has more features than you'll probably ever use.
If you're looking on getting out of SVN, you might want to evaluate Bazaar for a bit. It's one of the next generation of version control systems that have this distributed element. It isn't POSIX dependant like git so there are native Windows builds and it has some powerful open source brands backing it.
But you might not even need these sorts of features yet. Have a look at the features, advantages and disadvantages of the distributed VCSes. If you need more than SVN offers then consider one. If you don't, you might want to stick with SVN's (currently) superior desktop integration.