Being a self-taught hacker type, I've always had this struggle myself.
First and foremost: Figure out what branch of mathematics you need to learn. To do that, figure out what you want to do. Mathematics is an insanely large topic so it's best to focus your efforts.
For example, game programming math usage will differ from stock analysis, which will also differ from writing graphics such as Photoshop, Gimp, etc.
Game programming itself can differ slightly depending on what branch of game programming you want to specialize in (i.e. rendering, AI, etc).
My suggestion would be to brush up on your late high school math (geometry, trigonometry) and then become decently proficient with linear algebra (a subset of anyway). A lot of game programming professional (again, depending on what you focus on) don't employ a wide range of linear algebra topics on a daily basis. As long as you understand basic concepts such as 3d coordinate systems and vector operations (dot product, cross product, etc) in order to do things like moving actors around in 3d space, camera rotations, etc. you'll be well on your way. Note that this doesn't mean that you're guaranteed to get a job as a game engineer, but you'll at least understand some of the concepts that are employed in the code.
However, if you want to get a broader range of knowledge (which it sounds like you might be wanting to do), I might just start getting as much literature as I possibly can:
I'd also suggest taking a look at http://www.khanacademy.org/. They have a ton of short, easily digestible lecture videos on a wide range of mathematics as well as quizzes where you can test yourself on your new-found knowledge.
If you're not a total math nerd, don't give up. It can be particularly trying. Use resources such as math.SE if you find yourself stuck. If you have a friend who's a math geek, use them.
Also, understanding symbols used in math literature is pretty much a necessity, or you'll be hitting sections that really feel like you're looking at something written in Greek (unless you're Greek, then it's in another language ;)): http://en.wikipedia.org/wiki/List_of_mathematical_symbols (cheat sheet).
Good luck. You're in for fun if you enjoy challenges.
Edit: For core software engineering concepts, you might also want to check out http://www.udacity.com/.