I've actually been looking into doing something similar. As a good starting point, I found an open source project called mahout which implements most of the algorithms you need, although it is far from a plug and play solution.
The three use cases you might be interested in are clustering, recommendation, and classification. They all basically group items into related topics, but in subtly different ways.
- Use recommendation when you have a bunch of news articles and are trying to determine which ones you are most likely to like based on your past reading habits and those of readers similar to you.
- Use classification when you want to group the news articles into subjects, and you know beforehand what those subjects should be. This is most useful for long term, ongoing topics, like the weather, for example.
- Use clustering when you want to group the news articles into basic subjects, and you don't know beforehand what those subjects should be. This is most useful for one-time events, like the death of Hugo Chavez, for example.
If you're looking for a more complete solution, check out Carrot2. It is only able to handle around 1,000 documents, though. Perhaps useful if you are only interested in clustering one day's worth of news from a few select rss feeds.