This sort of feature is done with many Microsoft products, and usually is called "customer experience improvement program". The important things are to tell customers what it does and to give them an option to turn it off. Give system administrators a way to disable (or enable) it via group policy. Here are 2 example web pages that describe the MS one:
One blog that explained a lot of what was going on was the one by Jensen Harris, but with the blog revamps at MS has become almost unreadable:
I think he doesn't work at MS anymore as no attempt to contact him over a period of 2 years was successful.
It took me more than a year to find out, but folks who work at MS claim that implementing a feature set like the MS Customer Experience Improvement Program takes 2 teams of about 30 people per team: one per desktop application and one team to write and maintain the server-side collection application.
At my last employer, contact with actual customers had to go through the PHB, and trying to decide what features our customers really wanted or really used, all were filtered through the big guy. Like Mr Harris mentions in his older posts, some of the most requested "new features" were already in Office (or in our case, the existing applications being shipped), yet the customers were unable to find them through the maze of menu options. Since our customers were the opposite of "early adopters" we had plenty of time before trying to move to the Office 2007 look (existing apps looked more like Office 2003). The PHB wanted to know what size of effort it would take to get this data to make a move to the ribbon actually useful for our customers (we shipped one small application with the ribbon/fluent interface but we never really understood what the point of it was for about a year after it first shipped). I abandoned my attempts to implement such a thing when I finally got an idea of how big the MS teams were (namely, each team was many times larger than our entire developer staff).