Shawn Burke's Blog

Just a quick update on how we're dealing with taking bugs straight from customers via MSDN Feedback. We definitely were concerned about the impact this was going to have, time wise. We had to do some adjusting and hire a contractor to help us work through the bugs. At the very beginning we were getting a couple hundred issues a week. Just the time it takes to sort through those, make sure they are "well-formed" bug reports, not duplicates of existing bugs, etc. was at least a full time person. So for a while we had one of our testers doing that but it made a lot more sense to bring someone else in for it. So that's been much better, by the time the bugs make it to the product team's plates, they're pretty well scrubbed, have good repros, etc.

Once we got that nailed down, it's been a good experience. The most interesting statistic is that we have fixed about 30% of the bugs submitted by customers. That might not sound like much but that's the number of issues that we hadn't already fixed or found on our own, and some percentage of those would have likely gone out the door in the final product. Remember, by the time the community gets a new drop, it's about 3-5 weeks older than what we're currently working on. The other 70% were duplicates of existing issues (we'd already found it), did not repro (e.g. were probably already fixed), or were suggestions (some good, some, uhhhhh, unique) or misunderstandings about behavior (by design). Now we're getting about 50-75 reports a week and that will of course increase once Beta 2 ships to a wider audience.

We try pretty hard to put good comments into the issues so people can understand why we made the decision we did with a given issue. The bucketing is the best we can do too -- some suggestions are really kind of bugs, and some bugs are really mostly suggestions. Just to put some number behind it for Windows Forms, as of 12/27.

And how we've gone about resolving those issues: