Whidbey was a long road, which came directly after the long road that was VS2003/.NET 1.1, which came directly after the longer road that was VS7.0/.NET 1.0. Three products in 7 years (that's not very many). And while running Windows Forms definitely had it's share of upsides, it came with a fair amout of overhead too (see this post for some details). One of the tough things about Windows Forms is that not only did you have to be compatible with your old versions, but you had to be compatible with behaviors of existing systems, either for technical reasons, correctness, or just because users expected it. Taken all together, it made for very slow progress.
As the end of Whidbey neared, I decided it was time for me to take on a new challenge. .NET Client (the official Windows Forms team name) was great and had given me amazing opportunities for growth, but I was starting to get interested in other technologies and approaches to solving some of the problems we addressed in .NET Client.
I also saw some things in our software development process that I thought could be improved. In a nutshell, there are a bunch of really important things that need to happen before a product goes out the door that traditional development teams have trouble doing effectively, and it impacts software quality.
We've all had this happen. You get a new piece of software, you start using it and you find something wrong. You say to yourself (or to the monitor, usually not politely), "what the?!? how in the world could they have missed that?!?" When that's happened, it's because of one of a few possibiliites:
- The bug got introduced late and they decided the risk of fixing it was too high
- Even if it seems simple, you're doing something they didn't think of
- You're in the no-man's-land where two features written by two different groups of people interact and there's a gap there
- The software is not up to par quality wise and you're likely to find a lot of these
The first one is unfortunate, but it happens, and it's usually not fixed because it's not a blocking issue so you can work around it. The last one is definitely a problem, it also happens, and that's a much bigger conversation. But the middle two are addressible and occur in all large software projects (especially development tools), Microsoft's included.
Why does this happen? Because horizontal testing is hard. There is no systematic way to combat these things. You can do all the automation and unit testing you want but these are specific tests focused on a vertical chunk of the software. There are other things that these tests don't address well. You need to do ad hoc and/or scenario based testing. Which, in development tools, means writing applications.
Now, one of the first things you learn if you are running a team building a framework is that there basically is no common scenario. Because APIs can be put together in infinite ways, and it's very difficult to really constrain that and still have a useful framework, the very first thing customers tend to do is something you never thought of. Or if you did, your next thought was "who would do that?". You can shoot for 80% cases and known scenarios but that still leaves a lot of ground uncovered.
So you can have scenarios, and run through those and make sure everything works right. It's a start but the scenarios tend to have a similar problem: they are one well-trodden path through the product. So you can verify, over-and-over again that some scenario works well, and is a nice, safe, well lit path through the product. What happens if you step off that path? Hmmm, well, it should work. Right?
So you're back to ad hoc app building. And I'll just come out and say it: development teams suck at this. They suck at it for a variety of reasons. First, it's hard to have a hard exit criteria around it because it's subjective by nature. With FxCop or stress numbers or code-coverage I have a bar that I either hit or didn't hit. Not so with app building. There's an old saying in business that the "hard drives out the soft", meaning things with hard numbers behind them tend to trump things without. So when push comes to shove and teams get squeezed for time, app building tends to get cut short or cut altogether. And even when it happens, it isn't deep enough because real applications require a bunch of other stuff that a given set of people may not be set up for: getting a DB up and running, or some other kind of service, or a bunch of machines for web services, etc. Basicially a bunch of stuff that isn't really testing what you want to be testing. So it's hard, unmeasurable, and sometimes haphazard.
You can never fully solve this problem but there is lots and lots of room to make it better. So I started thinking you could start a team to do this sort of thing, but if you did it right it would solve these problems as a side effect rather than a direct goal, which is something I've always liked. Basically the new team takes new technologies here at Microsoft, uses them to build cool applications, and drives feedback and issues back into the originating product teams. And, if possible, you do all of this as transparently as possible which gives the community a chance to see the new product in action and have a working "best-practices" sample very early on. Think Quick Starts meets Starter Kits meets demoware.
A big re-org happened shortly after we shipped Whidbey and I started reporting to Scott Guthrie. I kind of off-handedly mentioned this to him, and he really liked the idea. So off I go...
So what am I working on exactly? Well it's cool, and we're shooting on releasing an early version of it next week at Mix 06 in Vegas. Want details? Well you won't find them here. I'm separating my personal blog (this one) from my work-related stuff, moving all of that to http://blogs.msdn.com/sburke/, check there for more info.