A follow-up rant about Connect

So my last blog post got a sur­pris­ing amount of atten­tion, not least from a/the (no clue how many there are of those) Prod­uct Man­ager of Visual Stu­dio, which is pretty neat.

So, here’s a quick fol­lowup, in order to fully exploit my 15 min­utes of fame!

First, I highly rec­om­mend any new­com­ers read Doug Turnure’s com­ment on my pre­vi­ous post. It is an inter­est­ing read, and seems sur­pris­ingly frank about the prob­lems with Con­nect as it is today. And where I would nor­mally have expected some canned PR speech about how all the per­ceived prob­lems aren’t actu­ally prob­lems, or can’t rea­son­ably be fixed, it actu­ally states that the prob­lems are real, and that they will be fixed. Thanks, Doug!

Of course, words are cheap, and it remains to be seen whether any of this will actu­ally result in tan­gi­ble improve­ments to Con­nect. But know­ing that some­one within Microsoft is actu­ally work­ing on it is a lot bet­ter than nothing.

But being the cyn­i­cal and demand­ing bas­tard that I am, I worry that this is only brush­ing the sur­face. Because the prob­lem is not just that bugs get for­got­ten on Con­nect. Let’s try to sep­a­rate the issues a bit:

Bugs are forgotten

As we saw, bugs are left to lan­guish until they are men­tioned on Twit­ter, imply­ing that the way to get a bug fixed is not to bother with Con­nect, but instead to make a lot of noise about it on social media. Doug answered this quite clearly, say­ing that bugs cur­rently “fall through the cracks”, which can, should and will be pre­vented in the future. That’s great, but it doesn’t solve the sec­ond part of the problem:

Even when bugs don’t fall off the radar, the way they’re han­dled is still broken

Bugs seem to be treated com­pletely ran­domly once they’re entered into Con­nect. Even if they don’t “fall through the cracks”, they are still treated to a line of over-zealous mod­er­a­tors or what­ever they are, who seem to be com­pet­ing on who can close the most issues the fastest. Bugs are closed hap­haz­ardly, some­times as a dupli­cate of a bug reported on an inter­nal sys­tem that cus­tomers can’t see, some­times as “won’t fix”, and some­times as “won’t fix”, but with a com­ment say­ing that it may get fixed for a future release (which means it’s not quite “won’t fix” after all).

Some­times the same sce­nario will result in a “deferred” sta­tus (which makes a lot more sense than “won’t fix”), and some­times the bug will be closed as “by design”. Some­times they’re Resolved as won’t fix, and other times they’re closed as won’t fix. What’s the dif­fer­ence? Are sta­tuses assigned com­pletely at ran­dom, or does it just look that way?

Why are issues closed when the team doesn’t have time to fix them in the upcom­ing release? They’re still bugs, and until a deci­sion is made to never fix the bug, it is hard to see why they should not be con­sid­ered as open, or unre­solved, at least from the users’ point of view (what sta­tus it is given inter­nally is of course up to Microsoft. But when cus­tomers see a bug closed as “won’t fix” merely because “we don’t have time to fix this in time for the next release”, they’re going to see red, and for good reason.)

Is any or all of this a con­se­quence of the messy inter­nal sys­tems that Doug described, or is it that new bugs are processed and cat­a­loged badly (whether due to the respon­si­ble staff receiv­ing insuf­fi­cient train­ing, the poli­cies they’re sup­posed to fol­low being unclear, or the tools they use being, well, bro­ken)? I don’t know, but some­thing is clearly wrong. And it is not just that bugs fall through the cracks and are forgotten.

So the real prob­lems go deeper…

What needs to be done

Once the “super­fi­cial” prob­lems with Con­nect are fixed, the next step is to make sure bugs reports are actu­ally han­dled cor­rectly: that bugs aren’t closed with “won’t fix”, unless that is actu­ally what the respon­si­ble team intends; that dupli­cate bug reports are closed either with the same rea­son, or marked as dupli­cates, and that in gen­eral, report­ing a bug gives the user the impres­sion that “the bug report isn’t par­tic­i­pat­ing in some ran­dom lot­tery, where a ran­dom per­son will even­tu­ally close it with a ran­dom reason”.

But even then, we’re not quite home free. Because even when a bug is “fixed”, there’s a sig­nif­i­cant problem:

It hardly ever is!

When a bug is marked as “fixed“on Con­nect, it usu­ally means “we have fixed this bug inter­nally, in our trunk, and at some unspec­i­fied point in the future, we will release some unspec­i­fied prod­uct built from this source tree”.

In other words, the bug has not been fixed in the prod­uct where it was reported. The bug may have been reported in Visual Stu­dio 2010, but it was “fixed” in Visual Stu­dio 2012 (or whatever).

That’s not what “fixed” means, is it? A bug is fixed when it is no longer present in the prod­uct where it was reported. In other words, if a bug was reported against VS2010, then I’d con­sider it “fixed” when the fol­low­ing con­di­tions are true:

  • the bug has been fixed in the VS2010 source tree, and
  • the fix is avail­able to cus­tomers who own VS2010, or at the very least, has been sched­uled for avail­abil­ity to users of VS2010.

Of course, some bugs are so insignif­i­cant that it’s not worth push­ing an update out to users of the cur­rent prod­uct, and some­times a fix would be so com­plex that it can’t be rea­son­ably imple­mented in the prod­uct. Of course there are cases where it makes sense to imple­ment the fix in the trunk of your source tree, and not in the prod­uct ver­sion where it was reported. But at least give it a dif­fer­ent sta­tus than “fixed” then.

When a bug is “fixed” in Con­nect today, the mes­sage to the per­son who reported the bug basi­cally boils down to this:

We have iden­ti­fied the prob­lem, and if you buy a dif­fer­ent prod­uct, priced at sev­eral thou­sand dol­lars1, to be released in a year or two, you won’t expe­ri­ence it any more”.

Gee, really? So where does Con­nect enter into it? I’m pretty sure I can do that today with­out ever sub­mit­ting the bug report to Con­nect. I could just buy a dif­fer­ent OS or IDE. I hear a lot of peo­ple are pretty happy with XCode, for example.

Of course, exac­er­bat­ing this prob­lem, Visual Stu­dio upgrades tend to be non­triv­ial for large code bases. The for­mat of solu­tion and project files is gen­er­ally incom­pat­i­ble between ver­sions (even when the only actual change to the for­mat is an incre­mented ver­sion num­ber), so not only does the user have to wait for a com­pletely sep­a­rate prod­uct to be released, and then buy it at the usual price, they also have to go through a poten­tially painful upgrade process… Just to see the fix to a bug they reported two years ago.

Even if all the other prob­lems are fixed, this one still ensures that report­ing bugs on Con­nect will feel like a waste of time. If we’re prac­ti­cally guar­an­teed that the prod­uct we reported the bug against won’t ever be fixed, why bother report­ing the bug at all?

So it’s not just Connect

So this plays into another of my pet peeves about Visual Stu­dio: I have no clue whose deci­sion this is, and if it is actu­ally an active deci­sion made by some­one who thinks it is a good idea, or if it is sim­ply a fos­silized holdover from the mid-80’s that no one have thought to change, but the pol­icy of never ever improv­ing on a released prod­uct has got to go. If I report a bug on VS2010 and you claim to have fixed it, I want the fix. If you don’t actu­ally fix VS2010, then you haven’t fixed the bug that I reported. And if you’re not going to fix the bugs I report, where’s my incen­tive for report­ing them?

Of course, once we accept that bugs should actu­ally be fixed, when prac­ti­cally pos­si­ble, it is worth ask­ing why the same shouldn’t apply to new fea­tures and other improvements.

If you have a bet­ter com­piler than the one I’m using, it really seems to be a win-win sit­u­a­tion if you let me use it. I get to use a bet­ter com­piler, I get more value out of the prod­uct I bought, and I am more inclined to buy VS vNext as well, if I know that it too will actu­ally improve over time. Forc­ing me to use an, on aver­age, 18 months old com­piler doesn’t seem like good sales tactics.

Of course, not all VS users are affected equally by this. When I worked with .NET, this both­ered me less, because the VS com­piler set the pace: the lan­guage got updated when the com­piler said so, and there were no alter­na­tive imple­men­ta­tions to speak of. Who cares if your com­piler is 2 years old? The lan­guage evolves in lock­step with the com­piler any­way. Sure, I’d still have loved to see improve­ments to the IDE itself, but I knew there would be lit­tle point in updat­ing the com­piler out of band.

But now that I work with C++ every day, it dri­ves me nuts. The VC++ team is (like any other C++ com­piler team) per­pet­u­ally play­ing catch-up to an inter­na­tional stan­dard, and to a half-dozen com­pet­ing com­pil­ers. When VC10 was released, it was on par with the com­pe­ti­tion for about 2 months. Sure, it lacked plenty of fea­tures that other com­pil­ers sup­ported, but it also sup­ported some that weren’t yet imple­mented in other com­pil­ers. Then, a cou­ple of months later, a new GCC ver­sion was released which sup­ported the few fea­tures that had pre­vi­ously been unique to MSVC. And after that, we’re wit­ness­ing a 2 – 3 year period of utter stag­na­tion, where the VC++ com­piler stands com­pletely still, while all around it, the com­pe­ti­tion is mak­ing huge strides towards com­pli­ance with the new C++ stan­dard, and mak­ing new fea­tures avail­able by the dozen.

How is that a good idea? How does this make Visual Stu­dio look good? Why would I pay thou­sands of dol­lars for a prod­uct that is obso­lete 95% of its lifetime?

And the prod­uct doesn’t even need to be obso­lete at all. The VC++ team is busy imple­ment­ing count­less improve­ments to the IDE and the com­piler. But they’re being “hoarded”, wait­ing for the next major release, rather than being shipped out to your pay­ing cus­tomers. It is as if some­one at the top of Microsoft’s Dev­Div con­sid­ers soft­ware to be like fine wine; as if new fea­tures should be given a few years in a cold cel­lar to “mature” before being sold to customers.

Does any­one really think that this makes VS a more attrac­tive product?

Con­nect is cur­rent bro­ken, sure, but the prob­lem goes deeper than a few “for­got­ten” bugs. Once that is fixed, there seems to be a lot of work to do in mak­ing Con­nect appear worth­while to users (by giv­ing bet­ter, more con­sis­tent, feed­back on received bugs, and prob­a­bly also through gen­eral site improve­ments, fix­ing the UX dis­as­ter that it is today), and even once those are solved, end-users are unlikely to notice much actual improve­ment, unless some changes are made to how the Visual Stu­dio team in gen­eral deal with bugs, fea­tures and prod­uct updates.

At the end of the day, until we, the end-users actu­ally get to ben­e­fit from the improve­ments made as a result of of our bug reports, the sys­tem will remain broken.

You can make Con­nect as respon­sive, user-friendly and all-round won­der­ful to use as you like — but if we, the end-users, don’t get to see fixes to the bugs we report, the sys­tem is still broken.

Of course, my rant­ing about this is unlikely to change much. I’m hardly in a posi­tion to tell Microsoft how to run their busi­ness. But on the other hand, there can be no harm in point­ing out what I see as prob­lems. So here we go. :)

  1. yes, most peo­ple and com­pa­nies are able to get pretty major dis­counts on that, but not all, and it’s the prin­ci­ple of it that both­ers me. 

Share and Enjoy: These icons link to social book­mark­ing sites where read­ers can share and dis­cover new web pages.
  • Digg
  • del.icio.us
  • StumbleUpon
  • Reddit
  • Technorati

Tags: , , ,

One Response to A follow-up rant about Connect

  1. Ben Voigt says:

    chalk up this VC++ MVP as 100% agreement

Leave a Reply

Name and Email Address are required fields. Your email will not be published or shared with third parties.