So what’s going on here? Freak coincidence? Supernatural powers? At a first glance, both sound ridiculous to me.

But I want to think about the more “interesting” explanation a bit: perhaps Paul really is psychic. Perhaps Paul can tell the future.

And then what? What does that really tell us? Does it answer our problem? Say we have a psychic octopus. It could show us anything. It could’ve predicted the financial crisis, it could have predicted the death of Michael Jackson, it could have given us next week’s lottery numbers. And yet this psychic octopus chose to show us the World Cup results of all things.

Why? Was there some significance behind this? Is God telling us that the World Cup is important? That we should forget about oil leaks and bank bailouts and climate change, and instead focus on who won the World Cup? That seems unlikely. Why would God, or anyone else, care about that? Or are we back where we started, that it’s just a freak coincidence that they chose to predict the World Cup, rather than the hundreds of other sporting events?

Does it mean that our lives are predetermined, that Spain had to win, that we have no free will? Or does it mean that Paul was able to control events? Perhaps Spain could have lost, but Paul exerted his mighty Psychic will and gave them the victory. (Is that cheating? Does it count as doping perhaps?)

How did Paul know the results? Reading people’s thoughts wouldn’t help, unless someone else also knew the results. Perhaps he can simply see into the future. But is he intelligent? Is he able to choose what to report back to us? Could he have chosen to instead arrange stones at the bottom of his aquarium into the shape of the next week’s lottery numbers? How did he know what it was we were asking him to predict? All Paul saw from us were two boxes with food in them, and a different flag on the top of each. How did Paul figure out “they must be asking about who will win the World Cup”? For that matter, how did he determine that we would interpret it as “the box I open indicates the winning team”? He could just as well have meant it was the team that was going to lose (he’s taking their food, after all)

The problem is that in order to explain anything, it is not enough to say that Paul has psychic abilities. We have to assume that Paul is an intelligent psychic octopus, that we live our lives along predetermined paths (unless Paul is able to control us) and that Paul has an interest in soccer specifically, and figures that of all the things he could reveal to us, the results of each World Cup match is what matters. And we’re still not able to say anything about how he manages to make these predictions.

Taken together, those assumptions makes the “supernatural” explanation sound at least as unlikely as calling it a pure statistical coincidence.

I’m generally a pretty skeptical person. I don’t believe in Gods, magic, ghosts, spirits or anything supernatural. Not because I’m sure none of it exists, but because it seems so absurdly unlikely for those specific beliefs to be true. Let’s say there really is something to the belief in ghosts. How do we know what it is? When you hear weird noises in an old house, how do you know it is the ghosts of dead people specifically? Perhaps instead the house itself is alive. Perhaps it’s the people who live in it in some alternate dimension? Perhaps it’s the Martians remote-controlling little dust clouds to mess with us. Believing in the supernatural isn’t an easy way to dodge the questions we can’t answer. Instead it just makes the problems bigger. Instead of explaining “Last night, I heard someone breathing even though I was home alone”, we now have to explain why somehow dead people are able to walk around here with us, and they’re able to make noises and for some reason they can think of nothing better to do than breathing heavily in my house. We have to explain how dead people come to be here (and that means we have to explain what happens to us when we die), and we have to explain how they can manipulate the world of the living. Somehow, they’re simultaneously insubstantial and invisible, and at the same time, able to make noises, or flick light switches or throw small objects around?

I think it was a lot simpler to explain back when it was just “I heard a weird breathing noise and I have no clue what it was”

Just like I’d rather have to come up with a plausible explanation for an octopus through random chance managing to predict World Cup matches, than having to do the same for the idea that the octopus can tell the future, is intelligent, cares about soccer, and cares about letting us know the result of soccer matches. Oh, and that the future is fixed and we can do nothing to change it.

Correctly guessing the outcome of 8 matches is pretty unlikely. Assuming it has a 50/50 percent chance of correctly guessing the outcome of each match (which sounds likely, given that octopuses probably don’t know much about soccer), the odds of this are 0.39%.

That’s low, very low, but not impossible. Statistically, one out of 256 octopuses should manage such a 8/8 success rate. Pretty lucky then that it happened to be Paul who got it right.

But that’s not quite right. Statistics don’t work like that. Paul made a number of predictions before he got famous, which were what brought him to our attention in the first place. The miracle here isn’t that he predicted 8 matches. Paul only really became famous when he predicted that Germany would beat England. If he can keep guessing correctly, we might start wondering if there’s something going on. But of course it’s going to look miraculous if we include past results. If you roll a die long enough, you’re going to get, say, four 6’es in a row. It’s bound to happen sooner or later. And once that happens, it’s hardly a miracle if you roll another 6. There’s a 16.6% chance that it’ll happen. We can’t include the first 4 rolls and say “I rolled five 6’es in a row! The odds of this happening are 0.013%,! It’s a miracle! This die is magical!”, because you cheated: you waited until you’d got the first four rolls right. The real coincidence is just that the final die roll came up a 6 as well.

So we only really started wondering about Paul’s predictive abilities after England’s defeat. Since then he’s made four predictions: the odds of that are much better. Even by a random coin toss, you’d have a 12.5% chance to get 4 matches right.

So now it’s no longer a supernatural phenomenon, but “just” a curious coincidence. But there’s another interesting thing to note: take a look at the results listed on Paul’s Wikipedia page. In particular, take a quick look at the flags.

Notice anything?

They don’t look very random. If you don’t look carefully, you won’t even notice that the flag in the “prediction” column varied in a few matches. They’re generally dominated by yellow and red. Nearly all the losers had flags with a lot of blue in them. Perhaps Paul just likes red and yellow better than blue. Serbia is really the only oddity then.

The trend is even more pronounced if we look at the results from 2008: Paul guessed that Germany would win every single match, regardless of the outcome of the actual match.

So perhaps Paul just likes the colors. Or perhaps he’s getting used to fact that whenever people show him a German flag, there’s food underneath it.

As a LaTeX user, I’d love to see this take off. Imagine that, not having to scour Google and random newsgroup archives and forum posts just to figure out why [X] isn’t working in your LaTeX document, or how to do [Y].

Imagine having a site based on the StackOverflow software, where knowledgeable users are actually going to see your question, and answer it, and even rate the answers so you know which ones are of the highest quality.

I wish we had a site like that when I was a student.

So go and commit to using the site:. The site won’t be launched until it has gathered a critical mass of supporters.

I graduated. My thesis defense went well and I’m no longer a student. Just thought I’d let you know

So what happens now? No clue, but I suppose it involves finding a job.

Of course, a few people assumed I was talking in absolutes, that because the practice of dogfooding is not perfect, it must be evil. That’s a bit of an exaggeration.

I never said that “dogfooding is bad”. But dogfooding generally means using your product on a daily basis. Not just poking around with it, or checking it out, but actually using it as your primary tool for… whatever the product is supposed to do for you.

This implies that you can’t really use a lot of other products as intensively while you’re dogfooding your own. A developer on the Visual Studio team can’t both use VS10 for all development, and at the same time use Eclipse, Vim and Emacs on a daily basis. If he were to use all those, there would no longer be enough time left to use VS10, and so he wouldn’t be dogfooding it.

And that is the problem. Dogfooding is a valuable way to incrementally improve and polish your product. But it also takes up time that could have been spent using someone else’s product. And when you use a product in your daily life, you become blind to a number of its weaknesses.

One of the Reddit commenters mentioned a lovely example: Java developers, people who write Java and nothing else, don’t really see the need for closures. They just don’t really consider it a valuable feature. But pretty much everyone else does. Does this mean that in Java, alone of all languages, closures are not relevant? C# programmers got closures and are ecstatic about them. C++ programmers are about to get closures, and everyone’s just itching for it to happen. Functional languages always had closures, and programmers in those languages just can’t live without them. Are we to believe that specifically in Java, closures just do not matter?

Or does it just mean that the Java developer doesn’t yet realize how much easier closures could make his job? He doesn’t realize this because he’s never had the chance to use them. He’s stuck in the world of Java as it looks today, and if you were to ask him what he’d like changed about the language, he’s not going to say “closures” or “higher order functions”, or even “templates” or “type inference”. He’s going to come up with some small incremental improvement. Wouldn’t it be nice if class X was added to the class library? Wouldn’t it be nice if Y was named differently? Wouldn’t some shorthand syntax for things you can do already be nice?

And the same is true for dogfooding. Visual Studio is slow. Many common operations cause multiple seconds of wait time. Adding an empty file to a project in VS2008 is painful sometimes. But oddly enough, I only really notice it when I’ve gotten used to Vim or Emacs or some other alternative IDE or code editor, which isn’t so slow. Its project structure is fundamentally broken, but I only really notice this when I’ve just spent a few weeks playing around with makefiles¹. And the same goes for the Windows Mobile phone I had a few years ago. While it worked, I tolerated its quirks. Apart from a few instances when it went really haywire, it wasn’t so bad. Sure, it was sluggish and needed to be rebooted regularly, and sure, the interface wasn’t exactly pretty or convenient to use. But it worked, most of the time, and I got used to it. I didn’t love it, and I didn’t think it was a particularly good phone, or platform or OS, but it was usable. Then the phone broke, and it took me a few weeks with my replacement phone to realize just how much better everything else was. My Windows Mobile phone sucked. I just didn’t realize it until I got to use a different one on a daily basis.

But let me reiterate, dogfooding is a good thing. There’s no doubt of that. Your product can only get better by having your developers actually use it in the same ways that end users are expected to. But if dogfooding is all you do, then the world will pass you by.

So why did I dig this old post up again?

Because Visual Studio 2010 is about to be released, and as we may have expected, this means Microsoft’s developers have to beat their drum a bit about how awesome it is because it’s been dogfooded.

Soma just seems to forget that the glaring performance issues in both beta 1 and 2 came as a huge surprise to Microsoft until the betas had been released in the wild. As much as they dogfooded it, it didn’t give them the information they needed: that VS10 is painfully slow compared to everything else, including VS9.

Both betas were released pretty much with the message “Don’t worry. It might use a lot of managed code and use a completely new WPF-based editor. But it runs really well and is just as fast as previous versions of Visual Studio”.

There were two reasons for this. One seems to be (according to another Microsoft developer’s blog post which I can’t seem to find at the moment, unfortunately) that they simply didn’t collect the right metrics from all their dogfooders. A lot of their developers did think it was painfully slow, but were never asked how they felt about the current performance level.

And the other is obviously that all their dogfooders were using VS10 in their day-to-day work. They had nothing else to compare it with. They weren’t using Eclipse or Vim or Emacs or even VS9 or VS6 in their daily work. So they became used to the downright painful performance.

But once they released the beta into the wild, it was obvious to everyone else, all those people who had not been using VS10 on a daily basis for months, that it was a huge step backwards.

So yes Microsoft, you’ve heavily dogfooded VS10. No doubt about that. But let’s not pretend that it solved all your problems, or that it gave you the best possible product. In reality, it led to you scrambling the last 6 months to bring the performance back up to where it should have been all along, and where you’d probably have kept it if you’d used other IDE’s occasionally so that you’d had a basis for comparing performance.

It’s not that dogfooding is bad. It’s just not enough. And it is not, in itself, a selling point or a proof of quality.

On monday the 12th of April, I’m going to defend my master’s thesis. If you’re in the area, and are geeky enough to find it interesting, feel free to drop by.

The precise place and time is: 15:00, April 12, 2010

Room S125 / 3–1-25 DIKU (Datalogisk Institut) Universitetsparken 1 København Ø

Looks like I’m going to be busy the next couple of days preparing my presentation.

That is all.

Here is what I’ve previously written about my thesis on the blog, here is what Wikipedia has to say on the subject, and here is the thesis itself, including source code.

I’ve been meaning to write this post pretty much for the last month. The reason I’m finally doing it is that I also wanted to drop a quick line on a cute aprils fool joke that should be of interest to a lot of gamers:

Rock, Paper, Shotgun dedicated the entire day to perfectly ordinary PC game reporting/blogging as if it’d been April 1st, 1993. Cute and intelligent, and served as a fun trip down memory lane. Nice idea, and a nice change from the usual fare of everyone trying to pull off outrageous or absurd stories ad nauseam. Especially as there seemed to be practically no worthwhile pranks to be found anywhere this year (even Google had some pretty tame ones), your up to the minute coverage of PC gaming news as of 17 years ago really made the day.

Many people are in love with the Singleton pattern. Others — a small minority, I suspect — consider it a mistake, an anti-pattern, or something that was only ever included in the Design Patterns book as a lifeline to procedural programmers who couldn’t really figure out this OOP thing.

I won’t pretend to be half as clever as all the people who have already written about the problems with singletons years ago, and I don’t think I have anything new to bring to the table. But it is a pattern I learned to loathe very soon after I first saw it in use. (Singletons do sound attractive when you first hear of them. But they pale a bit when you end up having to tear up and rewrite half your code just because all your singleton classes start revealing their shortcomings) And for a long time now, I’ve tried to convince other programmers that Singletons have some serious problems. Recently, it seems like I’ve even gotten noticed for it on StackOverflow.

First, GMan posts an answer to one question, and I comment with a mild disagreement, and the discussion goes on for a few more comments. As singleton rants go, this one is pretty mild, and I don’t really think about it any further. Then, a few weeks later, I discover his blog and this post. Wow! A convert. A person I know to be extremely bright and a knowledgeable programmer has changed his mind in response to something I said… I’m flattered.

And today, I noticed another question being posted, which had both Boost and Singletons in the title — how could I resist? Two subjects I enjoy talking about, even if the things I say about them are very different. Surprisingly, the comments there already mentioned me, and some of my earlier answers regarding singletons. Should I be flattered that people have started bringing my name up when discussing Singletons?

Anyway, one of the comments also suggested I write a blog post describing my argument in detail. So I will.

Two wrongs don’t make a right

There are a lot of problems with singletons. In fact, it’s surprising that so many people still consider the pattern useful, when it is afflicted with so many weaknesses and flaws. However, for now I will single out the two that I feel are the most fundamental: not just problems with how a singleton works, but with what they’re trying to achieve:

A singleton, as defined by the Gang of Four, combines two properties:

• it guarantees that exactly one instance of an object exists. While that one instance is typically created lazily, so it doesn’t technically exist throughout the entire application’s lifetime, it always seems to the programmer as if precisely one instance exists, and
• it guarantees global access to this one instance.

Let’s pick those apart a bit. The last one is easiest: it is, by now, fairly common knowledge that global state is bad. We don’t like global variables, we don’t like static class members, we don’t like anything that makes it harder to isolate bits of our code. Dependence on global state causes a lot of problems: it hurts parallelism, as access to global mutable state generally has to be serialized through the use of locks. It makes dependencies harder to detect and control (any function might silently decide to access our singleton. The function signature says nothing about this, so we have to read the source code of the function to determine if this is the case. And because it is so convenient to always just add a reference to a singleton, we tend to do it a lot. When you have a singleton, you quickly end up in a situation where three out of four classes depend on it. How did that happen? Why, logically speaking, do so many classes need direct access to the database? Or the renderer? Is that good design? Not only is this messy, it’s also painfully hard to fix after the fact. Once we have these dependencies on global objects everywhere, that’s a lot of code we need to change to eliminate the global. Almost every class will be impacted by the change, and a huge number of functions have to have their signatures modified to take that extra parameter replacing the global. Or even worse, the function has to be completely rewritten to eliminate the need for whatever service the singleton provided. The more globals you have in your project, the more your dependency graph starts resembling spaghetti. And the harder it gets to clean it up.

It hurts reusability, as code taken from one project and inserted into another may break because it depended on globals not present in the new project. It hurts testability partly for the same reason, a unit test testing a class must suddenly provide a number of globals as well just for the code under test to compile, but also because global state makes tests less deterministic. One test might change the state of this global, affecting the outcome of the next test to run.

Globals are bad for a lot of reasons. They have their uses, no doubt about that, but we should be suspicious whenever the solution to a problem involves global data. It might be the best solution, but often, it is more trouble than it’s worth.

The other point is more subtle. Why do I object to a class enforcing that “only one instance may exist”? It’s really just common sense. As the Agile movement tells us, we don’t really know what our code is going to look like tomorrow. Over the course of development, we have to adapt to changes, modify our code, revise decisions already made. Why put roadblocks in front of us? Why make it harder to adapt to unforeseen changes or requirements?

Today, I might think that I need only one logger instance. But what if I realize tomorrow that I need two? That’s not so far fetched. We may have one log we write ad-hoc messages intended for debugging purposes, solely to be read by developers, and another formalized log, where structured messages are written when predetermined events occur, so that the application can be monitored in production. Sure, we could define the two as completely separate classes, and then we’d only need one instance of each (but then we’d start duplicating code). Or we could use the same log instance to write to both logs (but then the logging code would become more complex, having to interleave two separate and non-overlapping logs.

Once we’ve accepted that an application may need more than one logger, shouldn’t we do ourselves the favor of ensuring that our loggers can be instantiated more than once, just in case it turns out to be the right thing to do? We’re not even adding any complexity, there’s no cost associated with this. On the contrary, we’re removing significant complexity. Thread-safe singletons are surprisingly hard to get right. Dependencies between singletons are tricky and circular ones can cause them to blow up in all sorts of fun ways. And let’s not even get into how to handle anything our singletons might do while the application is shutting down. What if the database singleton tries to write a simple “goodbye” log message to the log singleton? What if the log singleton got destroyed before the database one? Ouch.

Singletons are hard to write and hard to use. Removing them only simplifies our code, so if it also enables us to better adapt to unforeseen requirements, why shouldn’t we remove them?

Not convinced? Let’s think of some other examples then:

• the application configuration should be a singleton, right? We obviously can’t have more than one of those! Wrong. We can. We often do. Think about what happens when the user opens the “Options” screen and modifies the settings. During that time, two sets of settings exist: the “applied” settings that are currently in effect, and the “speculative” ones, currently being picked out by the user. Once he clicks OK, the speculative changes should be applied, replacing the ones that were previously in effect. But until then, we have two sets of settings to maintain.
• a database connection pool then! If we have more than one pool of connections, we can’t efficiently share them! Correct, but perhaps we don’t want to share them. Perhaps I want to ensure that library A has one pool of 10 connections available to it, component B has a smaller pool of 3 connections, an components C, D and E use the global pool with however many connections it supplies. That would ensure that no matter the number of threads running in component B, it’ll never starve out other components trying to access the database. It can never hold more than three connections, leaving room for other components. Of course, in the common case, we do want all connections to be shared in one single pool. But perhaps not always. So yes, there should probably be a globally accessible default pool available. But why shouldn’t it also be possible to define new local pools if the user deems it necessary? Why limit ourselves to one instance?

And even if you do come up with some case where we absolutely must never have more than one instance, where it would make the sky come crashing down on us, consider testing. Consider that each of your unit tests should set up the environment it needs, and run within that environment, in isolation from other tests. That means that every test should create its own logger instance, or database pool instance, or whatever else our singletons are doing, just so it can avoid being polluted by stateful changes made by earlier tests. Each unit test for the Direct3D renderer should set up its own renderer object. Each physics simulation test should initialize the physics engine first, and shut it down again after use. Singletons don’t easily allow that. Sure, we can extend them with explicit Create() and Destroy() methods, but then our abstraction is starting to get leaky. We can no longer assume that precisely one instance exists, because we might have just destroyed the one that existed.

The “exactly one instance” guarantee removes flexibility from our code that we may need, in order to enforce a constraint that we definitely don’t need. Where’s the harm in allowing the user to create more than one instance if he decides to?

C++ programmers are familiar with std::cout, the standard output stream. Funny thing about this, it is a simple global object. We can obviously never have more than one standard output stream. But we can have more than one stream. The standard library just initializes one of them to point to the standard output, and saves it as a global variable. We don’t need it to be a singleton, we don’t even need it to be a static class specially defined for the purpose. We just need a stream, defined somewhere where it’s globally accessible.

True, a sufficiently stupid programmer could create a new stream when he intended to write to std::cout, and true, a singleton implementation would have prevented that. But is it worth it? When was the last time you saw someone accidentally invoke std::ostream() << "Hello world";, when they intended to write std::cout << "Hello world";? It’s not the most common typo I’ve seen.

We don’t need to prevent multiple instantiations. If we want only one instance, we just instantiate the class once, and refer to that instance whenever we need it, end of story. We don’t need the compiler to slap us over the wrists if we do create multiple instances, because we never do so by mistake. If we do it, it’s because we have a reason. It’s because our initial assumption that only one instance was needed, turned out to be wrong!

So there you have it. A singleton combines two negative qualities. It takes the “you can never create a second instance of this class” constraint, which hardly ever makes sense, and even when it does, does not typically need to be enforced by the compiler, and combines it with a global object, giving us all the downsides of both!

Two wrongs don’t make a right. Not even if they were described as a good idea by some guys 15 years ago. They’re still no greater than the sum of their parts: two wrongs. One bad thing combined with another bad thing, creating a very bad thing.

Too many programmers rely heavily on singletons to solve a problem they never had. They never needed a compile-time guarantee that multiple instances of a class can never be created. They just needed one instance to be created.

Sometimes, we do need globals, yes. In those cases, make old-fashioned globals. Use static class members, or if the language allows it, global (non-member) objects. Or use the Monostate pattern, or whatever you feel is the cleanest solution. But remember that the problem you’re trying to solve is “enabling global access to this data”. No more, no less. You do not want a solution which sneaks completely unrelated constraints and limitations in through the back door.

And while I can’t personally think of many cases where this is true, you might also run into situations where it is truly necessary to prevent more than one instance of a class from ever existing. Again, I can’t think of what situation this might be, but I won’t rule out that it can occur. If it does, then enforce that constraint alone. But don’t go around providing global access to the object as well. Whatever specialized purpose your “one instance only” class serves, it’s highly unlikely that everyone should be allowed access to it. So don’t make it a global.

Most of the time, your classes should have neither of these attributes. Sometimes, rarely, they may need one of them. But the singleton pattern imbues the class with both properties, and that is just a plain bad idea.

The result is that a lot of people now have sensitive personal information floating around in public. An example of this (found via ArsTechnica) is this woman, who starts her post like this:

I use my private Gmail account to email my boyfriend and my mother.

There’s a BIG drop-off between them and my other “most frequent” contacts.

You know who my third most frequent contact is?

My abusive ex-husband.

Which is why it’s SO EXCITING, Google, that you AUTOMATICALLY allowed all my most frequent contacts access to my Reader, including all the comments I’ve made on Reader items, usually shared with my boyfriend, who I had NO REASON to hide my current location or workplace from, and never did.

Ouch.

Others, with less at stake personally, are also pissed:

See, I love the idea of neat new tech innovations that lead to streamlined communication, real-time updating, in-line video and photo posting, and supersimple friend and contact integration. I do not, however, like a product that bursts through my door like a tornado and opts me in to wanton in-box clutter and spam (or, more precisely, bacn) publicly reveals my personal contact list without asking me, threatens to broadcast my e-mail address anytime someone wants to @ me in a Buzz, and even appears to grab photos off my Android phone that I’ve never uploaded.

or this one

So…yeah, I guess I’m on Google Buzz. It’s linked to my Picasa and WordPress accounts, so you can follow everything I do. Cause that’s not creepy or anything. The best part is that the defaults for everything are public, and you end up broadcasting to a bunch of random people unless you sit down and sort through. I’m expecting this to backfire for a bunch of people, and not just eventually but almost immediately. It might not be a bad idea to start a betting pool on when the first child porn charges are filed as some highschool student accidentally sends herself to the entire school.

I could go on, but I really don’t want this to turn into some kind of link farm.

I’m not personally affected by this. I do have a GMail account, and yes, they opted me in to Buzz, but the account contains no personal information whatsoever, and no personal emails. I use it exclusively as a dumping ground for spam, and form mails I don’t want cluttering up my real email inbox. I’ve never even sent an email from the account.

I use the Google search engine, but I am not signed in to it, and have never created a profile or a customized homepage on it. I’m sure they could still identify me just by examining cookies or my IP address, but at least they’d have to work for it. And it’s not like my Google searches are state secrets anyway. As long as people are not able to search for my name and bring up a list of everything I’ve searched for, I’m satisfied.

I also use Google Analytics for this blog. I feel OK about that because this blog is already my public face on the internet. Google already knows a lot about it simply by indexing it for their search engine. I have no problem with them generating statistics on where my visitors come from, as long as they make the information available to me too. The only sensitive information associated with this blog is my login password, and I’m pretty sure Google doesn’t have that. And they’re not getting it, even if they launched a GPassword service tomorrow.

I use the WordPress software, but not hosted on WordPress.com. I don’t use Picasa or Google Reader. I don’t use Google Documents.

So all in all, yes, Google certainly knows a lot of fragments of information about me. Google searches can turn up quite a bit, they can collect a few more bits and pieces through cookies when I use their search engine, and they have a lot of statistics on who reads my blog. But they can’t read my emails. They don’t have any really sensitive information about me. Nothing related to my work, personal life or studies is tied to Google.

And this brings us to the point of this post:

Don’t blame Buzz, blame GMail

A lot of people are furious at Google for the mixture of incompetence and indifference towards users’ privacy with which Buzz was launched, and while that might be justified, it is missing a fundamental point.

Buzz is just doing what Google does best, what they’ve always done, and what they should be doing. Here’s what Google’s own website has to say on the company’s mission:

Google’s mission: to organize the world’s information and make it universally accessible and useful

Google is dedicated to making information universally accessible. For a lot of information, that’s a good thing. Their search engine turned the internet upside down — for the first time ever, users were able to actually find the information they needed. Google is good at this, and we’ve benefited hugely from it.

And social networking is right up Google’s alley as well: Social networking is all about making information about you and me accessible to the world in an organized manner. A lot of Facebook’s popularity relies on their ability to analyze our existing relations, friendships and networks, and use this to suggest new friends. My Twitter would be useless if I couldn’t follow the people I wanted to keep up with, and if others couldn’t find my tweets through searches. Buzz is simply more of the same, and there is nothing wrong with that. It’s another social networking service, and Google is exactly the right company to do something like this. No one is better at organizing information and telling us exactly what we want to know.

The problem is that another of their services is not so well suited for the company. Email is something almost everyone considers personal and private. Even the US government, in its desperate war on people who wear turbans, speak funny and pray to Allah, has only given itself permission to sniff the subject lines of people’s mails sent over GMail. This is considered the equivalent of reading the envelope, without opening it and looking at the letter inside. Because that letter is personal. And so are the bodies of our emails.

But if we consider our emails to be sensitive personal information, then why do so many people entrust them to a company whose stated mission is “to make the world’s information universally accessible”?

A company like that should never be entrusted with our sensitive information.

Facebook has made some major blunders regarding privacy, but their mission seems to be something like “can’t we just all get along”. In Facebook’s perfect world, everyone are friends with everyone else. This doesn’t excuse their privacy issues, but at least it tells us that they’re not directly opposed to the idea of privacy. They’re just clumsy and don’t think things through.

Google, however, is different. In the perfect Google world, privacy does not exist. In Google’s dream world, I could go take a look at Bill Gates’ emails or Steve Jobs’ search history. or Bono’s shopping list. It is information. It should be made available to the world.

So no, there’s nothing wrong with Google Buzz. It should absolutely broadcast everything Google knows about us to the world. The problem is that Google has been given sensitive information in the first place. Google shouldn’t know anything about us that can’t safely be published through Buzz. If GMail had never existed, Google would not know that the woman in the first example has received emails from her abusive ex-husband, and so they couldn’t have caused her any problems. The only things Buzz would have known about us would be what we told it.

Imagine if Twitter or Facebook had been built by Google, based on their search engine and their ability to categorize and organize information. That is what Buzz could potentially become, and that’d be nothing short of amazing. At least as long as we all take care to keep our emails and other sensitive information far away from Google.

Don’t opt out of Buzz because of privacy concerns. Opt out of GMail instead. Expect every new service Google launches to do as Buzz. Their mission is to make all information available to the world, and they’re going to keep trying. You’re fighting a losing battle. You can keep opting out of their services till the cows come home. It’s always a temporary solution at best. Instead, fix the root issue: Make sure Google is not given any sensitive information about you in the first place.

But perhaps there’s a counter-argument that people seem to miss. If you use a lousy piece of software on a daily basis, you get used to it. You stop thinking about how it should be, and only consider how it is.

I think the first place I heard of the term “dogfooding” was on the Windows Mobile team blog. And let’s be honest, is Windows Mobile really a competitive product? Is it worth using? Perhaps in a vacuum. If all you know is Windows Mobile, then, well, it’s not too bad. But there’s an obvious reason why the product is struggling in the marketplace. Compared to everything else, it feels horrible to use.

Perhaps the recipe for fixing Windows Mobile would be less dogfooding. Windows Mobile developers shouldn’t be forced to use their own buggy, slow, in-development OS all the time on their phones. Perhaps they should be given iPhones and Blackberries. Perhaps some of them should even be given simple old-school non-smartphones. The ones that didn’t need to be rebooted, and didn’t “feature” load times for opening your contacts list, or to write a new SMS (text message). Perhaps they need to be shaken up a bit, and see what else a phone can feel like when you use it. Windows Mobile 6.5 might be better than WM6.0. But that’s not the competition they need to beat. They need to beat the iPhone, they need to beat Android, Blackberry and Symbian. So those are the products they should use at least as much as they use Windows Mobile.

The same may be true for Visual Studio. It’s great that the team uses Visual Studio 2010 internally as much as possible during development. But that also means that they get used to its performance issues. And it means they get used to the assumption that “this is what an IDE is like”.

Perhaps Visual Studio would be a better product if the team was forced to use Emacs, Vim and Eclipse. Or perhaps even Notepad and makefiles.

And how much better would TFS be, if the developers had used Git or Bazaar instead of dogfooding TFS during development?

Dogfooding has its advantages, certainly, but I don’t think it alone is a recipe for a good, competitive product. It leads to an incremental improvement over the previous version of your product, but it doesn’t take into account what else is happening in the world. It doesn’t give you the opportunity to question your basic assumptions¹. Sometimes, incremental improvement is not what your product needs.

Just a thought.

If I had to pinpoint one thing that marked the difference between a skilled and an unskilled C++ programmer, it would be “do they understand RAII”. Many people don’t, hence this post.

RAII is, apart from being badly named, one of those deceptively simple concepts that you think you understand when you first hear of it, think “well duh, that’s obvious”, and then proceed to write code as usual, because you just don’t see how widely applicable it is.

But let’s get the name out of the way first. RAII stands for “Resource Acquisition Is Initialization”. And if you’re not already familiar with the idiom, then this has told you nothing at all. If you did know about RAII in advance, then you can, when you stop and think about it, kind of see how the name relates to it… vaguely… sort of.

What it actually means is simple: Resources should be managed by classes. When the class is initialized, the resource is acquired (hence the name). When the class is destroyed, the resource is released. And the lifetime of the object should exactly match the desired lifetime of the resource. That sounds obvious, and many programmers will (assuming they’re working in a language that has classes), say that this is what they always do.

Often, C++ developers think this just means “smart pointers. Wrap your memory allocation in a boost::shared_ptr and you’re done”. I see that as one not-very-often used border case though, rather than a typical example of RAII. So let’s take a step back instead.

The key idea isthat any kind of resource, not just memory, but file handles, sockets, database connections, or even more abstract resources like loggers or profiling timers or textures, really any concept or process which has a lifetime, should be mapped to an object.

Unlike the typical object-oriented line of thought which goes that “everything must be an object, because then.… well, everything will be an object, and your code will be better”, here we actually have a concrete reason: We want to use the object to manage the lifetime of the resource.

When I allocate memory with new, I have to deallocate it again sooner or later, with delete. (Or in C, with malloc() and free() respectively). And I have to make sure that this is done. And I have to make sure that it is not done twice. And that the object is not accessed after this is done. There are a lot of constraints we have to obey, all related to the lifetime of the resource. And this is why unmanaged programs have a reputation of leaking memory left and right. If we allocate memory, and it is to be used by a dynamic number of objects or functions all referencing the same allocations, which of the users is responsible for deleting it? And how do we know when it is safe to delete, when no users remain?

Ironically, most managed languages have not solved the problem. They have added a garbage collector (which yes, is very useful for a wide number of reasons), but that only solves one specific instance of the problem. It takes care of avoiding memory leaks, but it doesn’t avoid resource leaks in general.

The garbage collector ensures that this code won’t leak memory:

void foo() {
  SomeObject* obj = new SomeObject();
  bar(obj);
}

where without a garbage collector, we’d (at least without RAII) have to write code such as

void foo() {
  SomeObject* obj = new SomeObject();
  try {
    bar(obj);
    delete obj;
  }
  catch(...){ delete obj; }
}

In the garbage collected case, we don’t know what bar does, and we don’t need to know. It doesn’t have to delete the object. And neither does the foo function. So we have successfully dodged the problem of managing the lifetime of memory allocations. We haven’t really solved the problem though. We still don’t have any good tools to manage the lifetime. We’re just guaranteed by the system that it’ll last long enough.

In C++, this effect can be approximated using some kind of smart pointer¹.

Smart pointers allow us to write code like this:

void foo() {
  boost::shared_ptr<SomeObject> ptr = new SomeObject();
  bar(ptr);
}

and be sure we won’t leak memory. Of course, this solution isn’t perfect — reference counting is much more expensive than a good garbage collector, and if we create cyclic references, the objects will never be deleted, as the reference counts never reach zero. It is a decent approximation, but nowhere near as good and reliable as the garbage collector in managed languages.

But the problem shows up again if we use another type of resource. What if we’d opened a database connection instead? We’d have to write code such as this: (The following Java-like pseudocode is copied almost verbatim from this StackOverflow.com answer, courtesy of Martin York.)

void writeToDb()
{
  Db db = new Db("DBDesciptionString");
  try
  {
    // Use the db object.
  }
  finally
  {
    db.close();
  }
}

(And of course it gets even worse if db.close() can throw exceptions. Then we have to catch that exception, just to avoid it propagating out from the finally clause if we reached finally because of an exception being thrown in the try clause.)

The resource management problem still exists. We still have to wrap the code in exception handling just to make sure that the connection is closed as soon as we’re done with it. And we have to do this at every use. And it gets complicated fast.

Of course, .NET makes this a bit simpler:

using (Db db = new Db("DbDescriptionString"))
{
  // use the database object.
}

But the onus is still on the user of the class to ensure it is closed correctly. There is no obvious way to encode into the Db class that “once we’re done with an object of this type, the connection must be closed immediately”.

And in C++, smart pointers are no longer suitable solutions, since the resource to be managed is no longer a pointer allocated with new.

Instead, a more basic flavor of RAII comes to the fore:

void someFunc()
{
    Db db("DBDesciptionString");
    // Use the db object.
} 

Yes, that’s all. When the db object goes out of scope, at the end of the function, its destructor is called. The destructor internally calls this->Close() for us, so we don’t need to do it! We just have to trust the scoping rules of C++, which guarantee that destructors are called on local variables when they go out of scope, and on class members when the class is destroyed.

So in a sense, the key idea in RAII is simply that “resources should behave sensibly”. They should get copied safely if an assignment is made (or otherwise, assignments should be prevented), they should be available if their owning object is successfully created (if it can’t create the resource, it should throw an exception, aborting the creation of the object), and when they are no longer used, they should be cleaned up.

The C++ standard library class template std::vector is a wonderful example of RAII in action. The resources being managed by a vector are memory (the array allocated internally to hold the objects being contained in the vector, as well as the objects themselves. When the vector is destroyed, every object it holds must be destroyed too, and the array in which they were placed must be deallocated.

In the following examples, assume that a function foo is passed a vector of MyClass objects by value. We don’t know how many, if any, objects are stored in it, but since we are passed a copy of the original vector, we take ownership of it. It exists only in the function foo, and must be destroyed afterwards.

void foo(std::vector<MyClass> vec) {
  ...
 //  when we get to the end of the function, all local variables, including vec, 
 // are automatically destroyed by having their destructors invoked.
 // So no matter how many MyClass objects were stored in the vector, it ensures that they too have their destructors called.
 // And the vector also deallocates its internal array, leaving neither of its resources alive at the end of the function
}

void foo(std::vector<MyClass> vec) {
  throw std::exception("Oops");
  // as above, vec is automatically destroyed when we leave the function,
  // regardless of *how* we leave it. Even if we leave it because an exception was thrown and not caught.
} 

void foo(std::vector<MyClass> vec) {
  // other is constructed as a copy of vec. std::vector ensures that both of vecs resources are copied as well
  std::vector<MyClass> other = vec;
  // we now have two vectors, each owning a dynamically allocated array and a number of MyClass objects
  // and again, at the end of the function, both are deallocated cleanly
} 

void foo(std::vector<MyClass> vec) {
  std::vector<MyClass> other; // a second, empty, vector

  // perform an assignment, setting vec to be an empty vector
  // std::vector makes sure that if you do this, the resources previously held by vec are cleanly released
  // before copies are made of the resources held by other
  vec = other;

  // and so when the function ends, the MyClass objects originally held by vec
  // have already been destroyed, so their destructors are *not* invoked now
} 

As the above shows, vec owns its resources, and manages them tightly. Whenever a change happens to vec, it reflects this by updating its owned resources. If it is destroyed, it destroys its owned resources. If it is copied, it copies the resources it owns. If it is assigned to hold something else, it first destroys its existing resources. And so on. Nothing you do can bring it “out of balance”. It just works. That is RAII. Smart pointers are just convenient adapters turning raw pointers into RAII objects. But RAII is much more than smart pointers.

It is the broad and general idea that resources should be mapped to objects, so that the object can not be created unless it succeeded in acquiring its resource, and it can not be destroyed without also releasing its resource. This effectively saves C++ programmers from having to worry about resource management.

Take an example that’s guaranteed to cause pain without the use of RAII: Handling exceptions being through halfway through constructors. Say you have a class with multiple members which are initialized in its constructor. After the first member has been initialized, but before all of them have been initialized, an exception is thrown. Let’s use the following contrived example:

class Foobar {
  Foo f;
  Bar b;
  MyClass c;

public:
  Foobar() : f(42), b("hello world), c('a') {}
};

unfortunately, b’s constructor throws an exception. How to handle this? We know that in C++, partially constructed objects do not automatically have their destructors called. when the construction is aborted.

And since we want to avoid any resource leaks, we require that the following must happen: – a must have its destructor called (because a was successfully initialized before the error occurrd) – b must release any resources it acquired in its constructor before it threw the exception – c must do nothing. Its construction was not yet begun when the error ocurred, so it would be an error to attempt any kind of cleanup of c. – The Foobar object (the object pointed to by the this pointer) must ensure that the above, and nothing else, happens, and it must do so without relying on its own destructor (which won’t be called, as construction did not successfully complete).

And of course, pretending that only b can throw an exception may be a simplification over the real world. Perhaps every member could throw one from its constructor. Care to write a Foobar constructor which takes all this into account, providing enough try/catch blocks to correctly catch every exception that might be thrown, and release exactly the resources that have been allocated until then, and nothing else? A tall order, and an open invitation for bugs. And of course, it’d lead to a huge, bloated and error-prone constructor. It’d also prevent us from using the initializer list. We’d have to perform some kind of “safe” non-throwing default construction of both a, b and c before entering the constructor body, where exception handling is possible, and from there, attempt to perform assignments to bring the three members into the desired state.

In pseudocode, the constructor might look something like this:

Foobar() {
  a = new Foo(42);
  try {
    b = new Bar("hello world");
  }
  catch {
    destroy a;
    throw;
  }
 try {
    c = new MyClass();
  }
  catch {
    destroy b;
    destroy a;
    throw;
  }
}

Note that all this complexity is only necessary because we want to handle several different resources. a, b and c all contain resources that must be attempted acquired, and properly released if this fails. If there’d been only one resource, the job would have been much simpler. There wouldn’t be any point at which some resources have been acquired, and others have not. If we succeeded in acquiring that one resource, there’d be no risk of errors occurring afterwards, so we wouldn’t need complex conditional cleanup code. And if we failed to acquire the one resource, there’d be nothing to clean up — after all, the resource was never acquired!

So to keep down the complexity, the only safe way to define a class is to make it own at most one resource. And this one-to-one mapping of resources to classes is exactly what RAII is all about. If a, b and c had all been RAII objects, then the above code would work. Regardless of which members could or couldn’t throw exceptions. According to the rules of C++, we know that in the above case,

• the Foobar destructor (this->Foobar::~Foobar() will not be called, as *this was not successfully constructed.
• the a destructor will be called, as this member was fully constructed at the time of the error.
• the b and c destructors will not be called, as these members were not fully constructed at the time of the error.

So assuming that b’s constructor takes care of releasing any resources successfully allocated when the error occurred (the number of which, as pointed out above, should ideally be zero), we’re actually home free! What happens is exactly what we listed earlier as our goal. a has its destructor called, c’s constructor was never run in the first place, so it doesn’t have to do anything, and *this doesn’t have to do anything special in its constructor. All of its members take care of their own resources, so the number of resources managed by *this is zero!

We don’t even need to write a destructor for Foobar now, if all its members are RAII objects. Whether the Foobar object is partially or fully constructed, its members take care of themselves. That is the power of RAII. Once a resource has been mapped to a class, we can use it as much as we like, and even in very complex situations, and never have to worry about the resource being leaked. It is managed by its wrapping RAII object, and the C++ lifetime and scope rules ensure that this wrapper object gets destroyed when it goes out of scope