Response to a Jolt Award critic

This started out as a comment on someone else's blog, but "just growed".

Original anti-Jolt rant here. You'll want to go there first or this will make no sense.

My response:

Mr. McGovern,

I wanted to wait until the judging process was over for the year before responding to your blog post. I had to laugh at the general tone of your post, especially the George Bush graphic. If only we judges had a fraction of the mystery and power you seem to ascribe to "they" (i.e., us)! Not to mention the unanimity that you imply; the judges are without question the most cantankerously independent herd of cats I've ever encountered professionally.

As for the lack of reader input on nominations, you may have a point there. This is why the Reader's Choice Awards were instituted. I personally regret that they seem to have gone by the wayside, I was one of the judges advocating that readers' nominations be considered in the Jolt mainstream and was only partly mollified by Reader's Choice.

"Folks such as Martin Fowler, Scott Ambler, Dave Thomas, Uncle Bob and other agilists serve on panels that help choose the winning products"...well, I'm flattered to be a 'folks such as' THAT august group. Unfortunately you need to check your facts, only one of those people is a Jolt judge and he regularly has to recuse himself because he's a prolific author in the space.

The process is necessarily subjective; a popularity contest among the readership would be "fairer" but much less useful to people actually interested in choosing tools. You can take or leave my personal credibility, but when people like Mike Riley or Andrew Binstock or Larry O'Brien weigh in on a tool, you'd do well to listen if you care about your organization's bottom line. It's not as if divine fiat has decreed "The Jolt Awards are the most important honor in software development". The Jolts matter as long as they provide value to the community, and not a heartbeat longer. But guess what? The industry still cares about them. Must not be as corrupt and scandalous as you posit. Or perhaps we've successfully pulled the wool over the eyes of the notoriously pliant and ignorant software-development industry for seventeen years straight. (And if you believe that one...let me show you the mail I get from readers when I goof in print. Ain't much gets past THAT crowd, friend.)

If you think you can do better, more power to you, start your own awards. (You're an industry "thought leader", it says so right there on your blog.) But you'll find that, no matter how much you protest your absolute integrity, no one is going to pay you the slightest scrap of attention. We earned our credibility, Mr. McGovern, and we earn it again every single year.

We don't publish much about the process, for three reasons: We don't want people gaming the system, we can't stand getting jawboned by sore losers (which happens often enough to disgust all of us), and we want the judges to feel absolutely free to express their opinions without worrying about who's looking over their shoulder.

But to offer a little insight, here's how the finalist selection process works: Vendors submit their nominations. As the list has grown, we've started requiring them to provide specific information (rather than just a URL to their product page) to justify why their product should receive the award. The judges then discuss the information, applying their knowledge of the industry to decide what is likely to be the most significant of the nominees, and finally vote by secret ballot. Frequently judges have experience with many of the products and can report on them; sometimes we have to go with our guts and try to predict what is of potential significance, even if it isn't hugely popular. Believe me, the issue of what constitutes a Jolt-worthy product is hotly contested -- every year.

We apply a sizable list of criteria. Is a product truly new, or just a bug-fix point revision of last year's? Is there widespread adoption? If not...should there be? Does a product have 'buzz'? Is it so worthy that we should put the weight of the Jolts behind creating some? And on, and on.

Finally, Mr. McGovern, I caution you to carefully consider the use of words like 'scandal'. It feels good when you rip on The Man, but forgive me if I take that a little personally, because The Man is me and some people who have earned my respect the hard way. If you have evidence of chicanery, the judges would sure like to know, because we want those people out of the process. I'm proud to be a member of this team, and I stand by them.