Superforecasting: The Art and Science of Prediction“Beliefs are hypotheses to be tested, not treasures to be protected.” – Philip E. Tetlock and Dan Gardner

Thinking, Fast and Slow is one of my favorite books. In it, Daniel Kahneman details how the human mind works in two modes: one fast and effortless, the other slow and laborious. You engage the slow system to split the check among three friends. The fast system works automatically, filling in blanks and recognizing patterns. It allows us operate smoothly on partial information. However, that quick judgement system can also lead us into dangerous biases and overconfidence.

In Superforecasting: The Art and Science of Prediction, Tetlock and Gardner apply the principles of behavioral economics to the practice of forecasting. Tetlock is the researcher whose previous studies led him to conclude that most expert prognosticators predicted future events no more accurately than dart-throwing chimps.

Tetlock led a major prediction effectiveness study called The Good Judgement Project (GJP). Tetlock and his co-researchers enlisted several thousand volunteers as contestants in a prediction competition. To be statistically meaningful, contestants had to make hundreds of predictions. They were of the sort… Will Scotland vote to secede from the UK? Will the Swiss examination of Yasser Arafat’s exhumed bones find traces of polonium?  

They established a system by which competitors were scored based on a combination of correctness and confidence level. They identified the top 2% as superforecasters. These people consistently predict events with much higher accuracy than everyone else.

You might expect that intelligence is the primary factor that sets the superforecasters apart from the rest, but while they were of above-average intelligence (top 20% of the population), it was the superforcaster’s ability to stay objective, counteract their own biases, and question their own beliefs that make them different and so effective. In large part, they were better at avoiding the biases identified by Kahneman.

From the book, here is a summary of the traits of the superforecasters:

In philosophic outlook, they tend to be:

CAUTIOUS: Nothing is certain
HUMBLE: Reality is infinitely complex
NONDETERMINISTIC: What happens is not meant to be and does not have to happen

In their abilities and thinking styles, they tend to be:

ACTIVELY OPEN-MINDED: Beliefs are hypotheses to be tested, not treasures to be protected
INTELLIGENT AND KNOWLEDGEABLE, WITH A “NEED FOR COGNITION”: Intellectually curious, enjoy puzzles and mental challenges
REFLECTIVE: Introspective and self-critical
NUMERATE: Comfortable with numbers

In their methods of forecasting they tend to be:

PRAGMATIC: Not wedded to any idea or agenda
ANALYTICAL: Capable of stepping back from the tip-of-your-nose perspective and considering other views
DRAGONFLY-EYED: Value diverse views and synthesize them into their own
PROBABILISTIC: Judge using many grades of maybe
THOUGHTFUL UPDATERS: When facts change, they change their minds
GOOD INTUITIVE PSYCHOLOGISTS: Aware of the value of checking thinking for cognitive and emotional biases

In their work ethic, they tend to have:

A GROWTH MINDSET: Believe it’s possible to get better
GRIT: Determined to keep at it however long it takes

To learn what each of these traits mean and how superforecasters manifest them to make significantly better predictions than their peers, check out the book. It’s like Thinking Fast and Slow applied to prediction with a heavy dose of The Black Swan, The Wisdom of Crowds, and Mindset. It was a great read/listen, and I highly recommend it!

Free eBook on Azure ML and R

Data Science in the Cloud with Azure ML and R is a short eBook that steps you through building a model and deploying it to Azure ML as a Web service. The book assumes you already know how to use R; so, it’s not the best starting point if you are new to R. However, I’d go ahead and pick up the book. It covers the critical area of how to deploy a model once it’s built.

Audio Book Recommendations

I make use of my commute time to listen to podcasts and audio books from A coworker recently asked me about my favorite titles. Here are some of my favorites that would be of interest to software professionals.

Developing Software

The Lean Startup by Eric Ries: This is a fantastic book about how to develop a product in a lean/agile manner. Ries’s description of the build-test-learn loop is worth the price of the book.


Rework by Jason Fried and David Heinemeier Hansson: I found some interesting stuff here from the 37signals guys. My biggest complaint about this book is that the authors are way to quick to assume that all of their success is due to the choices they made. Just because they did something and had some success with it does not mean that the success was caused by the particular behavior in question.

Product Management

The Innovator’s Dilemma by Nathan Christensen: Christensen describes how disruptive technologies overthrow established technology when those incumbents believe they are least vulnerable to such a disruption.

The Innovator’s Solution by Nathan Christensen: Christensen continues where he left off. Another must-read for anyone who manages a technology product or company.


The Design of Everyday Things by Donald Norman: Norman wrote this back in 1988 (and you can tell), but it is an enduring classic and a product management must-read for good reason. A surprisingly fun listen.


Good to Great by Jim Collins: This is one of my favorite books of all time in any category. Collins describes the common attributes his research team found behind successful company leaders. I found the book to be quite inspiring and have made the attributes of leadership Collins describes personal goals for my own leadership.

Leading Change by John Kotter: Kotter describes eight steps to successful organizational change. I’ve seen these principles play out in commercial and social settings—sometimes where the leaders follow these steps and create effective change, but more often where they did not and the change efforts failed.

The Goal by Eli Goldratt: I first read The Goal in the late 1980s as an Industrial Engineering student. It is a good read and helps to clarify the folly of local optimization–in manufacturing or any complex system. This has become a favorite of proponents of Lean Software Development.

There are many more audio books that have really enjoyed, but I’ll save those for another day.

Teamwork Is an Individual Skill

I was intrigued by this interview with Christopher Avery about responsibility on agile teams at InfoQ. Intrigued enough to go download and read his book entitled Teamwork Is an Individual Skill: Getting Your Work Done When Sharing Responsibility.

The premise of the book is that, contrary to conventional wisdom, not only is there an “I” in team, but teams are constituted by nothing more than a bunch of “I”s. If none of those individuals takes responsibility for the team’s success, then no one is taking responsibility.

Whether it be the development team at work, the elder council at my church, or even my family, it seems there is virtually no area of life where my personal success is not dependent on the success of a team. Learning to take more responsibility for the success of those teams seemed like a good idea.

I found that the book could have used a good editor to tighten up some of the writing, and the formatting (at least on the Kindle version) was confusing at times; however, there was enough food for thought in there to make it worth the read.

Here are a few highlights to give you some flavor of the book…

Avery coins the term TeamWisdom, which he defines as follows.

TeamWisdom refers to all the individual mental skills and behaviors that lead to highly responsible and productive relationships at work. The idea is based on my definition of “team”: A team is a group of individuals responding successfully to the opportunity presented by shared responsibility. Thus someone with TeamWisdom takes responsibility for ensuring that the group rises to the occasion, and in the process, makes sure his own work gets done and done well.

Avery makes an important distinction between accountability and responsibility

…accountability can be assigned, but responsibility can only be taken.

Accountability and responsibility are not mutually exclusive. In fact, they are extremely complimentary. It is time for each of us in the workplace to take responsibility for relationships as well as accountability for deliverables, and to engage in the conversations that build productive relationships at work.

Avery makes the astute observation that if each team member acts in his own self-interest, then it is important to learn what motivates the other team members and assure that their interests are aligned with those of the team and with your interests. If your interests cannot be aligned, then you should withdraw from that team.

Once you understand your team members interests, it is in your interest to see that your teammates reach their goals.

The great philosopher/inventor, Buckminster Fuller, taught that the best way for one person to win is not by making others lose, but by making others win too. He taught from the 1940’s until his death that the more people a person helps to win, the more people that person can expect will help her win. Fuller’s teaching was in the forefront of a growing body of literature about the power and humanity of “servant leadership.” Being a servant leader means helping one’s followers become successful, instead of expecting followers to serve one’s personal success.

He then gives the following challenge in the Personal Challenge section that is included in with each TeamWisdom principle.

Do your partners and teammates provide you with access to their thoughts because they experience you as a person who helps them achieve their goals? Listen carefully to your associates to learn what is truly important to them. Check in with yourself to determine your level of commitment to them. If this level of commitment is low, ask yourself why. If it is high, ask yourself how you are willing to help. Then offer that help.

Avery talks about collaboration and includes valuable insights like this:

Most people find it much easier to grant a favor than to ask for one. However, people with TeamWisdom know that asking for a favor actually grants the other person a favor. Asking for a favor communicates to the other person that they are important to us, that we depend on them, and that we are even willing to owe them one. People with TeamWisdom understand that the person who asks for the first favor sets the tone for the collaboration.

Here is an example of Avery’s insight on the effect unmotivated team members:

Is the team leader the most powerful member of your team? Is the most inspired member the most powerful? The smartest member? Nope. None of the above. Like it or not, the most powerful member of your team is the one who cares the least about your team’s task. Sorry, but that’s the truth. The least-committed member of your team is the most powerful because his lack of commitment establishes a low baseline to which other
team members may fall. The success—or mediocrity—of your team likely will be determined by him.

Read the book. Take responsibility for the success of your teams, whether you are the leader or simply a contributor.

Peopleware: An Aging Classic

Peopleware: Productive Projects and TeamsSome months ago I was telling a friend at work how everyone in the software industry should read Peopleware by Tom DeMarco and Timothy Lister

I have always said that Peopleware should be required reading for anyone who manages software developers. I’ve also said that if you want to know my philosophy as a manager, just read Peopleware.

However, the first edition was written in 1987 and the second edition, which I had read, was written in 1999. I decided to read it again. I was curious to see how it holds up more than ten years later.

On second read, it is clear that Peopleware was written prior to the agile wave in software development. However, Peopleware was such a prescient work that in 2011 I still found it to be insightful and compelling. Much of what they call for has become more commonplace since 1999.

As DeMarko and Lister railed against public address systems in the office, I felt like I had taken a trip back in time. However, the principle that management too often optimizes the wrong factors in the workplace is as relevant as ever. I’ve seen too many situations where more care was put into color schemes and aesthetics than the mechanics of how the software team would work together (acoustics, workstation configuration, etc.).

The insights and anecdotes on management of software people made the second reading worth the investment. The chapters on building a high performing, jelled team were especially valuable.

Bottom line… I still believe that everyone in software should read Peopleware and that managers must read it.

Pair Programming Resources

Looking for a Good, Free Resource

After introducing the concept of pair programming, I went looking for (free, of course) resources to further explain how it worked and what the benefits are. At first I was surprised at how little I found. Sure, there are chapters in all of the XP books on pairing, but I was looking for something that I could just send a link to in an email. I figured there would be lot of good stuff to choose from.

I think there is not a lot of profound work on the Web explaining what pair programming is because, at it’s core, pair programming is pretty simple. In addition, there is not much written on its costs and benefits because it is a relatively new practice and doesn’t have a whole lot of academic research done on it.

What is Pair Programming?

As is the case with a lot of things, a good starting point for a brief overview of pair programming is to go to the Wikipedia entry. It has a surprisingly good, brief overview of the what and the why of the practice.

The Costs and Benefits of Pairing

Once you’ve got a basic idea of the concept, the next step is to analyze the pros and cons of the practice. For this, I recommend Alistair Cockburn and Laurie Williams’ paper The Costs and Benefits of Pair Programming. This paper appears to be about eight years old, but is still perfectly relevant. This might be the best free, less-than-ten-printed-pages introduction to pair programming.

Installing Pair Programming

My limited personal experience was pretty successful, but there are so many dynamics that come into play when introducing a practice like this into an organization where the concept in entirely new. So, I was still looking for more resources on how to introduce the concept and the practice of pairing. I ordered a copy of Pair Programming Illuminated by Laurie Williams and Robert Kessler. I’m only on the second of twenty-seven chapters, but what I’ve read so far seems to be insightful and helpful. I’ll post more about what I find as I work through it.