Superforecasting: The Art and Science of Prediction“Beliefs are hypotheses to be tested, not treasures to be protected.” – Philip E. Tetlock and Dan Gardner

Thinking, Fast and Slow is one of my favorite books. In it, Daniel Kahneman details how the human mind works in two modes: one fast and effortless, the other slow and laborious. You engage the slow system to split the check among three friends. The fast system works automatically, filling in blanks and recognizing patterns. It allows us operate smoothly on partial information. However, that quick judgement system can also lead us into dangerous biases and overconfidence.

In Superforecasting: The Art and Science of Prediction, Tetlock and Gardner apply the principles of behavioral economics to the practice of forecasting. Tetlock is the researcher whose previous studies led him to conclude that most expert prognosticators predicted future events no more accurately than dart-throwing chimps.

Tetlock led a major prediction effectiveness study called The Good Judgement Project (GJP). Tetlock and his co-researchers enlisted several thousand volunteers as contestants in a prediction competition. To be statistically meaningful, contestants had to make hundreds of predictions. They were of the sort… Will Scotland vote to secede from the UK? Will the Swiss examination of Yasser Arafat’s exhumed bones find traces of polonium?  

They established a system by which competitors were scored based on a combination of correctness and confidence level. They identified the top 2% as superforecasters. These people consistently predict events with much higher accuracy than everyone else.

You might expect that intelligence is the primary factor that sets the superforecasters apart from the rest, but while they were of above-average intelligence (top 20% of the population), it was the superforcaster’s ability to stay objective, counteract their own biases, and question their own beliefs that make them different and so effective. In large part, they were better at avoiding the biases identified by Kahneman.

From the book, here is a summary of the traits of the superforecasters:

In philosophic outlook, they tend to be:

CAUTIOUS: Nothing is certain
HUMBLE: Reality is infinitely complex
NONDETERMINISTIC: What happens is not meant to be and does not have to happen

In their abilities and thinking styles, they tend to be:

ACTIVELY OPEN-MINDED: Beliefs are hypotheses to be tested, not treasures to be protected
INTELLIGENT AND KNOWLEDGEABLE, WITH A “NEED FOR COGNITION”: Intellectually curious, enjoy puzzles and mental challenges
REFLECTIVE: Introspective and self-critical
NUMERATE: Comfortable with numbers

In their methods of forecasting they tend to be:

PRAGMATIC: Not wedded to any idea or agenda
ANALYTICAL: Capable of stepping back from the tip-of-your-nose perspective and considering other views
DRAGONFLY-EYED: Value diverse views and synthesize them into their own
PROBABILISTIC: Judge using many grades of maybe
THOUGHTFUL UPDATERS: When facts change, they change their minds
GOOD INTUITIVE PSYCHOLOGISTS: Aware of the value of checking thinking for cognitive and emotional biases

In their work ethic, they tend to have:

A GROWTH MINDSET: Believe it’s possible to get better
GRIT: Determined to keep at it however long it takes

To learn what each of these traits mean and how superforecasters manifest them to make significantly better predictions than their peers, check out the book. It’s like Thinking Fast and Slow applied to prediction with a heavy dose of The Black Swan, The Wisdom of Crowds, and Mindset. It was a great read/listen, and I highly recommend it!

Turtles All The Way Down

I enjoyed a moment today that would warm the heart of any geeky dad. Sitting in church this morning, the pastor told the story of his college days when he read Chariots of the Gods? and came to believe that life on Earth was seeded by aliens.  As an aside, he added that he hadn’t thought to ask the question of where the aliens came from, and if they came from aliens, then where did those aliens come from. My 11-year-old leaned over and whispered in my ear… “It’s turtles all the way down.”


hackathonIt’s only two days before the Marathon Data Systems sponsored Jersey Shore Comeback-a-thon, a 24-hour hackathon with a theme of bringing business back to the Jersey Shore in the first season after hurricane Sandy.

Many businesses along the Shore have worked very hard and made great investments to rebuild, clean up, and claw their way back into business in time for the summer season. Unfortunately, the images of devastation of the storm have left many would-be visitors thinking there is no beach to vacation to this year.

The $1000 grand prize will be awarded to the individual or team that comes up with the best application, system, or other creative use of technology to help bring business back to the Shore.  There will also be two $300 runner up prizes.

We’ve had good coverage from,, and triCity News. Get more details here.

Hashtag: #combackathon

Copy and Paste From Any Kindle

I have always wished I could copy and paste text from a Kindle book for quoting in a blog post or email. I could understand that this might not be possible from a Kindle device, but certainly it should be easy from my iPad or the Kindle cloud reader, right? Wrong!

A friend of mine wrote a great article on how to remember more of what you read on Kindle. He showed how will display a page with all of your highlights. This gave me a great idea… simply highlight the passage of interest, allow the reader to sync, and viola… you have the highlighted text on a Web page that you can copy and paste from. This even works with Kindle devices. Nice!

Audio Book Recommendations

I make use of my commute time to listen to podcasts and audio books from A coworker recently asked me about my favorite titles. Here are some of my favorites that would be of interest to software professionals.

Developing Software

The Lean Startup by Eric Ries: This is a fantastic book about how to develop a product in a lean/agile manner. Ries’s description of the build-test-learn loop is worth the price of the book.


Rework by Jason Fried and David Heinemeier Hansson: I found some interesting stuff here from the 37signals guys. My biggest complaint about this book is that the authors are way to quick to assume that all of their success is due to the choices they made. Just because they did something and had some success with it does not mean that the success was caused by the particular behavior in question.

Product Management

The Innovator’s Dilemma by Nathan Christensen: Christensen describes how disruptive technologies overthrow established technology when those incumbents believe they are least vulnerable to such a disruption.

The Innovator’s Solution by Nathan Christensen: Christensen continues where he left off. Another must-read for anyone who manages a technology product or company.


The Design of Everyday Things by Donald Norman: Norman wrote this back in 1988 (and you can tell), but it is an enduring classic and a product management must-read for good reason. A surprisingly fun listen.


Good to Great by Jim Collins: This is one of my favorite books of all time in any category. Collins describes the common attributes his research team found behind successful company leaders. I found the book to be quite inspiring and have made the attributes of leadership Collins describes personal goals for my own leadership.

Leading Change by John Kotter: Kotter describes eight steps to successful organizational change. I’ve seen these principles play out in commercial and social settings—sometimes where the leaders follow these steps and create effective change, but more often where they did not and the change efforts failed.

The Goal by Eli Goldratt: I first read The Goal in the late 1980s as an Industrial Engineering student. It is a good read and helps to clarify the folly of local optimization–in manufacturing or any complex system. This has become a favorite of proponents of Lean Software Development.

There are many more audio books that have really enjoyed, but I’ll save those for another day.

The Commoditization of Transactions

I’ve written in the past about how the relentless march of progress in software has made yesterday’s innovations today’s commodity.

This article from HBR captures the essence of how the software landscape is in the process of making traditional, data-oriented, system-of-record, transactional systems into commodity systems. Companies that have made a living with these traditional systems are going to wake up very shortly and find that their customers have become someone else’s community participants.

The combination of ubiquitous mobile connection, cloud computing, and the general adoption of social media is in the process of changing the expectations of software users. Keeping their data safe is no longer enough. They now expect (or soon will) a more immersive experience.

This doesn’t mean a link to a Facebook fan page that is merely a shallow marketing ploy. Users expect that their hosted application will allow them to interact with all of the other uses of your software (see Spiceworks as the best example of this). They expect that the software they run their business on enlists them into the community of users of your software. Don’t have a “community” of users? Your competition soon will.

Free Your Data, and UIs are Free

This video describes how the Massachusetts Department of Transportation began publishing real-time bus arrival data and within two months there were six compelling applications available that harnessed that data.

Joshua Robins of the Massachusetts Department of Transportation

This shows that if you make your data available, there are legions of developers willing to build applications on various platforms. Massachusetts could have spent months building a lame Web site and then making the data available. By making the data available first, they didn’t need any UI at all.

It’s easy to see how this can work with public data. I believe this same principle works inside the enterprise. If we build our line of business applications to publish their data in open formats, anyone can build the applications that display that data to users in whatever formats users need.

HT: Jack van Hoof

Why Software will Always be Hard, Part 2

My last post on Why Software will Always be Hard got me thinking about the implications for the software team that wants to create value (i.e., be worth more than they cost). If the bar is constantly being raised on what is considered commodity, then the software team creating value, must continue to get better in order to continue to deliver more next year than they did this year.


In the graph above, I’ve depicted the drive toward higher and higher expectations as the blue line. The green line represents the delivery level of the  high-performing team. The space between the two lines represents the value created by that team. In order to continue to deliver value, it must continue to innovate to deliver more for less.

While the commodity line drives steadily upward, the innovation line sometimes plateaus and may even drop prior to new increases. Innovation always brings risk and sometimes they will reduce your output before improving it. This is why even the high-performing team must continue to look for and expect improvements in its processed and technologies.

Why Software will Always be Hard

There is a funny thing about the software business… people expect that building great software should be getting easier. On the surface, that seems like a reasonable expectation. Software is a maturing field. The tools and technology are improving rapidly. The money to be made is attracting bright and capable people. So, why does making great software seem just as hard as it ever was?

There are two market forces working together to ensure that making software will be hard for a long time—at least as long as software will be worth building.

Market Forces

The first force in play is the fact that as soon as technology improves to allow us to build today’s software more efficiently and effectively, the market will raise the expectation of what it will pay for.

The second force in play is that the world is flat. This is especially true in the software business. Unlike automobiles and silly bands, software incurs no shipping costs; so the market of suppliers is truly global.


Those two market forces mean that as soon as a certain level of software functionality becomes easy to provide reliably (think on-time with no bugs), there is such a large supply of providers that no one will pay enough for that software to make a company profitable. Any software company (or department) that wishes to make a profit is going to have to provide something more: better integration to more complex systems; unseen functionality levels; innovative new platforms;  more immersive experiences, etc.

That pushing of the envelope means that software will be hard as long as there is any money to be made in it. My dad used to say to me “Eric, if it was easy, anyone could do it.” And if anyone could do it, it hardly ever pays well.

Teamwork Is an Individual Skill

I was intrigued by this interview with Christopher Avery about responsibility on agile teams at InfoQ. Intrigued enough to go download and read his book entitled Teamwork Is an Individual Skill: Getting Your Work Done When Sharing Responsibility.

The premise of the book is that, contrary to conventional wisdom, not only is there an “I” in team, but teams are constituted by nothing more than a bunch of “I”s. If none of those individuals takes responsibility for the team’s success, then no one is taking responsibility.

Whether it be the development team at work, the elder council at my church, or even my family, it seems there is virtually no area of life where my personal success is not dependent on the success of a team. Learning to take more responsibility for the success of those teams seemed like a good idea.

I found that the book could have used a good editor to tighten up some of the writing, and the formatting (at least on the Kindle version) was confusing at times; however, there was enough food for thought in there to make it worth the read.

Here are a few highlights to give you some flavor of the book…

Avery coins the term TeamWisdom, which he defines as follows.

TeamWisdom refers to all the individual mental skills and behaviors that lead to highly responsible and productive relationships at work. The idea is based on my definition of “team”: A team is a group of individuals responding successfully to the opportunity presented by shared responsibility. Thus someone with TeamWisdom takes responsibility for ensuring that the group rises to the occasion, and in the process, makes sure his own work gets done and done well.

Avery makes an important distinction between accountability and responsibility

…accountability can be assigned, but responsibility can only be taken.

Accountability and responsibility are not mutually exclusive. In fact, they are extremely complimentary. It is time for each of us in the workplace to take responsibility for relationships as well as accountability for deliverables, and to engage in the conversations that build productive relationships at work.

Avery makes the astute observation that if each team member acts in his own self-interest, then it is important to learn what motivates the other team members and assure that their interests are aligned with those of the team and with your interests. If your interests cannot be aligned, then you should withdraw from that team.

Once you understand your team members interests, it is in your interest to see that your teammates reach their goals.

The great philosopher/inventor, Buckminster Fuller, taught that the best way for one person to win is not by making others lose, but by making others win too. He taught from the 1940’s until his death that the more people a person helps to win, the more people that person can expect will help her win. Fuller’s teaching was in the forefront of a growing body of literature about the power and humanity of “servant leadership.” Being a servant leader means helping one’s followers become successful, instead of expecting followers to serve one’s personal success.

He then gives the following challenge in the Personal Challenge section that is included in with each TeamWisdom principle.

Do your partners and teammates provide you with access to their thoughts because they experience you as a person who helps them achieve their goals? Listen carefully to your associates to learn what is truly important to them. Check in with yourself to determine your level of commitment to them. If this level of commitment is low, ask yourself why. If it is high, ask yourself how you are willing to help. Then offer that help.

Avery talks about collaboration and includes valuable insights like this:

Most people find it much easier to grant a favor than to ask for one. However, people with TeamWisdom know that asking for a favor actually grants the other person a favor. Asking for a favor communicates to the other person that they are important to us, that we depend on them, and that we are even willing to owe them one. People with TeamWisdom understand that the person who asks for the first favor sets the tone for the collaboration.

Here is an example of Avery’s insight on the effect unmotivated team members:

Is the team leader the most powerful member of your team? Is the most inspired member the most powerful? The smartest member? Nope. None of the above. Like it or not, the most powerful member of your team is the one who cares the least about your team’s task. Sorry, but that’s the truth. The least-committed member of your team is the most powerful because his lack of commitment establishes a low baseline to which other
team members may fall. The success—or mediocrity—of your team likely will be determined by him.

Read the book. Take responsibility for the success of your teams, whether you are the leader or simply a contributor.