TDD and the Power of Habit

“Champions don’t do extraordinary things. They do ordinary things, but they do them without thinking, too fast for the other team to react. They follow the habits they’ve learned.” –Tony Dungyimage

The quote above comes from The Power of Habit: Why We Do What We Do in Life and Business by Charles Duhigg. This book is a fascinating look at habits—what they are, how they form, how they affect us, and how can we choose our habits.

The Habit Loop

Most of us are barely aware of the habits that make up so much of what we do, but psychology and neuroscience researchers have made great progress in understanding how habits form.

image

At the core of this research is what Duhigg calls the habit loop. We usually have no trouble identifying the routine, which is what we think of as the habit itself. The value comes from identification of the two other elements of the habit loop.

The first insight comes in realizing that habits are driven by some kind of reward. The reward could be virtually anything–relief from boredom, sense of completion, etc.image As our behaviors result in the reward we seek repeatedly, our brains literally restructure themselves. Our brains move the instructions for performing the actions needed to invoke the reward to areas that can operate without our conscience thought. Our habits become become so deeply encoded that they become almost impossible to separate from who we are.

The second insight comes in recognizing that there are particular cues that trigger the routine in pursuit of the reward. Cues could be just about anything as well–a feeling of boredom, the sight of a particular object, the presence of a particular person, etc.  Identifying these cues are key to understanding and changing our habits, but you’ll have to read the book for that.

Keystone Habits

Studies have documented that families who habitually eat dinner together seem to raise children with better homework skills, higher grades, greater emotional control, and more confidence. Making your bed every morning is correlated with better productivity, a greater sense of well-being, and stronger skills at sticking with a budget. It’s not that a family meal or a tidy bed causes better grades or less frivolous spending. But somehow those initial shifts start chain reactions that help other good habits take hold. If you focus on changing or cultivating keystone habits, you can cause widespread shifts. However, identifying keystone habits is tricky. To find them, you have to know where to look. Detecting keystone habits means searching out certain characteristics. Keystone habits offer what is known within academic literature as “small wins.” They help other habits to flourish by creating new structures, and they establish cultures where change becomes contagious.

Keystone habits are the drivers that lead to other habits. For example, studies have shown that people who adopt the habit of regular exercise find that seemingly unrelated areas of their lives change for the better (being more productive at work, smoking less, being more patient with their kids, etc.).

Small Wins

Small wins are exactly what they sound like, and are part of how keystone habits create widespread changes. A huge body of research has shown that small wins have enormous power, an influence disproportionate to the accomplishments of the victories themselves. “Small wins are a steady application of a small advantage,” one Cornell professor wrote in 1984. “Once a small win has been accomplished, forces are set in motion that favor another small win.” Small wins fuel transformative changes by leveraging tiny advantages into patterns that convince people that bigger achievements are within reach.

Small wins are those sure steps in the right direction that separate high performers from everyone else.

TDD: Keystone Habit of Great Developers

image

The agile software movement centers around incremental and iterative steps with feedback mechanisms that allow for lots of small adjustments to bring the project to success. So, when I hear Duhigg talk about small wins, it resonates with what we’ve found in building software.

Test-Driven Development (TDD) is the most central of agile practices. It fits Duhigg’s definition of a keystone habit.

TDD and Small Wins

The demands on software developers get bigger and more more ambitious all the time. We presented with complex problems that we generally don’t know how to solve. It is easy to become overwhelmed by the magnitude and get bogged down trying to boil the ocean with code.

The discipline of TDD forces us to methodically break problems down into small chunks. Each passing test is a win and one small step closer to the system the customer needs.

The Cue is obvious: a customer needs a new feature.

The Routine is the well documented (but seldom practiced) red-green-refactor loop.

The Reward is the passing test and a thoroughly executed unit of responsibility completed.

The cycle continues, with small wins coming one after another until the feature is complete.

TDD as Keystone Habit

The TDD habit has many follow-on benefits beyond completed, tested software. Every time we write a test before we write the code, we remind ourselves of several truths:

  • If the software is worth creating, it is worth ensuring that it works.
  • Each unit of functionality must have a single responsibility and its dependencies must be loosely coupled.
  • Keeping the code clean is vital. Keeping it clean is made possible by the protection provided by a comprehensive suite of tests.
  • Writing only enough to make the test pass keeps our focus on the customer’s requirement and not a speculative future feature.

I’ve found a common idea emerging among the colleagues that I respect most: we would rather hire a developer that practices TDD and does not have experience in the primary programming language our shop uses than hire a developer with that programming language experience that does not practice TDD.

Language syntax and style is relatively easy to learn. TDD is a keystone habit of great software developers.

Help Turn the “Jersey Shore” into “Silicone Shore”

SiliconeShoreWant to work with the latest technology on a self-directed team? Think you need to be in Silicone Valley to be part of a company setting the standard in cloud and mobile computing? Read on.

Marathon Data Systems, on the beautiful Jersey Shore, through our various vertical-focused brands, serves the people who provide the services that we all count on: HVAC, plumbing, lawn care, pest control, maid services, carpet cleaning, and more. We are in the process of building a mobility solution that will set the industry standard for field service worker and salesperson productivity.

Cross-Platform Mobile

Our new cross-platform mobile client is HTML5-powered and will initially run on iOS and Android devices. The technician or salesperson is continually operating on the latest information (no more periodic synchronization) and yet can be fully functional while out of network coverage.

SenchaPlusCoffee

The new mobile client is built in CoffeeScript on the Sencha Touch 2 platform. The combination of Sencha Touch and CoffeeScript making building off-line capable, singe-page applications actually pretty fun. Sencha takes care of the tedium of rendering just the right HTML to the screen for each device, while we focus  on building great user experiences.

Since WebSockets are not yet a reality (and pull-based applications are lame), we’re using SignalR to simulate server push from ASP.NET MVC (think Node.js scalability in ASP.NET MVC).

Message-Driven CQRS Back-End

DomainDrivenDesignOn the back-end we are are building out a scalable, event-driven CQRS system built around NServiceBus, following the principles of Domain-Driven Design (DDD).

In Domain-Driven Design, we recognize that the heart of the software is the domain-specific behavior that allows its users to solve their problems. We establish a vocabulary that spans across technical and nontechnical people. We iteratively tweak and adjust our domain model to better map to the problem space. We focus on building an effective model of our domain, while minimizing entanglement with infrastructure concerns.

nServiceBus_LogoNServiceBus (NSB) allows us to focus on the semantic meaning of our commands and events and what the system’s behavior should be. NSB handles interaction with queues, message routing, retries, and other plumbing-level concerns.

ravendb

Data persistence is achieved through the joy that is RavenDB. If you think ORMs like NHibernate are good, just wait until you have built a C# application with RavenDB.

RavenDB is a high-performance, second generation document database, native to C#. Queries are made through LINQ. Joins and transforms are handled through map/reduce functions also written in LINQ. The impedance mismatch between object-oriented code and a relational database is not merely bridged, but is eliminated.

Agile Team

Agile can mean almost anything you want it to mean, but at Marathon, it means…

Agile at Marathon

A belief in the core values of the agile manifesto:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

The Scrum process, where a self-organizing team works together to plan and deliver iteratively and incrementally.

The Extreme Programming (XP) practices of test-driven development, pair-programming, continuous integration, collective code ownership.

A commitment to software craftsmanship. Being a professional means a commitment to quality work and continual improvement. Great software comes less from particular technologies that come and go, but from fundamentals like the SOLID principles and object-oriented design principles and patterns that stand the test of time.

This job may be for you if…

  • You are good, but you expect to keep getting better.
  • You can do it yourself, but you’d much rather work together with a team of dedicated developers.
  • You love cool technologies, but you care about user loving and using your software more.
  • You love coding in C#.
  • If you are not currently test-driven, you want to learn to be.
  • If you are not currently pair-programming, you are willing to give it a try.

If that sounds like you, please send your resume to jobs@marathondata.com. Come join us as we create great software to serve the people who serve the world!

Lightweight Context/Specification BDD in C#

Behavior-Driven Development (BDD) provides all of the engineering benefits of traditional Test-Driven Development (TDD) while additionally resulting in a specification that non-developers can read and validate. At its heart, BDD transforms the tests of TDD into specifications. Those specifications are expressed in English sentences that are expressed in business value as opposed to coding or engineering terms.

The most popular structure for BDD today is called the Gherkin format and follows a Given/When/Then format, like…

Given a new bowling game
When all frames are strikes
Then the score should be 300

There are frameworks like SpecFlow to help you arrange your specifications (test) into this format. However, I find this format awkward and forced. I prefer the simpler format known as Context/Specification (aka When/Should)…

When all frames are strikes
Should have a score of 300

There are frameworks, like MSpec, that attempt to make the specifications read more like English sentences. However, I find that these frameworks get in the way as much as they help. It is also nice to be able to write readable tests with just PONU (plain old NUnit). Over time, I’ve developed a convention that I find easy to write and easy to read. I’ve also developed a tool  that turns the tests into a markdown file that can be turned into a pretty HTML report.

To show the approach at work, I present some snippets from a hypothetical order pricing system I created as a “developer test” provided by a prospective employer last summer. Here is what I was given:

Instructions: Build a system that will meet the following requirements. You may make assumptions if any requirements are ambiguous or vague but you must state the assumptions in your submission.

Overview: You will be building an order calculator that will provide tax and totals. The calculator will need to account for promotions, coupons, various tax rule, etc… You may assume that the database and data-access is already developed and may mock the data-access system. No UI elements will be built for this test.

Main Business Entities:

  • Order: A set of products purchased by a customer.
  • Product: A specific item a customer may purchase.
  • Coupon: A discount for a specific product valid for a specified date range.
  • Promotion: A business wide discount on all products valid for a specified date range.

*Not all entities are listed – you may need to create additional models to complete the system.

Business Rules:

  • Tax is calculated per state as one of the following:
    • A simple percentage of the order total.
    • A flat amount per sale.
  • Products categorized as ‘Luxury Items’ are taxed at twice the normal rate in the following states
    • FL
    • NC
    • CA
  • Tax is normally calculated after applying coupons and promotional discounts. However, in the following states, the tax must be calculated prior to applying the discount:
    • FL
    • NM
    • NV
  • In CA, military members do not pay tax.

Requirements:

Adhering to the business rules stated previously:

  • The system shall calculate the total cost of an order.
  • The system shall calculate the pre-tax cost of an order.
  • The system shall calculate the tax amount of an order.

Deliverables:

  • A .NET solution (you may choose either C# or VB) containing the source code implementing the business rules.
  • Unit tests (you may choose the unit testing framework).
  • A list of assumptions made during the implementation and a relative assessment of risk associated with those assumptions.

You can see that there are quite a few specifications here. It’s a perfect scenario for a BDD approach. Lets take a look at the specification that most states charge taxes on the discounted price, while a few states require taxes to be calculated on the original price.

Here is a specification that a standard tax state calculates taxes on the discounted price…

namespace Acme.Tests.ConcerningCoupons
{
    [TestFixture]
    public class When_coupon_is_applied_to_item_on_order_in_standard_tax_state
    {
        private Order _order;
        [TestFixtureSetUp] public void Context()
        {
            Product product = new Product(10);
            Coupon coupon = CreateCoupon.For(product).WithDiscountOf(.5m);
            _order = CreateOrder.Of(product).Apply(coupon).In(StateOf.NC);
        }

        [Test] public void Should_calculate_tax_on_discounted_price()
        {
            _order.Tax.ShouldEqual(.25m);
        }
    }
}

You can see that the test fixture class name defines the context (the when) The test method name specifies the specification (the should). The Context method sets us the context in the class name. Also note the ConcerningCoupons in the namespace. This allows us to categorize the specification.

Here is the code that specifies the prediscount tax states…

namespace Acme.Tests.ConcerningCoupons
{
    [TestFixture]
    public class When_coupon_is_applied_to_item_on_order_in_prediscount_tax_state
    {
        private Order _order;
        [TestFixtureSetUp] public void Context()
        {
            Product product = new Product(10);
            Coupon coupon = CreateCoupon.For(product).WithDiscountOf(.5m);
            _order = CreateOrder.Of(product).Apply(coupon).In(StateOf.FL);
        }

        [Test] public void Should_calculate_tax_on_full_price()
        {
            _order.Tax.ShouldEqual(.50m);
        }
    }
}

Now take a look at a section of the report generated from the tests…

orders

Anyone can now compare the generated report to the original specification to verify we hit the mark. It’s a little more work to structure your tests this way, but the benefits are worth it.

The full source for the sample and the report generator are available here.

Sprint Planning and Decision Fatigue

This article explores the physiological and psychological effects of fatigue brought on by making decisions. The fatigue that comes from making decision after decision immediately reminded me of my team’s Scrum sprint planning days.

The Scrum method breaks software development into iterative cycles called sprints. Our sprints were the highly typical two weeks in length. The idea is that two weeks is a short enough planning horizon that we can pull in enough work from the backlog to fill that time period. Then we demo what we’ve done to the various stakeholders in the company, adjust existing backlog priorities, plan another sprint, and on it goes.

Sprint planning day looked something like this…

9:00 Demos (any stories that haven’t been shown yet)

9:30 Close out the Previous Sprint (closing stories in VersionOne, splitting any unfinished stories, etc.)

10:00 Retrospective (look back over the prior sprint and identify things that worked well that we want to do more of, what didn’t work so well, and identify any impediments to progress)

10:45 Start Sprint Planning (Story Breakdown)

1:30 Finish Sprint Planning (Story Breakdown)

We often wouldn’t finish sprint planning until after 4:30. 

Story Breakdown

Sprint planning is the process of taking the high-level stories and breaking them down into tasks. We did this as a team; so, we had everyone’s input and everyone knew how we were going to go about implementing each story. This is vital to maintaining a team approach to building the product.

This story breakdown, however, is the hardest part of the whole sprint. We have to make decision after decision about how we are going to implement a feature…

Will there be a new database table? Will it be a variant of some existing feature or something new? Is there some new UI element that we haven’t tackled before? and so on.

Then for every decision, we have one more decision: how long do we think it will take.

I believe the hardest part is that we move from one decision to the next without actually doing anything. We are simply adding our decisions to the inventory to be acted on over the next two week. This makes the decision fatigue factor even greater.

By the time we got to 3:00 or 3:30 the team would often be so fatigued that we would start placing two tasks on each story: Plan it and Do it. During the sprint, if we came across a story with a “Do it” task, it was a safe bet that it was planned late in the day.

Remedy

I can see two ways to reduce the decision fatigue that comes with Scrum planning day: 1) reduce the Sprint length, or 2) don’t do Scrum.

For most teams doing Scrum, I think shrinking the Sprint length to one week will reduce the planning day fatigue. Our team, for other reasons, switched to a Kanban continuous flow model. Under that model, we did the story breakdown as the queue of planned stories got low. It was never two weeks worth at one time, and we had fewer “Do it” tasks.

Shifting from Scrum to Kanban

My team had been following (more or less) the Scrum process for two years. The Scrum rhythm of biweekly planning, demo, and retrospective with daily stand-ups served us well most of the time. However, we have since shifted to a process that looks more like Kanban. 

The Transition

We had the most friction with the Scrum process after initial releases when our focus turned more toward support and training of our customers. The two-week planning cycle does not accommodate this kind of reactive, responsive work. The dissonance between our stated process and the reality of the work day felt like a process breakdown even though what we were doing was clearly the most important thing for out team. It tended to make everyone a little cranky. Since agile is all about inspect and adapt and choosing a path that gets results, it was time for some adapting.

Our Flavor of Kanban

Unlike Scrum, which has identified creators and a set of doctrines that define orthodoxy, Kanban is more loosely defined. Our idea of Kanban starts with Scrum and and makes a few modifications.

We no longer plan a sprint. We simply pull stories from the backlog as we need more work to do. The VersionOne storyboard gives us a visual of how many items are in each status, and we only pull in a new story if the inventory level are low across the board.

Instead of planning for an entire day before a sprint, we now plan stories whenever the backlog of planned stories is low (less than a few days worth). We still demo the product and hold retrospectives every two weeks.

Since we no longer have sprint boundaries, we needed to make some changes to the way we integrate stories so that we always have a potentially shippable product. We now do all story or defect development on an activity branch in the repository and only merge back to the trunk after it has been thoroughly tested. This creates a bit of new overhead, but we’ve found it worth the effort.

Result

Now we are able to be responsive to new items that come in. The development team feels less jerked around when support issues come in and priorities change. We’d all like to get back to pure product development, but better to face reality as it is than to pretend it is something it is not.

Peopleware: An Aging Classic

Peopleware: Productive Projects and TeamsSome months ago I was telling a friend at work how everyone in the software industry should read Peopleware by Tom DeMarco and Timothy Lister

I have always said that Peopleware should be required reading for anyone who manages software developers. I’ve also said that if you want to know my philosophy as a manager, just read Peopleware.

However, the first edition was written in 1987 and the second edition, which I had read, was written in 1999. I decided to read it again. I was curious to see how it holds up more than ten years later.

On second read, it is clear that Peopleware was written prior to the agile wave in software development. However, Peopleware was such a prescient work that in 2011 I still found it to be insightful and compelling. Much of what they call for has become more commonplace since 1999.

As DeMarko and Lister railed against public address systems in the office, I felt like I had taken a trip back in time. However, the principle that management too often optimizes the wrong factors in the workplace is as relevant as ever. I’ve seen too many situations where more care was put into color schemes and aesthetics than the mechanics of how the software team would work together (acoustics, workstation configuration, etc.).

The insights and anecdotes on management of software people made the second reading worth the investment. The chapters on building a high performing, jelled team were especially valuable.

Bottom line… I still believe that everyone in software should read Peopleware and that managers must read it.

Unit Tests Just a Happy Side Effect of Test-Driven Development

Interviewing developers has given me an interesting, although not scientific, sampling of the thinking and mindset of software developers in the Atlanta area. While there have been a few bright spots, most of what I’ve learned has been disheartening.

I have found that TDD is still only an idea that most developers have heard about but never practiced. Of the ones who have practiced it, almost none of them understand what TDD was meant to achieve and only view it as automated unit testing. I believe this is due to two factors working together derail the original intent of TDD. First, TDD was named poorly. “Test” has specific meaning and has for a long time. It naturally biases us toward a certain type of thinking (strangely enough… testing and quality assurance). Second, most developers and development organizations would rather apply a formula or recipe to what they do than to take the time and effort to deeply understand what makes software development success or failure (and all of the degrees in between).

Scott Bellware is the guy who first got me thinking about a new way to approach TDD. It started with a new name: Behavior Driven Development (BDD). I actually think the “D” should be changed to Design to complete the mind shift. In a nutshell, BDD attempts to get back to the XP view of the tests being documentation of what the code should do. In this way of thinking, the set-up code is referred to as context and the assertions are referred to as specification. It has helped reform my thinking about TDD for the better.

You can catch Scott pontificating on the subject on the latest Hanselminutes podcast. Scott doesn’t get as specific as I would have liked, but this interview shows the side of Scott that I like so much. He’s passionately advocating a valuable practice and philosophy without the insults and vitriol that he is prone to fall into. I highly recommend it.

[Update: My reference to “Scott” in the above paragraph is to Bellware–not Hanselman, who is always pleasant and rarely vitriolic.]

Craftsmanship over Compromise

(Uncle) Bob Martin made a keynote address at Agile 2008 where he spoke about many of the concepts in his book Clean Code. In that talk, he proposed amending the Agile Manifesto to add the value of “Craftsmanship over Crap.” He later realized that this wouldn’t work because no one actually values crap. He later changed his proposal in a blog post where he suggested “Craftsmanship over Execution”. In that post he offered up a challenge to find a better word than Execution.

I hereby offer my entry as Craftsmanship over Compromise. I like it because we all value compromise, just like we value processes and tools, comprehensive documentation, contract negotiation, and following a plan. Compromise is a necessary part of the software business. However, we should value the ethic of creating an effective, lasting solution over shortcuts that are instantly gratifying, but are more costly in the long-run.

Looking for a ScrumMaster

[Update 1/29/2009: I’m happy to report that we are no longer looking!]

We (Agentek) are looking for an experienced ScrumMaster. If you know of someone in the Atlanta area that is looking for a position with a growing company that is early in its transition to agile software development and is committed to the Scrum process, please send him or her our way.

Here is the job description…

Agentek is a leading provider of mobile field force automation and communications solutions that enable businesses to achieve a predictable, reliable and repeatable customer experience from commitment to service fulfillment in real-time. From customized and pre-packaged applications to backend enterprise integration, Agentek delivers best-in-class mobile computing solutions that address the challenges companies face in areas such as field service, transportation and logistics automation. For additional information visit www.agentek.com.

We currently seek a Software Development Manager/ScrumMaster to join our expanding technical team.


Description:


Reporting to Agentek’s Vice President of Products and Engineering, our Software Development Manager/ScrumMaster will be working in an environment specifically designed for rapid application development of wireless applications for the Field Service Automation and Logistics market. The ideal candidate will have experience managing a team of 8-10 developers AND have experience as a ScrumMaster in a highly Agile development environment. Will be ScrumMaster for 2 teams and provide guidance to developers as a manager and Scrum coach.


Responsibilities:

  • As ScrumMaster, coordinate and facilitate daily stand-up meetings, Sprint planning sessions, Sprint reviews, and Sprint retrospectives
  • Work closely with the Product Owners, ensuring that product backlogs are updated and ready for the teams
  • Facilitate discussion and conflict resolution
  • Help team members to understand roles in the Scrum process (planning, review, estimating, etc.)
  • Work with team to estimate story points
  • Help team to choose stories and tasks in a Sprint, decompose stories into appropriate size tasks, and ensure that team does not over commit
  • Work with team to define what “done” means
  • Ensure that Sprint burndowns are updated daily and are visible to the team
  • Encourage team to adopt Agile engineering practices
  • Ensure accuracy of information tracked during Sprints
  • Manage departmental resources and staffing – enhancing and maintaining a best-of-class engineering team, hiring and terminating as appropriate to ensure a solid team
  • Conduct performance reviews of team members, coaching team members in how to be most effective in the Agile/Scrum environment

Required Qualifications:

  • Bachelor’s Degree in Computer Science or equivalent
  • Certified ScrumMaster
  • Deep understanding of Agile principles including eXtreme Programming and Scrum
  • 5-10 years experience in software development and 2-5 years as development Manager or Director managing teams of at least 6 engineers.
  • Previous development experience with Visual Basic, C#, ASP.NET, and MySQL
  • History of managing projects, both successfully and unsuccessfully
  • Ability to balance the servant/leader role
  • Strong business and technical vision
  • Excellent verbal and written communication skills
  • Strong presentation, leadership, and mentoring skills as a motivated person who encourages others
  • Strong team work and interpersonal skills – ability to communicate and work well with people at all levels
  • Ability to field and manage multiple competing priorities in a fast-paced environment
  • Experience working in a early-stage or venture-funded company
  • Ability positively to engage conflict and constructively resolve problems
  • Mobile application experience a big plus, but not required

To learn more about this exciting opportunity with a leading provider of mobile computing solutions, please send an email to greg.schillo@agentek.com, including a cover letter describing your experience of required skills in the body of the letter, and include or attach a Word, HTML, or PDF version resume.