Marathon Data Systems, on the beautiful Jersey Shore, through our various vertical-focused brands, serves the people who provide the services that we all count on: HVAC, plumbing, lawn care, pest control, maid services, carpet cleaning, and more. We are in the process of building a mobility solution that will set the industry standard for field service worker and salesperson productivity.

Cross-Platform Mobile

Our new cross-platform mobile client is HTML5-powered and will initially run on iOS and Android devices. The technician or salesperson is continually operating on the latest information (no more periodic synchronization) and yet can be fully functional while out of network coverage.

The new mobile client is built in CoffeeScript on the Sencha Touch 2 platform. The combination of Sencha Touch and CoffeeScript making building off-line capable, singe-page applications actually pretty fun. Sencha takes care of the tedium of rendering just the right HTML to the screen for each device, while we focus  on building great user experiences.

Since WebSockets are not yet a reality (and pull-based applications are lame), we’re using SignalR to simulate server push from ASP.NET MVC (think Node.js scalability in ASP.NET MVC).

Message-Driven CQRS Back-End

On the back-end we are are building out a scalable, event-driven CQRS system built around NServiceBus, following the principles of Domain-Driven Design (DDD).

In Domain-Driven Design, we recognize that the heart of the software is the domain-specific behavior that allows its users to solve their problems. We establish a vocabulary that spans across technical and nontechnical people. We iteratively tweak and adjust our domain model to better map to the problem space. We focus on building an effective model of our domain, while minimizing entanglement with infrastructure concerns.

NServiceBus (NSB) allows us to focus on the semantic meaning of our commands and events and what the system’s behavior should be. NSB handles interaction with queues, message routing, retries, and other plumbing-level concerns.

Data persistence is achieved through the joy that is RavenDB. If you think ORMs like NHibernate are good, just wait until you have built a C# application with RavenDB.

RavenDB is a high-performance, second generation document database, native to C#. Queries are made through LINQ. Joins and transforms are handled through map/reduce functions also written in LINQ. The impedance mismatch between object-oriented code and a relational database is not merely bridged, but is eliminated.

Agile Team

Agile can mean almost anything you want it to mean, but at Marathon, it means…

A belief in the core values of the agile manifesto:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

The Scrum process, where a self-organizing team works together to plan and deliver iteratively and incrementally.

The Extreme Programming (XP) practices of test-driven development, pair-programming, continuous integration, collective code ownership.

A commitment to software craftsmanship. Being a professional means a commitment to quality work and continual improvement. Great software comes less from particular technologies that come and go, but from fundamentals like the SOLID principles and object-oriented design principles and patterns that stand the test of time.

This job may be for you if…

• You are good, but you expect to keep getting better.
• You can do it yourself, but you’d much rather work together with a team of dedicated developers.
• You love cool technologies, but you care about user loving and using your software more.
• You love coding in C#.
• If you are not currently test-driven, you want to learn to be.
• If you are not currently pair-programming, you are willing to give it a try.

If that sounds like you, please send your resume to jobs@marathondata.com. Come join us as we create great software to serve the people who serve the world!

Why should you even care?

Before getting into the review, why would you want to build a distributed, service-oriented system? In a word, scale.

Scaling the software: Asynchronous messaging and eventual consistency allows the system to scale out to redundant, cheap hardware. Brewer’s CAP theorem shows, centralized systems that rely on two-phase commits for all updates sacrifice reliability/availability in favor of an immediate consistency that isn’t always necessary.

Scaling the development team: A well-factored service-oriented system avoids the problem of a single monolithic system that becomes increasingly difficult to enhance and maintain and requires scale up of a single development team and resists scale out to multiple teams.

Review

The production quality leaves a lot to be desired. There is an annoying amount of background noise through all of the videos. The video is shot with a single camera that pans inconsistently between Udi, the whiteboard, and the projector screen. Honestly, you have to really want the content to make it through all 40 hours. However, you will be well rewarded.

The best way to describe the course is to think of it as a five day walk through all of the factors that led Udi to develop NServiceBus. I’ve followed Udi since way before NServiceBus. I listened to the Ask Udi podcast and attempted to grasp the essence of SOA. No matter how hard I tried, I was never able to implement the tenants of SOA with the traditional Microsoft tools and WCF. It wasn’t until I built a system with NServiceBus that I started to make the mental transition. Every time something I was trying to do was difficult and I was feeling that NServiceBus was overly constraining, I eventually realized it’s not you, it’s me. When you are doing it right, NServiceBus feels natural. When you violate good design for distributed, service-oriented systems, NServiceBus nudges you back by becoming difficult, causing you to go looking for a work-around, and eventually finding a post by Udi that gently explains why what you are trying to do would eventually bite you in the rear end and what you should be doing instead. This course lays the groundwork that will help you avoid the wrong turns and false starts you are bound to make otherwise if you are moving from the familiar world of centralized, monolithic systems.

He spends a significant portion of the course methodically and deliberately dislodging ideas that most of us have held for so long that we no longer question. He also works to temper and ground some of the techniques that have caught on with a small but vocal minority of.net developers, such as CQRS and event sourcing.

I recommend this course if any of these apply to you:

• I don’t know how to model autonomous services. Everything is related to everything else. My domain is too hard to tease apart into independent components. (Hint: this is hard at first, but Udi will take you through enough examples to get you headed in the right direction. Oh… and your domain is not too hard to do this.)
• I love Domain-Driven Design, but I don’t know how to model the domain and application layers. I recognize that I have fallen into the anemic domain model and all of my logic is in procedural application layer functions. (Hint: There is a place for simple CRUD and the rest cannot be modeled without sagas.)
• I like the idea of CQRS, but I don’t know how to approach it, or I tried and it didn’t work out as well as I expected. (Hint: Event sourcing is not the silver bullet of CQRS.)
• I am using NServiceBus, but I don’t understand sagas; so, I’m not using them. (Hint: you’re doing it wrong.)

All of these were true for me at some level. Getting these issues addressed made the course extremely valuable to me.

Conclusion

The entire team feels the course was a great investment and is entering this new project with anticipation and confidence that we will avoid many mistakes that we would have otherwise made if we had not taken the virtual course. I believe we will save the money we spent on the course and the time away from coding many times over on this project and beyond. If you are building a moderately complex collaborative system, you owe it to yourself to check out this course. If you can attend a live offering, I recommend that. But if you cannot, then the videos are well worth the investment as the next best thing.

The most popular structure for BDD today is called the Gherkin format and follows a Given/When/Then format, like…

Given a new bowling game
When all frames are strikes
Then the score should be 300

There are frameworks like SpecFlow to help you arrange your specifications (test) into this format. However, I find this format awkward and forced. I prefer the simpler format known as Context/Specification (aka When/Should)…

When all frames are strikes
Should have a score of 300

There are frameworks, like MSpec, that attempt to make the specifications read more like English sentences. However, I find that these frameworks get in the way as much as they help. It is also nice to be able to write readable tests with just PONU (plain old NUnit). Over time, I’ve developed a convention that I find easy to write and easy to read. I’ve also developed a tool  that turns the tests into a markdown file that can be turned into a pretty HTML report.

To show the approach at work, I present some snippets from a hypothetical order pricing system I created as a “developer test” provided by a prospective employer last summer. Here is what I was given:

You can see that there are quite a few specifications here. It’s a perfect scenario for a BDD approach. Lets take a look at the specification that most states charge taxes on the discounted price, while a few states require taxes to be calculated on the original price.

Here is a specification that a standard tax state calculates taxes on the discounted price…

namespace Acme.Tests.ConcerningCoupons
{
    [TestFixture]
    public class When_coupon_is_applied_to_item_on_order_in_standard_tax_state
    {
        private Order _order;
        [TestFixtureSetUp] public void Context()
        {
            Product product = new Product(10);
            Coupon coupon = CreateCoupon.For(product).WithDiscountOf(.5m);
            _order = CreateOrder.Of(product).Apply(coupon).In(StateOf.NC);
        }

        [Test] public void Should_calculate_tax_on_discounted_price()
        {
            _order.Tax.ShouldEqual(.25m);
        }
    }
}

You can see that the test fixture class name defines the context (the when) The test method name specifies the specification (the should). The Context method sets us the context in the class name. Also note the ConcerningCoupons in the namespace. This allows us to categorize the specification.

Here is the code that specifies the prediscount tax states…

namespace Acme.Tests.ConcerningCoupons
{
    [TestFixture]
    public class When_coupon_is_applied_to_item_on_order_in_prediscount_tax_state
    {
        private Order _order;
        [TestFixtureSetUp] public void Context()
        {
            Product product = new Product(10);
            Coupon coupon = CreateCoupon.For(product).WithDiscountOf(.5m);
            _order = CreateOrder.Of(product).Apply(coupon).In(StateOf.FL);
        }

        [Test] public void Should_calculate_tax_on_full_price()
        {
            _order.Tax.ShouldEqual(.50m);
        }
    }
}

Now take a look at a section of the report generated from the tests…

Anyone can now compare the generated report to the original specification to verify we hit the mark. It’s a little more work to structure your tests this way, but the benefits are worth it.

The full source for the sample and the report generator are available here.

Ever since I stumbled across the original C++ Report articles that have become known as the SOLID principles, I have been a disciple of Robert Martin (aka Uncle Bob). He is a leader within the agile and software craftsmanship movements. He has as good a sense of what makes good software as anyone currently writing and teaching. If he thinks it’s worth writing, then it’s worth reading.

I’m not usually a fan of video for learning. I like the random access referenceability of books and I like the on-the-go accessibility of audio recordings. I find video to be the worst of both worlds: I cannot flip to the a particular page or go at my own pace, nor can I consume it while driving or mowing the lawn. Further, there are so many good conference videos available for free that I find it difficult to justify paying for video content.

However, when Uncle Bob began releasing the Clean Code video series, I thought I’d at least check them out. I found them to be so good, that I’m now having my development team watch them together over lunch hours. Uncle Bob does a great job teaching (and preaching) the techniques that lead to clean, maintainable software. As we watch the sessions together, we are creating a common baseline of understanding that we can all refer to as we work together.

I highly recommend them for any software development team that is looking to get better (and if your team is not working to get better, it is on it’s way to obsolescence).  If you have trouble justifying the cost ($12 per viewer per video) to management, have them take a look at this excellent explanation of the value of software craftsmanship and professionalism.

I’ve written in the past about how the relentless march of progress in software has made yesterday’s innovations today’s commodity.

This article from HBR captures the essence of how the software landscape is in the process of making traditional, data-oriented, system-of-record, transactional systems into commodity systems. Companies that have made a living with these traditional systems are going to wake up very shortly and find that their customers have become someone else’s community participants.

The combination of ubiquitous mobile connection, cloud computing, and the general adoption of social media is in the process of changing the expectations of software users. Keeping their data safe is no longer enough. They now expect (or soon will) a more immersive experience.

This doesn’t mean a link to a Facebook fan page that is merely a shallow marketing ploy. Users expect that their hosted application will allow them to interact with all of the other uses of your software (see Spiceworks as the best example of this). They expect that the software they run their business on enlists them into the community of users of your software. Don’t have a “community” of users? Your competition soon will.

The Field Service domain is all about getting people to Locations to perform Activities. The Activities and the Location are defined by a Work Order. The person performing the Activities (and possibly the time) are defined by an Appointment. You can think of the Work Order as the what and where and the Appointment as the who and when.

The model is simple as long as there is one Work Order and one Appointment. In fact, you’d be tempted to combine them into one entity. However, things get more complicated when you have one Work Order that requires multiple Appointments to complete it. It might be two technicians at one time or the same technician on two different visits or multiple technicians over multiple visits.

In these cases, we are splitting the Activities of the Work Order over multiple Appointments. Some of the Activities are associated with one Appointment and some with another.

The Work Order is complete when all Activities over all Appointments are complete. Not too complicated.

Now also imagine that a technician goes out on one Appointment but services more than Work Order.

In simplest terms, we have a many-to-many between Appointments and Work Orders. You can imagine the twisted case where a given Appointment services two different Work Orders, and each of those Work Orders has other Appointments serving them.

We’ve all implemented many-to-many relationships in databases and in object models. However, this situation is a little different because of the role of the Activity. The linkage between the Appointment and the Work Order is through their association with a common Activity.

A given Activity is associated with one Work Order and one Appointment. We can now traverse that relationship to discover the relationship between Appointment and Work Order.

We simply have Work Orders with collections of Activities and Appointments with collections of Activities. With that, the complex relationship between Appointment and Work Order is completely modeled, with no redundant connections (like something directly linking Appointments and Work Orders). I thought this was pretty cool. And it was.

But here is where the problem comes in… In the real world, Appointments really are associated with Work Orders—and not just by the happenstance of common Activities. (One could argue this actually is true because you would only schedule an Appointment to service a Work Order if there was some Activity to perform, but this is not how anyone thinks about the relationship.)

With our model, to answer the question “Where does this Appointment take place?” we have to go to one of the Activities in our collection, navigate to its Work Order, and look at its Location property. This not a big deal to do in code, and we can even put a property on the Appointment that hides this messiness and gives the illusion that an Appointment has a Location. But this is not the only difficulty.

Try looking in our database to see what Work Order a given Appointment is associated with. You have to go to the Activity table and find all the Activities associated with this Work Order, then look at the Work Order column of those Activities to find the ID(s) of the Work Order(s). One day I tried to create a view that shows an Appointment and its first Work Order (there was only ever one Work Oder per Appointment in practice). I gave up after 20 minutes.

Now imagine explaining this model to a programmer trying to integrate with our system. Its possible, but there is much more confusion than you’d like.

We had a technically tight and elegant implementation, but it ended up obscuring one of the most fundamental relationships in our domain.

Remedy

As much as we liked the normalization of our current implementation, the team decided that it would be better to model the relationship between Appointment and Work Order explicitly. If we had focused more on the domain and less on the cleverness of the implementation, we could have avoided this rework of the design.

Two forces have converged: 1) Mobility has gone from an optional differentiator to an expected component of any software offering, and 2) HTML5 has been crowned as the solution that will solve the cross-platform problem that Java, Flash, and Silverlight failed to solve before it.

This collision of forces has turned HTML5 into a buzzword with a life of its own. In fact, it appears to be on its way to becoming as detached from reality as the all-time-champion of promising technology turned meaningless buzzword: SOA.

Don’t get me wrong, I believe HTML5 is the best current answer to cross-platform mobile software. Before recommending to my executive management that we build our cross-platform offering in JavaScript, using the family of features loosely known as HTML5, I looked at pure native, cross-platform native with Mono, frameworks like Titanium.  I made that recommendation before HTML5 became the cool thing to do, and I haven’t regretted it for a second.

A sure sign that a technology has reached fad level is when articles start to appear pointing out that said new technology will not, in fact, usher in world peace. Such an article was the popular 11 hard truths about HTML5.

The title was a little ominous and I began to read it with some trepidation, as we were still a couple of months away from being ready to ship our HTML5 client application. However, as I read, I was comforted by the fact that although all 11 truths were valid challenges, our team had faced and dealt with each of them.

The article lays out eleven challenges when building an HTML5 application:

1. Security is a nightmare.
2. Local data storage is limited
3. Local data can be manipulated.
4. Offline apps are a nightmare to sync.
5. The cloud owes you nothing.
6. Forced upgrades aren’t for everyone.
7. Web Workers offer no prioritization.
8. Format incompatibilities abound.
9. Implementations are browser-dependent.
10. Hardware idiosyncrasies bring new challenges.
11. Politics as usual.

It’s a scary list, and they are all true. My team was able to handle and mitigate each of these challenges largely because we had years of experience building native mobile applications and desktop Web applications and much of what we learned there applied in the HTML5 world.

If you don’t have solid answers for each of these challenges, you really ought to get someone on the team who has confidence in dealing with each of them.

You can check my current availability here.

This article explores the physiological and psychological effects of fatigue brought on by making decisions. The fatigue that comes from making decision after decision immediately reminded me of my team’s Scrum sprint planning days.

The Scrum method breaks software development into iterative cycles called sprints. Our sprints were the highly typical two weeks in length. The idea is that two weeks is a short enough planning horizon that we can pull in enough work from the backlog to fill that time period. Then we demo what we’ve done to the various stakeholders in the company, adjust existing backlog priorities, plan another sprint, and on it goes.

Sprint planning day looked something like this…

9:00 Demos (any stories that haven’t been shown yet)

9:30 Close out the Previous Sprint (closing stories in VersionOne, splitting any unfinished stories, etc.)

10:00 Retrospective (look back over the prior sprint and identify things that worked well that we want to do more of, what didn’t work so well, and identify any impediments to progress)

10:45 Start Sprint Planning (Story Breakdown)

1:30 Finish Sprint Planning (Story Breakdown)

We often wouldn’t finish sprint planning until after 4:30. 

Story Breakdown

Sprint planning is the process of taking the high-level stories and breaking them down into tasks. We did this as a team; so, we had everyone’s input and everyone knew how we were going to go about implementing each story. This is vital to maintaining a team approach to building the product.

This story breakdown, however, is the hardest part of the whole sprint. We have to make decision after decision about how we are going to implement a feature…

Will there be a new database table? Will it be a variant of some existing feature or something new? Is there some new UI element that we haven’t tackled before? and so on.

Then for every decision, we have one more decision: how long do we think it will take.

I believe the hardest part is that we move from one decision to the next without actually doing anything. We are simply adding our decisions to the inventory to be acted on over the next two week. This makes the decision fatigue factor even greater.

By the time we got to 3:00 or 3:30 the team would often be so fatigued that we would start placing two tasks on each story: Plan it and Do it. During the sprint, if we came across a story with a “Do it” task, it was a safe bet that it was planned late in the day.

Remedy

I can see two ways to reduce the decision fatigue that comes with Scrum planning day: 1) reduce the Sprint length, or 2) don’t do Scrum.

For most teams doing Scrum, I think shrinking the Sprint length to one week will reduce the planning day fatigue. Our team, for other reasons, switched to a Kanban continuous flow model. Under that model, we did the story breakdown as the queue of planned stories got low. It was never two weeks worth at one time, and we had fewer “Do it” tasks.