Getting Started with the RavenDB Index Replication Bundle

imageBeing able to push Raven data out to a SQL-based data repository for reporting will be a huge boost to RavenDB adoption. The Index Replication Bundle does a great job of solving that problem, but I found the documentation to be a little weak.

I created a minimal program that implemented it for myself and decided to post it to GitHub so someone else looking to experiment with the Index Replication Bundle can get started more smoothly. This Stack Overflow question was helpful along the way.

This sample has a simple Raven store with a collection of trivial documents. Those documents are directly reflected out to SQL Server for reporting purposes. Once you have all of the necessary plumbing worked out, making the mapping between Raven and SQL Server more sophisticated is just a matter of making the indices more complicated.

Install The Plugin

Create a “Plugins” directory under the directory where Raven.Server.exe resides. Copy Raven.Bundles.IndexReplication.dll there…

image

Configure the Connection String

Add a connection string to Raven.Server.exe.config…

<connectionStrings>
     <add
       name="Reports"
       providerName="System.Data.SqlClient"
       connectionString="Data Source=(local);Initial Catalog=Sample;Integrated Security=SSPI;"/>
</connectionStrings>

Create a database in SQL Server called “Sample”. The rest of the steps are implemented in the sample program.

Create Table

A bit of code to create the table in SQL Server…

using (SqlConnection connection = new SqlConnection(_sqlConnectionStringTextBox.Text))
{
    connection.Open();
    SqlCommand createTable = new SqlCommand(@" CREATE TABLE [dbo].[Dogs]( [Id] nvarchar(64) NULL, [Name] nchar(255) NULL, [Breed] nchar(255) NULL) ON [PRIMARY]", connection);
    createTable.ExecuteNonQuery();
    connection.Close();
}

 

 

Create Index

Next, we create a Raven index…

public class Dogs_List : AbstractIndexCreationTask<Dog>
{
    public Dogs_List()
    {
        Map = dogs => from d in dogs select new { Breed = d.Breed, Name = d.Name };
    }
}

 

 

Configure Replication

Applying the replication configuration described in the docs was not clear to me. It is simple once you see it, but not necessarily obvious…

var replicationDocument = new Raven.Bundles.IndexReplication.Data.IndexReplicationDestination
    {
        Id = "Raven/IndexReplication/Dogs/List",
        ColumnsMapping =
            {
                { "Name", "Name" },
                { "Breed", "Breed" }
            },
        ConnectionStringName = "Reports",
        PrimaryKeyColumnName = "Id",
        TableName = "Dogs"
    };

using (var store = new DocumentStore { Url = _ravenUrlTextBox.Text })
{
    store.Initialize();
    store.DatabaseCommands.EnsureDatabaseExists(RavenDatabaseName);
    using (var session = store.OpenSession(RavenDatabaseName))
    {
        session.Store(replicationDocument);
        session.SaveChanges();
    }
}

 

 

Add Items to Raven

Now, once you add items to Raven, they should appear in the Dogs table in SQL Server.

If you don’t see them, try turning debug logging on.

Help Turn the “Jersey Shore” into “Silicone Shore”

SiliconeShoreWant to work with the latest technology on a self-directed team? Think you need to be in Silicone Valley to be part of a company setting the standard in cloud and mobile computing? Read on.

Marathon Data Systems, on the beautiful Jersey Shore, through our various vertical-focused brands, serves the people who provide the services that we all count on: HVAC, plumbing, lawn care, pest control, maid services, carpet cleaning, and more. We are in the process of building a mobility solution that will set the industry standard for field service worker and salesperson productivity.

Cross-Platform Mobile

Our new cross-platform mobile client is HTML5-powered and will initially run on iOS and Android devices. The technician or salesperson is continually operating on the latest information (no more periodic synchronization) and yet can be fully functional while out of network coverage.

SenchaPlusCoffee

The new mobile client is built in CoffeeScript on the Sencha Touch 2 platform. The combination of Sencha Touch and CoffeeScript making building off-line capable, singe-page applications actually pretty fun. Sencha takes care of the tedium of rendering just the right HTML to the screen for each device, while we focus  on building great user experiences.

Since WebSockets are not yet a reality (and pull-based applications are lame), we’re using SignalR to simulate server push from ASP.NET MVC (think Node.js scalability in ASP.NET MVC).

Message-Driven CQRS Back-End

DomainDrivenDesignOn the back-end we are are building out a scalable, event-driven CQRS system built around NServiceBus, following the principles of Domain-Driven Design (DDD).

In Domain-Driven Design, we recognize that the heart of the software is the domain-specific behavior that allows its users to solve their problems. We establish a vocabulary that spans across technical and nontechnical people. We iteratively tweak and adjust our domain model to better map to the problem space. We focus on building an effective model of our domain, while minimizing entanglement with infrastructure concerns.

nServiceBus_LogoNServiceBus (NSB) allows us to focus on the semantic meaning of our commands and events and what the system’s behavior should be. NSB handles interaction with queues, message routing, retries, and other plumbing-level concerns.

ravendb

Data persistence is achieved through the joy that is RavenDB. If you think ORMs like NHibernate are good, just wait until you have built a C# application with RavenDB.

RavenDB is a high-performance, second generation document database, native to C#. Queries are made through LINQ. Joins and transforms are handled through map/reduce functions also written in LINQ. The impedance mismatch between object-oriented code and a relational database is not merely bridged, but is eliminated.

Agile Team

Agile can mean almost anything you want it to mean, but at Marathon, it means…

Agile at Marathon

A belief in the core values of the agile manifesto:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

The Scrum process, where a self-organizing team works together to plan and deliver iteratively and incrementally.

The Extreme Programming (XP) practices of test-driven development, pair-programming, continuous integration, collective code ownership.

A commitment to software craftsmanship. Being a professional means a commitment to quality work and continual improvement. Great software comes less from particular technologies that come and go, but from fundamentals like the SOLID principles and object-oriented design principles and patterns that stand the test of time.

This job may be for you if…

  • You are good, but you expect to keep getting better.
  • You can do it yourself, but you’d much rather work together with a team of dedicated developers.
  • You love cool technologies, but you care about user loving and using your software more.
  • You love coding in C#.
  • If you are not currently test-driven, you want to learn to be.
  • If you are not currently pair-programming, you are willing to give it a try.

If that sounds like you, please send your resume to jobs@marathondata.com. Come join us as we create great software to serve the people who serve the world!

Shout Out to Doxygen

Software Architecture is largely a matter of communication. One big area of communications is documenting a library, whether for internal or external consumption. It is well-known that written documentation goes out of date the minute you publish it.

The only hope of keeping the documentation up-to-date is to have the it embedded within source code it is illuminating. This allows you to update the doc as you update the code. The Javadoc and C# XML comments are good for this, but don’t give you a good way to provide anything beyond documentation of classes and methods, which often falls short in properly documenting a library or code base.

Doxygen is a capable, open source library that combines the comment-based documentation with a full mark-up language that allows you to add text, diagrams, pages, sections, lists, etc. that allows for robust documentation.

Some of my favorite features include:

  • Auto-generated class and collaboration diagrams
  • The ability to include hyperlinked source
  • Compiled Help format
  • PDF format

Update Route Tables on Windows Mobile

Our dev team needed to do something that, evidently, very few people need to do… update the routing tables on a Windows Mobile device that is simultaneously connected to both WiFi and the wireless carrier. Internet searching was very little help, but we eventually stumbled across a nifty little shareware app called PocketLAN by z2 Software. If you are doing any advanced networking on a Windows Mobile device, PocketLAN is well worth the $14.99.

Spell-Checking for ReSharper

Your code will be written once. It will be read (by you and others) many times. Code should be easy to read. This means that I should be able to scan a screen full of code and get a sense of what it does quickly. Layout and spacing plays s big part in that. Naming things with real, recognizable words just as important.

If you buy that, it goes without saying that those words that make up your identifiers must be spelled correctly. I am a terrible speller. As a C++ coder, I loved that Visual Assist would give me the red squiggles under any word that I misspelled. This has been the only regret I have about moving from VA to ReSharper. In every other way, ReSharper is a win over what VA offered.

I’ve just discovered Agent Smith as a plug-in to ReSharper. I had to make a few tweaks and drop one of the checking rules, but I am loving it. My ReSharper experience is now complete!