Getting Started with the RavenDB Index Replication Bundle

imageBeing able to push Raven data out to a SQL-based data repository for reporting will be a huge boost to RavenDB adoption. The Index Replication Bundle does a great job of solving that problem, but I found the documentation to be a little weak.

I created a minimal program that implemented it for myself and decided to post it to GitHub so someone else looking to experiment with the Index Replication Bundle can get started more smoothly. This Stack Overflow question was helpful along the way.

This sample has a simple Raven store with a collection of trivial documents. Those documents are directly reflected out to SQL Server for reporting purposes. Once you have all of the necessary plumbing worked out, making the mapping between Raven and SQL Server more sophisticated is just a matter of making the indices more complicated.

Install The Plugin

Create a “Plugins” directory under the directory where Raven.Server.exe resides. Copy Raven.Bundles.IndexReplication.dll there…

image

Configure the Connection String

Add a connection string to Raven.Server.exe.config…

<connectionStrings>
     <add
       name="Reports"
       providerName="System.Data.SqlClient"
       connectionString="Data Source=(local);Initial Catalog=Sample;Integrated Security=SSPI;"/>
</connectionStrings>

Create a database in SQL Server called “Sample”. The rest of the steps are implemented in the sample program.

Create Table

A bit of code to create the table in SQL Server…

using (SqlConnection connection = new SqlConnection(_sqlConnectionStringTextBox.Text))
{
    connection.Open();
    SqlCommand createTable = new SqlCommand(@" CREATE TABLE [dbo].[Dogs]( [Id] nvarchar(64) NULL, [Name] nchar(255) NULL, [Breed] nchar(255) NULL) ON [PRIMARY]", connection);
    createTable.ExecuteNonQuery();
    connection.Close();
}

 

 

Create Index

Next, we create a Raven index…

public class Dogs_List : AbstractIndexCreationTask<Dog>
{
    public Dogs_List()
    {
        Map = dogs => from d in dogs select new { Breed = d.Breed, Name = d.Name };
    }
}

 

 

Configure Replication

Applying the replication configuration described in the docs was not clear to me. It is simple once you see it, but not necessarily obvious…

var replicationDocument = new Raven.Bundles.IndexReplication.Data.IndexReplicationDestination
    {
        Id = "Raven/IndexReplication/Dogs/List",
        ColumnsMapping =
            {
                { "Name", "Name" },
                { "Breed", "Breed" }
            },
        ConnectionStringName = "Reports",
        PrimaryKeyColumnName = "Id",
        TableName = "Dogs"
    };

using (var store = new DocumentStore { Url = _ravenUrlTextBox.Text })
{
    store.Initialize();
    store.DatabaseCommands.EnsureDatabaseExists(RavenDatabaseName);
    using (var session = store.OpenSession(RavenDatabaseName))
    {
        session.Store(replicationDocument);
        session.SaveChanges();
    }
}

 

 

Add Items to Raven

Now, once you add items to Raven, they should appear in the Dogs table in SQL Server.

If you don’t see them, try turning debug logging on.

On Summer Internships

SummerInternship

I had my first experience with hiring a summer intern this summer. It turned out to be a great experience, but I credit this to the fact that I work with a great team and our intern was a stellar young guy. I realized that I need to have a plan for future internships. First though, I need to clarify my philosophy of internships. In other words, I have to answer the question… Why have summer internships?

Future Hires

Summer interns make great hires. What better or more thorough interview process than working with someone for three months? Finding and eventually hiring great developers is reason enough to have an internship program.

Networking

At the end of the summer, those interns will go back to school. That is where many of tomorrow’s great developers are hanging out. What better way to reach those future developers than to send students back to school having had a fantastic experience at your company?

Giving Back

A summer internship gives your developers an opportunity to have a positive influence on future developers. Even your most junior developers get to become mentors to an intern. Giving back in this way reinforces the value that software development is a profession and we are professionals. Taking the time to invest in the career of a young software developer is one way to separate professional developers from hackers.

Getting Work Done

I put this last because I think it is the least important of the list, but you just might find that that your interns do some really great work. My team was fortunate enough to have such a great young developer this summer that the work he contributed to was outstanding. Fresh perspective and ideas can be very valuable to a team.

Copy and Paste From Any Kindle

I have always wished I could copy and paste text from a Kindle book for quoting in a blog post or email. I could understand that this might not be possible from a Kindle device, but certainly it should be easy from my iPad or the Kindle cloud reader, right? Wrong!

A friend of mine wrote a great article on how to remember more of what you read on Kindle. He showed how kindle.amazon.com will display a page with all of your highlights. This gave me a great idea… simply highlight the passage of interest, allow the reader to sync, and viola… you have the highlighted text on a Web page that you can copy and paste from. This even works with Kindle devices. Nice!

Audio Book Recommendations

I make use of my commute time to listen to podcasts and audio books from audible.com. A coworker recently asked me about my favorite titles. Here are some of my favorites that would be of interest to software professionals.

Developing Software

The Lean Startup by Eric Ries: This is a fantastic book about how to develop a product in a lean/agile manner. Ries’s description of the build-test-learn loop is worth the price of the book.

 

Rework by Jason Fried and David Heinemeier Hansson: I found some interesting stuff here from the 37signals guys. My biggest complaint about this book is that the authors are way to quick to assume that all of their success is due to the choices they made. Just because they did something and had some success with it does not mean that the success was caused by the particular behavior in question.

Product Management

The Innovator’s Dilemma by Nathan Christensen: Christensen describes how disruptive technologies overthrow established technology when those incumbents believe they are least vulnerable to such a disruption.

The Innovator’s Solution by Nathan Christensen: Christensen continues where he left off. Another must-read for anyone who manages a technology product or company.

 

The Design of Everyday Things by Donald Norman: Norman wrote this back in 1988 (and you can tell), but it is an enduring classic and a product management must-read for good reason. A surprisingly fun listen.

General/Management

Good to Great by Jim Collins: This is one of my favorite books of all time in any category. Collins describes the common attributes his research team found behind successful company leaders. I found the book to be quite inspiring and have made the attributes of leadership Collins describes personal goals for my own leadership.

Leading Change by John Kotter: Kotter describes eight steps to successful organizational change. I’ve seen these principles play out in commercial and social settings—sometimes where the leaders follow these steps and create effective change, but more often where they did not and the change efforts failed.

The Goal by Eli Goldratt: I first read The Goal in the late 1980s as an Industrial Engineering student. It is a good read and helps to clarify the folly of local optimization–in manufacturing or any complex system. This has become a favorite of proponents of Lean Software Development.

There are many more audio books that have really enjoyed, but I’ll save those for another day.