About me

Michael L Perry

Improving Enterprises

Principal Consultant

@michaellperry

User login

Scaling collaborative applications with Azure and Correspondence

Azure is a great way to quickly deploy and scale your web sites. Correspondence is a rich framework for bringing collaborative experiences to multiple platforms. When these two come together, amazing things can happen.

The traditional web application has multiple stateless servers backed by a single database. You can scale out by adding new web servers, but this only goes so far if they all connect to the same database. Once you have saturated the database, where can you go?

The answer is to allow each web server to have its own local copy of the data, completely isolated from the other servers. Each new instance can serve requests without having to connect to an shared resource, like a SQL database. But how do you keep all of those local copies synchronized? That’s where Correspondence comes in.

imageCorrespondence

Correspondence is a collaborative application framework. It synchronizes data across several clients, including Windows Phone apps, WPF and Silverlight desktop apps, and Web sites. When you deploy a new instance of a Correspondence web site, it will pull down a copy of the application’s data locally. It serves all of its requests out of that local copy, so instances never compete for shared resources. And if the user changes the data on one server, it is pushed to all of the other servers so that they are kept up-to-date.

So where is that data coming from? And how is it being shared? At the heart of Correspondence is a distributor. Unlike a shared database, Correspondence applications don’t send every request to the distributor, so its load is much lighter. Instead, an application subscribes to the distributor. Each subscriber will receive changes as they occur. An application publishes data to the distributor, which in turn pushes it down to all of the other subscribers.

A web application instance is one such subscriber. When it receives data from the distributor, it stores it in the App_Data folder. Each Azure instance has its own App_Data folder. The Correspondence framework manages that App_Data folder and turns it into objects that the application can use.

MyCon

You can find an example of a Correspondence web application at https://github.com/MichaelLPerry/MyCon. Pull down the code using either Git or the Zip download link. You will need to enable NuGet Package Restore in order to build. This example app is running on Azure at http://awesomefest.azurewebsites.net/.

When you first run the solution, it will throw an exception indicating that you don’t have a conference ID. The exception contains a conference ID randomly generated for you. Just copy this conference ID into the web.config file and CommonSettings.cs.

MyCon is a template for a conference web site and phone application. It includes:

  • Silverlight 4 client for entering conference, speaker, and session data.
  • Web site for viewing that data.
  • Windows Phone application for attendees to use at the venue.

We will be concentrating on the web site. This project was built on MVC 3. It configures Correspondence to use the App_Data folder to store data. It also configures it to synchronize with an HTTP distributor. You will find that configuration in the SynchronizationService class:

public void Initialize()
{
    HTTPConfigurationProvider configurationProvider = new HTTPConfigurationProvider();
    string path = Path.Combine(HostingEnvironment.MapPath("~/App_Data"), "Correspondence");
    _community = new Community(FileStreamStorageStrategy.Load(path))
        .AddAsynchronousCommunicationStrategy(new BinaryHTTPAsynchronousCommunicationStrategy(configurationProvider))
        .Register<CorrespondenceModel>()
        .Subscribe(() => _conference)
        ;
}

The HTTP configuration provider loads the settings from web.config that identify the distributor:

    <add key="CorrespondenceAddress" value="https://api.facetedworlds.com/correspondence_server_web/bin" />
    <add key="CorrespondenceAPIKey" value="*******" />
    <add key="CorrespondencePollingIntervalSeconds" value="120" />

This distributor is hosted by my mobility company, Faceted Worlds. Feel free to use this host for your own instances of MyCon.

The last couple of lines of the configuration are registering the model and subscribing to the conference. This is where Correspondence is different from other frameworks. The model is only on the client, and the client subscribes to facts.

Facts

To make the data easier to distribute via publish/subscribe, Correspondence does not use a relational, document, or object-oriented model. Instead, it models data as historical facts. A historical fact is both a message and a record. As a message, it can be published and subscribed to. As a record, it can be queried.

The facts for the MyCon application are described in the file Model.fact. This file is located in the project FacetedWorlds.MyCon.Model under the WP7 folder. The other clients simply link to this one model. Facts are described as related data structures. For example:

fact RatingQuestion {
key:
    string text;
}

fact EssayQuestion {
key:
    string text;
}

fact Survey {
key:
    RatingQuestion *ratingQuestions;
    EssayQuestion *essayQuestions;
}

A fact can include references to other facts within its key. This acts like a foreign key in a relational database, allowing queries to join from the parent record to the child records. Parent facts are called “predecessors”, while child facts are called “successors”. That’s because predecessors come first in history.

In some cases the successor is published to its predecessor. For example:

fact ConferenceNotice {
key:
    publish Conference conference;
    time timeSent;
    string text;
}

When a successor is published, the application can subscribe to the predecessor. This will cause the distributor to push successors down to the application. In MyCon, each of the instances subscribes to the conference so that they all receive updates to conference data.

Deploy to Azure

Deploying a Correspondence web site to Azure is incredibly easy. First, sign up for an Azure account at http://windowsazure.com. Click on the “Portal” link and create a new Web Site. Use the “Quick Create” option. We won’t be creating a new web site with a database, since Correspondence will be our data store.

image image

Click on the arrow next to your application name to go to the application details. From here, click “Download publish profile”.

image

Back in Visual Studio, right-click on the FacetedWorlds.MyCon.Web project and select “Publish”. Click “Import” to import the publish profile that you downloaded from Azure. All of the settings that you need to publish your new site are copied into the dialog. Just click “Publish” and it will be uploaded for you. Once the site is published, it will open in a web browser.

Enter session data

Next, start up the FacetedWorlds.MyCon application in the Silverlight 4 folder. Using this application, you can enter a conference name, tracks, speakers, and sessions. Add rooms and times to the schedule grid, then drag the sessions onto the grid to arrange them.

As you enter data about the conference, it is published to the Correspondence distributor. The data is immediately pushed to all subscribers. Each Windows Phone running the conference app will subscribe to these changes. And so will each instance of the web site that you just deployed. Refresh the browser page to see that the web site has updated. It will serve the conference schedule from its own cache in App_Data. It does not need to hit a central database to serve a request.

Scale out

imageScaling out by adding new Azure instances is also extremely easy. Click on the “Scale” menu, move the slider over to allocate more instances, and then click “Save”.

When you add a new instance, it will subscribe to the conference. The distributor will send it all of the conference data entered so far. It will also set up a push notification channel for future changes. That way, whenever you change a session schedule using the Silverlight application, the change will be immediately reflected in the web application. It will be pushed to all instances of the web app so that they all remain synchronized.

Correspondence takes care of the hard problem of synchronizing data across independent instances of a web application. This allows the application to scale almost linearly as new instances are added. The instances don’t have a shared resource like a relational database that has to be available to respond to every request. Each instance can simply serve requests out of its own local copy in its App_Data folder.

Allocating new instances in Azure is a simple matter of moving a slider. As the big day approaches, ramp up the number of instances to handle new load. When the day has passed, ramp it back down to conserve resources. Correspondence and Azure are a powerful combination for creating scalable web sites.

For more information on how to build a Correspondence application, please visit http://correspondencecloud.com.