I took the last two weeks of 2011 off work. During that time, I ported Correspondence to Windows 8, published HoneyDo List for Windows Phone, and submitted an entry for the First Apps contest. Here is a demo of HoneyDo List running on a Windows 8 table that Chris Rajczi generously loaned me.

HoneyDo List uses Zebra Crossing to generate and recognize QR codes. These codes contain a unique ID to which both devices can publish and subscribe.

The tablet publishes all of its lists to the ID, and subscribes for updates. The phone, upon scanning the ID, subscribes to receive all of the tablet’s lists, and then publishes the ones that the user selects.

This two-way collaboration continues after the lists are exchanged. Both devices subscribe to the shared lists. New tasks published on one device will be received on the other.

It is my hope that the story of Correspondence – synchronizing facts among all your devices via publish/subscribe – will align with the story of Windows 8 – access to all of your applications and data wherever you happen to be. If so, I expect this to be the start of an exciting opportunity.

.NET 4.5 introduces the async and await keywords. In the very near future (Windows 8, WinRT), most API functions will be asynchronous. Code that you wrote using synchronous APIs will no longer work.

For example, suppose you used to write to a file like this:

   1: lock (this)

   2: {

   3:     FileHandle file = FileHandle.Open();

   4:     file.Write(value);

   5:     file.Close();

   6: }

Now you have to write to it using the asynchronous API. Just changing it to this won’t work:

   1: lock (this)

   2: {

   3:     FileHandle file = await FileHandle.OpenAsync();

   4:     await file.WriteAsync(value);

   5:     file.Close();

   6: }

This code fails to compile with the error “The 'await' operator cannot be used in the body of a lock statement”. So you get rid of the lock:

   1: // BAD CODE! Multiple writes allowed.

   2: FileHandle file = await FileHandle.OpenAsync();

   3: await file.WriteAsync(value);

   4: file.Close();

And now it works most of the time. But every once in a while, you get two writes to the same file. If you’re lucky, this will just result in an exception like “The process cannot access the file because it is being used by another process”. If you aren’t, this will corrupt your file.

The solution

So how do you protect a shared resource, such as a file, without blocking the thread? Enter the AwaitableCriticalSection.

The call to EnterAsync() will only return for one caller at a time. It returns an IDisposable, so once you leave the using block, it will allow the next caller to enter. Threads are not blocked, but only one caller can be in the critical section at a time.

   1: using (var section = await _criticalSection.EnterAsync())

   2: {

   3:     FileHandle file = await FileHandle.OpenAsync();

   4:     await file.WriteAsync(value);

   5:     file.Close();

   6: }

Download the source code and add it to your project. Feel free to change the namespace.

How it works

The AwaitableCriticalSection keeps a collection of Tokens. Each Token represents one call to EnterAsync(). A token implements IAsyncResult. It’s basically just a wrapper around a ManualResetEvent, which can be signaled when it’s your turn to enter the critical section.

Think of a Token as a “take-a-number” ticket, or a restaurant pager. AwaitableCriticalSection is the dispenser, or the hostess. When you call EnterAsync(), it dispenses a Token to you. At the same time, it places this token in a queue.

If there is no one else in the critical section at that moment, then your token is instantly signaled. It passes true to the Signal method so that CompletedSynchronously will be set. This causes await to continue without relinquishing the thread.

   1: public Task<IDisposable> EnterAsync()

   2: {

   3:     lock (this)

   4:     {

   5:         Token token = new Token();

   6:         _tokens.Enqueue(token);

   7:         if (!_busy)

   8:         {

   9:             _busy = true;

  10:             _tokens.Dequeue().Signal(true);

  11:         }

  12:         return Task.Factory.FromAsync(token, result => _disposable);

  13:     }

  14: }

Await will open up the async result, calling the lambda expression that returns _disposable. This is simply an object that calls Exit when you leave the using block.

   1: private void Exit()

   2: {

   3:     lock (this)

   4:     {

   5:         if (_tokens.Any())

   6:         {

   7:             _tokens.Dequeue().Signal(false);

   8:         }

   9:         else

  10:         {

  11:             _busy = false;

  12:         }

  13:     }

  14: }

Exit signals the next token in line. This time, it passes false to let you know it didn’t complete synchronously. It had to wait. If there is nobody left in line, then it sets _busy to false. There is nobody in the critical section.

Locks block threads. New operating system APIs won’t allow you to block threads. So use an awaitable critical section to protect shared resources while maintaining a fast and fluid UI.

The Windows Phone version of the HoneyDo List app is finished and has been submitted to the Marketplace. Here’s a demo of the list sharing feature.

Next I’ll start on the Windows 8 version. The phone app will make a good companion to the desktop or tablet app.

The multi-platform HoneyDo List application starts with the web. This version of the application, hosted on App Harbor, let’s you share lists between devices using your phone’s camera. Create a list in your browser, then share it via QR code with your phone. The lists will be kept in sync. Here’s a demo.

My next step is to get the Windows Phone companion app reading these QR codes and sharing lists. Once that’s submitted to the Marketplace, I’ll start working on the Windows 8 app.

Correspondence queues don’t work like your typical queues. A subscriber sees the entire history of facts that were published to the queue. Retrieving a fact does not remove it from the queue.

While this is a good thing for synchronization among multiple parties, it does have one drawback. When you first encounter a queue, you have to suffer through all of history before you get to the facts that you are really interested in.

To eliminate this problem, I added a feature to Correspondence that allows it to unpublish a fact. When the fact is no longer important, it does not appear in the queue. Here’s a demonstration of the feature.

To unpublish a fact, simply define a where clause in the publish statement. The where clause needs to use a predicate directly on the published fact.

fact Task {
key:
    publish ListContents listContents
        where not this.isComplete;

query:
    bool isComplete {
        exists TaskComplete c : c.task = this
            where not c.isUndone
    }
}

A Task will only be published to ListContents as long as it is not complete. Once the task is marked completed, it will be unpublished. People subscribing to the list later will not see any of the completed tasks.

Microsoft is sponsoring a Windows 8 app building contest. Entries are due January 8. I have two weeks off for Christmas and New Years. So I know what I’ll be doing.

For the next two weeks, I will keep regular office hours. By the end of the two weeks, I will have a HoneyDo List Metro application and a companion Windows Phone application.

I’m starting with:

Correspondence for Silverlight, Web, and Windows Phone

Bare bones HoneyDo List for Silverlight

Bare bones HoneyDo List for Windows Phone started at the Mobile Hackathon at the AT&T Foundry in Plano

Bare bones HoneyDo List for Web running on App Harbor

My tasks are:

Implement un-publishing in Correspondence

Port Correspondence to Windows 8 Metro

Display and edit lists in Windows 8 Metro

Display a QR Code in Windows 8 Metro

Consume the QR Code in Windows Phone

Display a QR Code in Windows Phone

Submit Windows Phone app to the Marketplace

If I have time left over, I’ll spend it styling the web application.

Follow my progress on Twitter. I will also post regular video demonstrations on this blog.

Grady Booch is creating a TV series called Computing, and he needs your help.

I want to donate $50 to the project, but you can help me to give more. Here's what you do:

1. Create a Visual Studio unit test project
2. Add a NuGet reference to Correspondence.model
3. Click Transform all templates
4. Add a NuGet reference to Correspondence.Desktop.UnitTest
5. Run all tests
6. Write a new test
7. Email it to mperry@mallardsoft.com with the subject "Correspondence Unit Test"

For the first 5 tests, I will donate $10 each. For the next 10, $5 each. Then $1 for each test thereafter up to a total of $500. All entries must be in by December 20, at which time I will post the best tests.

Our industry needs this documentary. Let's make it happen.

Correspondence was originally launched as a solution for occasionally connected smart clients. It provided local storage, queuing, and synchronization for Silverlight and Windows Phone applications. This satisfied some common scenarios (collaborative mobile apps, out-of-browser business applications, etc.), but left a significant gap.

Correspondence 1.3 closes that gap by adding support for MVC 3. As the back-end data store for a web application, Correspondence enables many more use cases. Some of these use cases are difficult to achieve with other alternatives.

Cloud application, local services

A cloud-based application works fine in an isolated silo. But it is much more difficult to integrate a cloud application into an enterprise solution, especially when mission-critical business processes run on-premises. Correspondence brings the cloud and enterprise together.

A cloud-hosted web application connects to a Correspondence Synchronization Server, just as a client would. The web application has its own local cache of the shared data, but it continuously synchronizes that cache with other consumers. A service located within the enterprise can subscribe to facts published on the web, and publish its results back to the web application.

For example, suppose you wanted to host an e-commerce web application in the cloud. This would give you the elasticity you require to scale up during peak load, and back down again as demand falls off. However, you want your ERP system to run in-house. With Correspondence, your web application publishes orders that the customers place on-line. Your ERP system subscribes to those orders, fulfills them, and publishes status. Users log on to the web site to check order status.

Continuous backup of your data

A web application, whether hosted in the cloud or in your own datacenter, is an active accumulation of data that must be regularly backed up. How frequently should you back up your database? How much data loss can you tolerate? How quickly can you get back on-line after restoring from backup?

If your web application is backed by a Correspondence Synchronization Server, then your data is backed up continuously. You don’t have to plan downtime for maintenance to take a snapshot of your database. And when your server goes down, simply stand up another one and point it to the same Synchronization Server. It will pull down all of the data.

Better yet, have the backup standing by live, pointing at the same Synchronization Server as the production system. Data will be automatically synchronized between the two systems. When production goes down, switch over to your hot backup and minimize the interruption in business. And when you can restore the production system, it will synchronize and pick up exactly where the hot backup left off.

Graceful degradation of UI

Correspondence works well for WPF, Silverlight, and Windows Phone applications. These cover a large number of platforms, including Windows, Mac, and Phone. But what if you want to run on an iPhone, iPad, or Android device? What if the user you want to reach doesn’t have Silverlight installed?

With a Correspondence MVC 3 application, you can create a web alternative for your rich application. Go ahead and serve the Silverlight client on the web site. But if the user doesn’t have or cannot run the browser plug-in, replace the static Download Silverlight image with an HTML-based experience.

Rich client integration without a custom API

Many of the popular apps on mobile devices are rich front-ends to web applications. Twitter, Foursquare, Facebook, and TripIt are all great examples. Each of these web applications has a custom REST API designed for use by mobile applications.

If you want to build a web application with a rich mobile client, don’t spend your time writing REST APIs. Use Correspondence as both your web and mobile application back end. The app will synchronize with the web through the Correspondence Synchronization Service.

Reporting against an off-line database

Your web application is collecting valuable data. You want to report on it. But the application database is not optimized for reporting. And any reports that you run against the application database will hurt production performance.

Correspondence 1.3 includes a new SQL Server storage strategy. This is a great choice for caching data in the web application, but it also works well as a back-end for an on-premises reporting server. Write a service that subscribes to facts, and turns them into inserts, updates, and deletes against a relational store. Run your reports against a local relational database for the best possible performance without impacting transaction processing in production.

Engage with your site’s visitors using rich applications

Visitors to your site are only there for a short time. You need to entice them to come back. The best way to do that is to engage your visitors in a collaborative workflow. They leave an email address, and you give them regular updates. They leave a comment, and you respond.

If you rely upon traditional tools, you will be using the same web site as your visitors. But while they are casual guests, you are a permanent resident. You should be able to take advantage of a rich, responsive application.

A Correspondence-backed web application synchronizes with a rich Correspondence application. You can build the best possible experience for yourself, and still get the reach that you need to engage with your customers.

These are just some of the use cases that Correspondence for the web enables. Watch this short video for a demonstration, and visit Correspondence to get started building your own collaborative site.

Correspondence for the Web and WPF from Michael L Perry on Vimeo.

Correspondence is a framework for creating collaborative applications. This makes for some interesting unit testing.

[TestClass]
public class ModelTest
{
    private Community _communityFlynn;
    private Community _communityAlan;
    private Identity _identityFlynn;
    private Identity _identityAlan;

    [TestInitialize]
    public void Initialize()
    {
        var sharedCommunication = new MemoryCommunicationStrategy();
        _communityFlynn = new Community(new MemoryStorageStrategy())
            .AddCommunicationStrategy(sharedCommunication)
            .Register<CorrespondenceModel>()
            .Subscribe(() => _identityFlynn)
            .Subscribe(() => _identityFlynn.MessageBoards)
            ;
        _communityAlan = new Community(new MemoryStorageStrategy())
            .AddCommunicationStrategy(sharedCommunication)
            .Register<CorrespondenceModel>()
            .Subscribe(() => _identityAlan)
            .Subscribe(() => _identityAlan.MessageBoards)
            ;

        _identityFlynn = _communityFlynn.AddFact(new Identity("flynn"));
        _identityAlan = _communityAlan.AddFact(new Identity("alan"));
        _identityFlynn.JoinMessageBoard("The Grid");
        _identityAlan.JoinMessageBoard("The Grid");
    }

    [TestMethod]
    public void InitiallyNoMessages()
    {
        Assert.IsFalse(_identityAlan.MessageBoards.Single().Messages.Any());
    }

    [TestMethod]
    public void FlynnSendsAMessage()
    {
        _identityFlynn.MessageBoards.Single().SendMessage("Reindeer flotilla");

        Synchronize();

        Message message = _identityAlan.MessageBoards.Single().Messages.Single();
        Assert.AreEqual("Reindeer flotilla", message.Text);
    }

    private void Synchronize()
    {
        while (_communityFlynn.Synchronize() || _communityAlan.Synchronize()) ;
    }
}

75 years ago this month, Alan Turing published a math paper that described a theoretical computer. This “computing machine” was capable of running a program constructed from simple state transitions, but performing complex calculations. This was the foundation of computer software as we know it today.

All of the capabilities and limitations of software are found in Turing’s first machine. Others have proven that any software written in our modern programming languages can be executed by a Turing Machine. And the converse is also true. So any limitation of the Turing Machine is a limitation of all software. This is a property known as Turing Completeness.

Dependency Tracking

The first Turing Machines were deterministic. The actions of a deterministic Turing Machine are based only upon the internal state of the machine and the symbol that it is scanning. If the machine never scans a symbol, then changing that symbol would have no impact on the behavior of the machine. Since this is true of a Turing Machine, it is true of all software.

If you flip this around, you can see how to track dependencies within a software system. If you need to calculate one number (say the balance of your checking account), you can write a program to scan a bunch of other numbers (the amounts of your checks) and come up with an answer. The behavior of the machine, and therefore the final outcome, depends upon the scanned values. If you were to change a value somewhere in the computer’s memory that it didn’t scan, it would still come up with the same value.

The balance of your checking account depends upon the amounts of all your checks. You can tell by keeping track of which symbols the machine scans while it’s making that calculation. If any of those symbols is changed, then the balance needs to be recalculated. What if the amount of a check is modified? The machine scanned the amount of that check while calculating the balance, so it needs to be recalculated. What if a new check is added? The machine at some point scanned a symbol that told it that it was done with the list of checks. To add a new check, you must have modified this symbol. So, again, the balance would need to be recalculated.

Dependency tracking systems keep a record of every symbol that a program scans while it is calculating a value. Then, when one of those scanned symbols changes, it knows that the value needs to be recalculated. I know of two such dependency tracking systems: Knockout.js for JavaScript, and Update Controls for .NET.

Determinism can always be converted into dependency. If it is true for a Turing Machine, then it is also true of any modern programming language. That’s the advantage of knowing that a language is Turing Complete.