About me

Michael L Perry

Improving Enterprises

Principal Consultant

@michaellperry

User login

75 years of software

q1

None

R

q2

q2

None

R

q3

q3

None

P0

R

q4

q4

None

R

q5

q5

None

P1

R

q6

q6

None

R

q7

q7

None

P0

R

q8

q8

None

R

q9

q9

None

P0

R

q10

q10

None

R

q11

q11

None

P1

R

q12

q12

None

R

q13

q13

None

P0

R

q14

q14

None

R

q15

q15

None

P1

R

q16

q16

None

R

q17

q17

None

P1

R

q18

q18

None

R

q3

This month marks the 75 year anniversary of the publication of Alan Turing’s paper On Computable Numbers, with an Application to the Entscheidungsproblem. In that paper, Turing defined a universal computing machine, a theoretical construct capable of running programs. He didn’t set out to invent software. But that’s exactly what happened.

The Ensheidungsproblem

Turing’s goal was to show that there are statements in mathematics that are well-formed, but not decidable. That isn’t to say unprovable. That is to say that we can’t even decide whether a proof exists. This is an extremely important mathematical proof, but what is more revolutionary was the way in which Turing proved it.

Turing invented a machine that is capable of running programs. Each program prints out the digits 0 and 1, in addition to possibly doing some record keeping. A program is a finite series of instructions in a state transition table like the one above. If you “compile” the program, you will get an integer. It’s a really big integer, but it is finite. This lets you count the programs and put them in order.

Now, suppose that you had a program that would run all of these programs in order, but stop the nth program when it prints the nth digit. Would this program keep printing digits, like any good Turing program should? Or would it seize up?

If every program compiles to a finite number, then surely this program does too. Eventually, the program will run itself. When it does, it has already printed n-1 digits. It will run itself until it prints the nth digit. But it never will!

Turing equated deciding whether a program gets caught in an infinite loop with deciding whether a statement is provable. Since we can’t evaluate a program to see if it will enter an infinite loop, neither can we decide whether an arbitrary statement is provable.

Modern day provability

These days, we have theorem provers, statically analysis, and code contracts that verify assertions about programs. And yet when Turing invented programming, he proved that these things could not work! Was Turing wrong?

No, Turing was absolutely right. These tools have limitations. There are some programs that you could write that these tools would not be able to prove correct. They won’t work in the general case.

But even though there is a vast infinity of programs that we cannot prove, there are plenty that we can. If we constrain ourselves to the provable set, then these tools can be of incredible value. Knowing where the boundary is helps us to get the most of these tools. And that is the true value of the Entsheidungsproblem.

Please read “The Annotated Turing” by Charles Petzold for a fantastic journey through this foundational paper. This is where our industry was born, only 75 years ago.



Keep functional and non-functional requirements separate

Functional requirements change frequently. System architecture is difficult to change. If system architecture depends upon functional requirements, then the system is going to be brittle and expensive. Instead, system architecture should only be based on non-functional requirements. It should be isolated from changes in functional requirements.

The difference between functional and non-functional requirements

Functional requirements describe how the system behaves within the problem domain. A banking application will express its functional requirements in terms of customers, accounts, and transactions – financial transactions, not database transactions. If a branch manager decides to offer a free checking account with every home mortgage, then he is changing functional requirements.

Non-functional requirements describe how the system behaves from a technical perspective. They are independent of the problem domain. Whether it’s a banking application, a healthcare application, or a property management system, non-functional requirements are expressed in the same terms. If data needs to be replicated to a different location, that’s a non-functional requirement. If records should be denormalized for reporting, that’s a non-functional requirement. If the branch manager wants to launch a new web site, he is changing non-functional requirements.

Web services

Too many of our foundational systems conflate functional and non-functional requirements. The result is software that is difficult and costly to change. Take, for example, web services. Every web service defines two things:

  • A message contract
  • A delivery mechanism

The message contract describes what is to be sent from the client to the server, and what is to be returned from the server to the client. The contents of a message are elements of the problem domain. A banking application will have web services to list a customer’s accounts, and to get transactions by account and date range. These are functional terms. When functional requirements change, the message contract has to change.

The delivery mechanism describes how messages are sent between the client and the server. Practically speaking, web services use HTTP. They are synchronous in nature, and initiated by the client. The message contract is defined by the server. Message delivery is not guaranteed. These are the non-functional consequences of choosing web services.

A web service conflates the functional and non-functional requirements through its use of the Web Service Definition Language (WSDL). WSDL contains a definition of the message contract, a functional construct. When functional requirements change, the WSDL is updated. The server must publish the new WSDL for all of its clients to consume. The non-functional relationship between clients and servers is now dependent upon functional requirements expressed in the message contract.

Relational databases

Another example of conflated requirements can be found in a relational database. Relational databases require developers to define a schema, which describes tables and columns. Tables are entities in the domain, and columns are attributes of those entities. A schema is dependent upon functional requirements. When the functional requirements change, the schema changes.

A relational database has tools to help us satisfy non-functional requirements. Non-functional requirements will describe the Service Level Agreement (SLA) governing how quickly transactions must be processed. They will also describe how frequently reports are run. Based on these requirements, a DBA may choose to index the data differently. They may choose to replicate from the transactional database to the reporting database. They may even decide to denormalize the reporting database for better performance. All of these decisions are non-functional in nature.

Indexing, replication, and denormalization all depend upon the database schema. When the schema changes, these decisions must be revisited. The schema changes whenever functional requirements change. As a result, expensive architectural decisions are affected by quickly changing functional requirements.

This can only lead to one of two outcomes:

  1. Changes to functional requirements are expensive, or
  2. Changes to functional requirements are discouraged.

Either outcome is death to business.

Application-agnostic architectures

To allow functional requirements to change frequently, architectural decisions should be completely isolated from the problem domain. If an architectural decision depends upon a functional requirement, then it runs the risk of becoming invalidated by future requirements changes. Architectural decisions should therefore be made with knowledge of non-functional requirements alone.

To some, this may sound like Big Architecture Up Front. Well, it is Up Front, for all the reasons described above. But it doesn’t have to be big. We generally know before getting into the details of the problem domain whether we will need a web application, a reporting database, or a service bus. The up front architectural decisions will simply select these components based on non-functional requirements. You may even decide to use technologies like web services or relational database that conflate requirements. Just be careful that you don’t centralize these technologies, allowing them to cause friction as functional requirements change.

One consequence of isolating architectural decisions from the problem domain is that architecture can be amortized. Since architecture does not depend upon the problem domain, it can be done up front with no knowledge of the application. Transaction processing is a common non-functional requirement. Reporting is a common non-functional requirement. A good architecture can be crafted for performing transaction processing and reporting with no knowledge of the applications for which it might eventually be applied. That architecture can then be used in many different applications, with no additional cost.

It is common for a consultant to answer a question with “it depends”. If the question is an architectural one, then “it depends” should be followed with a non-functional requirement. If architectural decisions depend upon functional requirements, then you will be paying that consultant over and over again as you make changes to your business model. But if we as an industry create a ready-made set of application-agnostic architectures, we can pull the right one from the shelf and apply it to a large number of problem domains. If these architectures are carefully constructed not to conflate requirements, then they can drive down costs over time.



Correspondence on Android

Correspondence is all about synchronizing data between users, between devices, and now between platforms. Correspondence is running in Mono for Android.

I created a Silverlight application called ThoughtCloud for a demo. Users share thoughts with this collaborative mind mapper. This Android app presents a read-only list of the clouds shared in the Silverlight application. It can be extended to view those clouds and make changes.

User interface updates

Correspondence is built on top of Update Controls, which is responsible for updating the user interface. The Silverlight and Windows Phone versions of Update Controls work with data binding to implement MVVM. But in Android, there is no data binding. Hence, there is no MVVM. Instead, Android uses adapters.

A ListView is the Android equivalent of the Silverlight ListBox. It is attached to a ListAdapter. It works like an ObservableCollection: any item added to the ListAdapter will appear on the ListView. The following code adds the name of each cloud to the adapter.

private void UpdateCloudArray()
{
    _cloudArrayAdapter.Clear();
    foreach (Cloud cloud in _identity.SharedClouds)
    {
        Thought centralThought = cloud.CentralThought;
        string text = centralThought == null
            ? "<null>"
            : centralThought.Text.Value ?? "<empty>";
        _cloudArrayAdapter.Add(text);
    }
}

It’s not very smart to clear the whole adapter each time, but this is proof-of-concept code. In a more sophisticated system, it will update items in the adapter in place.

Tracking dependencies

Now we have to make sure that UpdateCloudArray is run whenever it needs to be. That’s where Update Controls comes in. Update Controls observes what your code touches. What your code reads is what it depends upon. This code reads the SharedClouds, which is a query in the factual model.

fact Identity {
key:
    string anonymousId;

query:
    Cloud* sharedClouds {
        Share s : s.recipient = this
        Cloud c : s.cloud = c
    }
}

It also reads the CentralThought of each cloud, and the Text of each central thought. These are both mutable properties in the factual model. Whenever any of these things changes, Update Controls can tell. The following code hooks it up.

_depCloudArray = new Dependent(UpdateCloudArray);
_depCloudArray.Invalidated += delegate
{
    RunOnUiThread(delegate
    {
        _depCloudArray.OnGet();
    });
};
_depCloudArray.OnGet();

The Dependent object keeps track of dependencies. It takes an update method as a constructor parameter. Whenever it needs to update, it calls this method. It depends upon the things that this method reads.

When one of the dependencies changes, the Dependent goes out-of-date. At that moment, the Invalidated event is raised. The delegate above brings the Dependent back up-to-date on the UI thread.

Finally, the Dependent starts its life out-of-date. So we call OnGet() to bring it up-to-date immediately.

Next steps

The above code is more than I’d like to write for each adapter. Instead, I intend to create adapter classes that build dependency tracking right in. This is the process I went through for the Update Controls library for Windows Forms. WPF and Silverlight have data binding hooks, so I didn’t have to individually augment every control for those versions.

The Silverlight version of Thought Cloud uses HTTP long polling to push changes to other users, so your thoughts immediately appear on their machine. On a phone, this is done through push notifications. I haven’t yet written the push notification service for Android. So this code will not automatically reflect changes made online. That is my next milestone. For now, you have to exit and reenter the app.

This Mono version is slow. I’ve written Android apps in Java, and they were not nearly as slow as this. It could just be that I have to do some optimization in Correspondence. Or it could be the extra layers of the Mono .NET runtime. My Faceted Worlds partner Russell and I are in the middle of the Java port. We’ll know soon whether Correspondence can be performant as a native Android library.



XAML has Come Home to Windows

Yesterday Steven Sinofsky presented Windows 8 from a developer’s perspective. He emphasized that Windows 8 has a brand new API, that it enables native application development in multiple languages, and that markup is a significant component of the rendering engine. In so doing, he mentioned that all of our Silverlight skills transfer to the new operating system. He explicitly did not say that Windows 8 is running Silverlight. This speaks to the branding that Microsoft has developed around Windows and .NET.

In 2007, Microsoft released .NET 3.0. Despite it’s major version number, it was not a new version of the .NET runtime. Rather, it was four new libraries running on top of .NET 2.0: Windows Communication Foundation (WCF), Windows Workflow Foundation (WF), Windows CardSpace, and Windows Presentation Foundation (WPF). Not .NET Presentation Foundation – Windows.

In 2008, Silverlight 2.0 took the .NET Framework and WPF outside of the Windows platform. With this release, applications written with XAML and C# or VB ran in the browser on Windows and the Mac. Later releases would take those applications out of the browser. Silverlight was originally intended as a bridge to take Microsoft technology across all platforms. But then the strategy around Silverlight shifted.

In the first Windows 8 video, Sinofsky emphasized that Windows apps could be written in HTML 5 and JavaScript. He made no mention of Silverlight. This worried the Silverlight community, reigniting fears that Silverlight was dead. Yesterday’s keynote cleared things up a bit. Windows 8 apps will be written primarily in four languages: C#, VB, C++, and JavaScript. The markup will be based on XAML when C#, VB, or C++ is employed, or on HTML 5 when JavaScript is employed. This is good news for Silverlight developers, because their skills are still of value.

This is also a good strategy for Microsoft. Before, Microsoft was attempting to build the cross-platform bridge. They faced severe push-back from other platform vendors, particularly Apple. Sure, Silverlight worked on the Mac, but Microsoft was forbidden from porting it to the iPhone and iPad. So instead of Microsoft bridging out, they are letting the industry build bridges into Windows. Let HTML 5 and JavaScript be the cross-platform markup and programming language. XAML and C#/VB will be first-class Microsoft tools.

What is telling is the words that Sinofsky used on his architecture slide.

  • WinRT – Not the .NET Framework
  • C# and VB – Not .NET
  • XAML – Not Silverlight

WPF is not dead. Silverlight is not dead. The components that made them successful are now core components of Windows. XAML has always been a Windows technology. Sinofsky has just pulled it back into the fold.



What’s so special about Windows Phone?

I’ll be presenting at DalMob this Tuesday evening. DalMob is a mobile development group that crosses all of the platforms. Most of the attendees are iPhone and Android developers. I’m going to demonstrate the key features that make Windows Phone different from a developer’s perspective. These are:

  • Panorama and pivot
  • Live tiles
  • Storyboard animation
  • Data binding

Demo code is borrowed from Chris Koenig. Slides are borrowed from Megan Donahue on Advanced Application Design for Windows Phone at Mix’11.



Correspondence presentation from Dallas .NET User Group

The recording of my Correspondence presentation is up on UserGroup.tv. This one covers Silverlight out-of-browser applications. I used Thought Cloud as a demo, and opened it up for the audience to participate. It makes for some interesting on-screen shenanigans.



DevLink app results

DevLink 2011 was a fantastic conference. I met some well-known members of the development community. And I met some brilliant people. Occasionally, these were one-and-the-same!

I presented two sessions at DevLink. The first was on the CAP Theorem. I’ve posted the slides and demo for your perusal. The second was on Correspondence, the same session I gave at Dallas TechFest the week prior. Both sessions were well attended, though I did have problems with my live coding demo in the Correspondence session. Fortunately, the first demo had already gotten the point across.

Like I did for Dallas TechFest, I created a conference application using Correspondence. I pushed this app to the Marketplace for attendees to use. This time, I got 30 downloads out of about 650 attendees, and only one review. So the overall percentage was lower this time around, but still useful.

The one reviewer gave the app 4 out of 5 stars, with the following comment:

So far I really like it. It seems a little slow but I like the format a lot better than the eventboard program.

The EventBoard application that he references was the official app available on the Windows Phone and iPhone platforms. It’s really cool to be favorably compared with a professional app. I think the primary differentiator between the two is that, where EventBoard is all about the event, my app was all about the attendee. The first page of my app shows your schedule. The first page of EventBoard shows a list of events.

Performance enhacements

The reviewer noted that the application seems a little slow. This is because of performance problems in Correspondence. I made some improvements, but was unable to get them into the Marketplace in time. More performance enhancements are on the way.

The first enhancement was to reuse the IsolatedStorageFile object rather than recreating it. Every example of using isolated storage that I’ve seen follows the same pattern:

public class IsolatedStorageStorageStrategy : IStorageStrategy
{
    public static IsolatedStorageStorageStrategy Load()
    {
        var result = new IsolatedStorageStorageStrategy();

        using (IsolatedStorageFile store = IsolatedStorageFile.GetUserStoreForApplication())
        {
            using (BinaryReader input = new BinaryReader(store.OpenFile(ClientGuidFileName, FileMode.Open)))
            {
                result._clientGuid = new Guid(input.ReadBytes(16));
            }
        }

        return result;
    }
}

Isolated storage is notoriously slow. It was easy to isolate the slowness to the creation of the IsolatedStorageFile object. So I just changed the pattern:

public class IsolatedStorageStorageStrategy : IStorageStrategy
{
    private IsolatedStorageFile _store;

    private IsolatedStorageStorageStrategy(IsolatedStorageFile store)
    {
        _store = store;
    }

    public static IsolatedStorageStorageStrategy Load()
    {
        IsolatedStorageFile store = IsolatedStorageFile.GetUserStoreForApplication();
        var result = new IsolatedStorageStorageStrategy(store);

        using (BinaryReader input = new BinaryReader(store.OpenFile(ClientGuidFileName, FileMode.Open)))
        {
            result._clientGuid = new Guid(input.ReadBytes(16));
        }

        return result;
    }
}

This was low-hanging fruit. Other performance enhancements that I have in the queue are:

  • Pooling the Stream objects instead of opening and closing files.
  • Switching from XML to binary serialization of facts for HTTP communications.
  • Pulling more facts at a time from the server.
  • Evaluating SQL CE as an alternate storage strategy.


The CAP Theorem and its Consequences

Here you will find the slides and code from my talk “The CAP Theorem and its Consequences”. The slides are in Silverlight, so use these controls to navigate:

  • Page down/up to go forward and back by slide.
  • Arrow down/up to go forward and back within a slide (Definitions, Proof, Choices, and Event Sourcing).
  • Scroll wheel to zoom in.
  • Drag to pan.

You can also download the source code and run the examples yourself. Run databases.sql to create the three databases. Populate the databases with test data in accounts “12345”, “23456”, and “34567”. The application transfers money among these accounts.

Get Microsoft Silverlight

For more information, please read the summary article.



Dallas TechFest 2011

I presented two sessions at Dallas TechFest this year. I also attended some excellent sessions, hung out with old friends, and made some new contacts. Thanks to Tim Rayburn, Teresa Burger, Shane Holder, and other organizers, this was a fantastic event. They’ve really set the bar high for next year.

My sessions

My first session, Provable APIs, was very well attended. This talk presents techniques that you can use to ensure that people consuming your API do so correctly. These techniques take advantage of the compiler, and the type system in particular. I present samples in C#, but the techniques work well in any statically typed language.

The session was very well attended. We had people using not only .NET, but also Java, and C++. We even had some dynamically-typed languages represented including JavaScript and Ruby. Although they don’t have a compiler and static type system to leverage, they can still use some of these techniques.

The audience was extremely interactive. They posed some excellent questions. They were particularly interested to see a refactoring of the .NET Socket class, with its notoriously error-prone API.

My second session, Correspondence: Occasionally-Connected Windows Phone Apps, was not as well attended. Those that did participate were rather quiet during the theory portion, but started to interact during the demo. Based on their feedback, I will move the demo to the beginning of the talk, and save the theory only for after I get questions about how it works.

The app

The demo for the Correspondence session was actually the Dallas TechFest 2011 Windows Phone 7 app. I got some great feedback from people who were using the app during the conference. I also saw a few bugs that others reported and that I experienced myself. I’ll be rolling out bug fixes for upcoming conferences.

All told, I had 22 downloads of the app. Out of those, I received 2 ratings with feedback, for an average of 4.5 stars. 22 downloads out of a possible set of about 200 attendees, only about 30% of which carry Windows Phones, is pretty decent reach. And to get 10% of the downloads giving feedback is excellent. Sure, the absolute numbers are too low to make any generalizations, but I’ll take what I can get.

The DevLink 2011 app is in the Marketplace now. If you are attending DevLink and you carry a Windows Phone 7, please give it a try.

Old friends and new contacts

Tim, Teresa, and Shane are all old friends, so to see them pull off such an outstanding conference was a real treat. As an added bonus, I got to see a good friend from work present in his first conference. Girish Gangadharan presented on JQuery, and packed the room. Actually, I didn’t see him present because my Correspondence session was at the same time. But I saw pictures, so I know it was a full house.

I also got to meet some new folks. Devlin Liles from Improving Enterprises and Jay Smith from Tyson Foods are active members of the open source community. Their tools help out user groups and event organizers across the country and the technology spectrum. I look forward to working with them on community-centered open source projects.

I had a great time at Dallas TechFest this year. I’m looking forward to following up on all the contacts I’ve made, to applying all the techniques that I’ve learned, and most importantly to participating next year.

Now on to DevLink.



Correspondence

Correspondence is a collaboration framework for occasionally-connected clients. Express your model once, and it gives you local storage, synchronization, and push notification across devices. We currently support Silverlight, Windows Phone, WPF, and MVC 3. Android is coming soon.

Collaborative framework

LogoPeople don’t just own one computer anymore. Now they have a desktop, a laptop, and a phone. They want their data to seamlessly flow across all of their devices.

People take their devices with them. These devices aren’t always connected. And even when they are, people don’t want to wait for them to connect to a central server. Everyone should have their own data on their own device for immediate access.

People use software to collaborate with each other. Some domains are overtly collaborative, such as social networking and gaming. Others are more discreet, like customer relationship management and project planning. Actions performed by collaborators affect the user experience.

Systems built with Correspondence

We have built a number of systems using Correspondence.

  • HoneyDo List: Todo lists that you can share with your family.
  • Commuter: A continuous podcast playlist for iTunes on Windows.
  • Faceted Reversi: Head-to-head reversi game for Windows Phone.
  • Dallas TechFest 2011: Personal conference schedule for Windows Phone.
  • Thought Cloud: Collaborative mind mapper (demo).

Slides

The slides for the presentation are rendered in Silverlight. Once the page loads, click on it to give it focus. Then use Page Down to progress through the presentation. Hit F11 to enter full-screen mode. The sample code used in this presentation is on GitHub.

Videos

Get started

These are the resources you will need to build an occasionally connected Windows Phone 7 or Silverlight application using Correspondence.

  1. Install the NuGet Package Manager through the Visual Studio 2010 Extension Manager. Detailed instructions on the NuGet project site.
  2. Add the Correspondence.WindowsPhone.AllInOne or Correspondence.Silverlight.AllInOne package to a Windows Phone 7 or Silverlight 4 application.
  3. Follow the walkthrough on the Correspondence project site to learn how to build a Correspondence model.
  4. Sign up for a synchronization server API Key. Put the API key in your POXConfigurationProvider.