This post will describe how tests are setup within Kino. If you haven’t already you can download the source from here, play around and tell me what you think.

What is the motivation?

What motivates you to do unit testing? Is it to see gray dots turn green, or the ability to do major refactoring without breaking functionality. Maybe it is the overall quality improvement of your code that motivates you. For me it is how unit tests make it visible how extremely hard it is to write high quality code.

One of the best motivators for any kind of work is watching your progress, a thing that can be hard when it comes to software development, and even harder when it comes to testing. To be able to measure progress you’ll have to know when you’re done, and when are you really done with testing? What does it take for you to say, I’ve tested enough. Let’s move on.

Different kinds of tests

Normally you have two kinds of tests in your development environment. Unit tests are the ones I described in my previous post where you test the behavior of individual units of the program. Integration tests are testing that units work well together. An example of integration test in Kino would be the RssDocumentTest.WriteToTest.SerializedXmlConformsToSchema. This test creates an RssDocument, serializes it to Xml, and then validates it to the RSS 2.0 specification schema. This test involves pretty much the whole RSS namespace and is a good representation of an integration test.

Though system tests and acceptance tests are very important I would most often leave these to the client. They can make decisions about how much acceptance they need before a release, or if they want to do performance testing on a regular basis. These are the kind of tests that will verify the system and not inflict changes on the code base with such impact as unit/integration tests.

Unit testing and integration testing are the things that you as a developer should focus on in your daily work because it is as close to the source as you get.

When you are done with unit testing

Writing unit tests is a paradox. You write them to spend less time on maintenance, but if you write too many tests or if you do it wrong you could end up spending more time on maintaining unit tests than you would maintaining a legacy system. Testing every behavior in your system is really not an option.

Any system is like a tree. There is a trunk in your code that a majority of the system calls will traverse. Some system calls will deviate from the trunk and choose to go through one of the smaller branches, but this is quite unusual and that is why the branches are much thinner than the trunk.

Most of the bugs you’ll find in a program will be in the trunk, because this is where the majority of the system calls take place. There will be just as many bugs in the branches, but some of these bugs may stay hidden for years before they turn up. The technique you should focus on in your testing is to make sure the trunk is as bug free as possible and with as high quality as possible. Then you can take care of the special cases as they turn up. Don’t live with the illusion that you will write a completely bug free program.

Finding the trunk of your program

You should know where the trunk of your program is because you wrote the code, right? It is not always that easy, and you will need a tool that can show you this information. One such tool is NCover.

What is NCover?

NCover is a tool that will hook up any process and track what code that process traverses in your program. By monitoring unit tests, integration tests and also acceptance tests you can make sure that you know where the trunk of your program is. This will give you a rough idea of what to tests.

Test to know what to test, is he mad? No, testing is an incremental process where you learn more and more about your system for every test you write. As you start writing your tests you have limited knowledge of what to test, but this will become much clearer along the path.

Above is the result of NCover monitoring my source code while doing functional testing (running Kino through IIS and start clicking around). The number next to the code is tells me how many times the program has been there. A higher number will indicate that this is definitely part of the trunk of the program and here I should focus my testing.

Measure unit test coverage with NCover

When you run NCover on your unit test runner you will get a clear view on what you’ve really tested. You’ll also get a report on how much of your assemblies you’ve covered with your unit tests. This is an important aspect in unit testing, indeed but not really as important as knowing what to test. 100% coverage is not always a good thing.

For every test you write you close your test suite closer to the system under test. This will make maintenance harder, because if you change any code in your SUT, you will definitely break some tests. This anti-pattern is called “the fragile test”.

The conclusion here is that you should use a coverage tool to make sure that you test the right things, and you should focus your testing where it is needed the most. Even with a very small amount of tests and a thought through test strategy you will reach 91% coverage without any tears (except those of joy).

I will get back to NCover to review its features in a future article.

This post will describe how tests are setup within Kino. If you haven’t already you can download the source from here, play around and tell me what you think.

Choice of framework

As we all know, there are a variety of unit testing frameworks out there where the main actors are MsTest, NUnit, MbUnit and xUnit. This time I’ve used MsTest as my unit testing framework. It is easy that when you compare frameworks to one another, you pick a good feature from one framework and make it a missing feature in another framework. This is mostly the case with MsTest

The good things

• Naively integrated with Visual Studio 2008 Pro. This makes it an obvious choice for the beginner. I think it is great that we finally have native unit testing support in Visual Studio Pro (not only TFS) because it opens up the possibility for a much broader range of programmers.
• Built in support for MsTest unit tests in the TFS. If you’re using TFS for build system and project management, you would want to create your unit tests with MsTest, because it makes your life much easier when it comes to Team Build. Like any Microsoft product, “it just works”.

The bad things

• You need to have Visual Studio installed to your system, if you want to be able to run your unit tests. So if you’re using CruiseControl.NET as your build server, you will have to install Visual Studio on that machine, which most probably is not what you want. You will not be able to run the unit tests on your client machines either, which could be good if you have a suite that tests if the prerequisites are in place for the system to work.
• CategoryAttribute is missing in the MsTest framework so you can’t categorize your unit tests. I use this with NUnit mostly to markup those tests that differ from the main set of tests. Microsoft wants you to create separate assemblies to categorize your tests, but this is not always what you want. In Kino we have one test assembly, because this project is just too small for two. In that assembly unit tests and integration tests lives together, and I have no other mean than by comment to distinguish what is what. With CategoryAttribute I would be able to run just that category that I specify, regardless of what assembly/namespace they are in.
• RowTestAttribute is also a missing feature that originally comes from MbUnit. It allows you to execute the same test several times but with different in-data. For example, when you want to test a regular expression, you send in different string that should match or not. This allows you to narrow down what the specified expression can handle, and what it can not. Yes, there is something called data driven test in MsTest, but I’ve found it too clumsy to be a substitution for the Rowtest.
• As one of my kind readers pointed out, MsTest is not available on Microsoft Visual Studio Express, which makes any test project using this unit test framework unusable for any Exress developers. I guess that Mono users have the same problem here. (?)

I’ve chosen MsTest for Kino, simply because I have to learn its weaknesses before a project I will have later this year demanding MsTest as testing framework, simply because we’re going TFS all the way.

Structure your unit tests

One very important aspect as in all programming is to find a great structure and stick to it. I’ve worked out a technique for writing unit tests that I’m especially comfortable with, and I would like to show it to you.

[actor] should [condition]

Writing unit tests, you’re testing the inner most parts of the program. It is very seldom that you test a whole class, but you go down into methods and test specific cases. I’ve found that using the above statement helps me to find out what to test.

The actor is most often a method or a property. It could also be a class, but that is quite unusual. Please examine the following statements.

• Constructor should throw ArgumentNullException on name
• Find should return empty list when nothing was found
• RssDirectoryGenerator should derive from RssChannelGenerator
• Instance should return the same reference every time

The only way I’m able to do this is to recreate the namespace structure in my testing assembly, and actually set SUT classes as namespaces, and their methods/properties as classes. Please study the following screenshot from Visual Studio object browser.

The above picture could be read as

• ContainerFactory.GetContainer should throw ConfigurationErrorsException when container was not found
• ContainerFactory.GetContainer should get RssDocuments container

This kind of thinking is called behavior testing, because we’re testing the behavior of these methods and classes. It gives our tests a much better focus and they will become easier to read and to maintain. It even gives us better coverage, something I will talk about in my next article.

This is the second part of the article series about Kino. Please note that you can download the full source here.

Dependency injection

As I shown in the previous article, dependency injection plays an important part in Kino. It glues everything together and the way it is configured is through XML.

What you will notice if you browse around for Unity configuration examples, is that most configuration examples are through code, and not through XML. Why is that? When is it reasonable to configure your dependency injection framework through code, and when is it reasonable to do it through XML?

DI configuration through XML or code?

The purpose of the DI framework is to help isolate parts of the program and to improve extensibility. This means that the use of a DI framework will change what our code looks like, in a good way.

What does it mean to put the DI configuration as code? It will become much more readable and in that way, easier to maintain. But, it will also require deployment of new binaries when you want to change your configuration or extend it with a new type. This is the main purpose why you don’t want to compile your configuration into the binaries. But does it really matter if you’re the only one changing the code? It makes perfect sense to put the configuration as XML if you want some third party to be able to extend your program without the possibility to update the code. But if you’re the sole owner of that code base?

XML is a markup language and the DI configuration is nothing like markup. To put the configuration in a markup language and not in a CLR language is crazy. It makes the configuration very hard to read, but you will have the power to update the configuration without updating the binaries, and that is the big deal, right?

The XML configuration will in its own nature make your program brittle, because if you break the XML you will also break the program. This is however true for all configuration files of your system.

Note on the future: In .NET 4.0 we will get the DLR with the ability to mix dynamic languages with static langauges. This means that we could use a more proper language like Python or Ruby for DI configuration, and still have it dynamic in the sense that you can change it without recompiling the source code.

DI configuration in Kino

I’ve chosen to put my DI configuration as XML in Kino, because I want to show you how it is done. I also have a whole different purpose with this.

If a purist would open up Unity.config he would probably scream! I have mixed the DI configuration with the application configuration. This means that you use the DI configuration to setup the program. The purist would claim that you can only use the DI configuration to specify what parts of the program connects with others, but I actually put configuration keys and values as constructor injections to types, making this not only a DI configuration but also app configuration.

A better way to do this would be to create a different configuration file for the application configuration. I made the decision to merge them to keep the system as simple as possible. This would not be possible if we had a more complex or larger problem on our hands.

Every decision is a good decision if it has been well throught through.

This is how it works!

The following code is placed in your App.config (or web.config if it is a web project).

<configuration>
  <configSections>
    <section name="unity" type="Microsoft.Practices.Unity.Configuration.UnityConfigurationSection, Microsoft.Practices.Unity.Configuration, Version=1.2.0.0" />
  </configSections>
  <unity configSource="Unity.config" />
</configuration>

This tells your program that there is a configuration section handler inside the Microsoft.Practices.Unity.Configuration assembly that knows how to handle the configuration inside the file Unity.config. I like to keep my Unity configuration in a separate file because this configuration tend to get quite massive, and web.config is bloated as it is.

You will find the following code to access the unity container in my little singleton ContainerFactory.

public UnityContainer GetContainer(string name)
{
    if (configurationSection == null)
        throw new InvalidOperationException("No unity configuration was found, could not instansiate container");

    var container = new UnityContainer();

    UnityContainerElement containerConfiguration = configurationSection.Containers[name];
    if (containerConfiguration == null)
        throw new ConfigurationErrorsException("No unity configuration for " + name + " was found");

    containerConfiguration.Configure(container);
    return container;
}

If you would setup the container through code, this is probably where you would do it. We will just use it for creating a new container and configure it through the configuration. Or you could use it to invoke some dynamic method in IronPython or similar that would have the same effect.

Now it comes to the configuration file Unity.config. If you check it out I think you will get how it works. I would like to share a small schema that has helped me a lot in figure it all out.

As you know from the previous article the first thing resolved will be the RssDocument, and it will be resolved by name.

<type type="RssDocument" mapTo="RssDocument" name="Movies">
  <typeConfig extensionType="Microsoft.Practices.Unity.Configuration.TypeInjectionElement,
                             Microsoft.Practices.Unity.Configuration">
    <constructor>
      <param name="version" parameterType="System.String">
        <value value="2.0" type="System.String" />
      </param>
      <param name="channelGenerators" parameterType="RssChannelGeneratorArray">
        <array>
          <dependency name="MovieList" />
        </array>
      </param>
    </constructor>
  </typeConfig>
</type>

Here name equals “Movies” is the argument given to the RssController. This means that when a client request the http://localhost/Rss/Movies/ url, the RssController will tell the UniyContainer to resolve this RssDocument. This document contains an array of channelGenerators. In this case only one generator defined “MovieList”, but with the ability to have several. The <dependency name=”MovieList” /> refers to this piece of declaration.

<type type="RssChannelGenerator" mapTo="Kino.Lib.Rss.RssDirectoryGenerator, Kino.Lib" name="MovieList">
  <typeConfig extensionType="Microsoft.Practices.Unity.Configuration.TypeInjectionElement,
                             Microsoft.Practices.Unity.Configuration">
    <constructor>
      <param name="title" parameterType="System.String">
        <value value="Movies" type="System.String" />
      </param>
      <param name="link" parameterType="System.String">
        <value value="http://mint.litemedia.se" type="System.String" />
      </param>
      <param name="description" parameterType="System.String">
        <value value="Sharing of movies" type="System.String" />
      </param>
      <param name="documentation" parameterType="System.String">
        <value value="http://blogs.law.harvard.edu/tech/rss" type="System.String" />
      </param>
      <param name="managingEditor" parameterType="System.String">
        <value value="john.doe@litemedia.se" type="System.String" />
      </param>
      <param name="webMaster" parameterType="System.String">
        <value value="john.doe@litemedia.se" type="System.String" />
      </param>
      <param name="basePath" parameterType="System.String">
        <value value="C:\Movies" type="System.String" />
      </param>
      <param name="searchPattern" parameterType="SearchPatterns">
        <array>
          <value value="*.wmv" type="System.String" />
          <value value="*.avi" type="System.String" />
          <value value="*.mpg" type="System.String" />
        </array>
      </param>
    </constructor>
  </typeConfig>
</type>

This is really the same thing as the RssDocument, and you can match the constructor declaration with that of the RssDirectoryGenerator and see that it matches. More interesting here is to look at the type=”RssChannelGenerator” mapTo=”Kino.Lib.Rss.RssDirectoryGenerator, Kino.Lib”. This tells us that we want to resolve an RssChannelGenerator, but it could be any kind of type that derives from this abstract class. Here is the extensibility where you can implement your own RssChannelGenerator and really get Anything to Rss.

This post will describe the architecture and overall design of Kino. If you haven’t already you can download the source from here, play around and tell me what you think.

Architecture

The main purpose of Kino is to serve RSS, and the easiest way of doing this is through a web server. The most modern way of writing web applications in .NET today is through ASP.NET MVC. This is lightweight enough for our purposes, and we will actually not even use but a small part of this frameworks functionality.

The project structure will be quite straight forward. We need the web project that is going to serve the RSS flow to web clients, and we need a “library” project to separate presentation from logic (or model, in MVC terms). This is very important and something I do in every project of mine. Logic should be isolated where it can be safely tested, updated and maintained.

Last of all we have our test project that where we will put tests for our application. Since this project is such a small project, I’m not going to separate unit tests from integration tests, but keep them in the same assembly. I will not either have a separate test assembly for the web project or the logic library. When we start talking about testing you’ll understand why.

A common error is that you create one assembly for every single module that is possible to isolate. This is ok to do as a starting point, but you’ll have to be prepared to join assemblies later. Lots of assemblies makes the project hard to test, and hard to maintain because of long compile times. Yes, more assemblies actually takes longer to compile than a few. So, isolation of logic is not always good. Most often you can substitute a couple of assemblies with namespaces instead.

Design

One way to design for extensibility is to place a dependency injection framework in the center of your program and let every class that has a dependency go ask the framework for an instance to that dependency. This gives you a highly extensible platform to build on, but it can also make your program very complicated to understand when overused.

In our case, we have a very small and simple system.  A little bit of added complexity will not do much harm, but you would have to consider the use of your dependency injection tool in a larger system, so you don’t reduce complexity through isolation, but increases it through configuration of your DI framework.

With that said, the web is connected to the logic through our dependency injection tool Unity.

The chart above describes the full complexity of Kino. On a web request, the method Index is called on the RssController with a name, representing the RSS feed the client wants to retrieve. The controller does not know how to create the specified feed, but it knows that Unity knows, and therefor it uses the ContainerFactory to retrieve the UnityContainer that could create this RssDocument. When retrieved, it will ask the container to Resolve this unknown dependency and the container will return the RssDocument created. Once done, the RssController will write the contents of that document to the HttpResponse and return it to the client.

That is in fully, exactly how Kino works.

Model

We’re not quite finished yet. This program is not only extensible because I say it is, but because we really haven’t said anything about the RssDocument yet. What is it? As a client we don’t really care, because we know that the RssController can retrieve it for us. As an RssController we know what the RssDocument is, but we don’t care about how it works as long as we can write the contents to an output stream.

Unity knows, and how you configure Unity I will talk about in the next part of this series. So, let us move on to the details of this RssDocument.

An RssDocument is really a collection of RssChannels, that are a collection or RssItems. An RssItem could be compared with a blog post and the RssChannel could be the blog. The model is simplicity itself, but to allow dynamic generation of channels, I’ve included RssChannelGenerators as a parameter to the constructor of RssDocument.

Plainly spoken, the RssDocument has a way to generate channels. How channels are generated, the RssDocument does not care about, as long as it gets populated by RssChannels and RssItems. We could pass any implementation of RssChannelGenerator into RssDocument as we would like, but here I’ve given an example of two generator, RssFileGenerator and RssDirectoryGenerator, that will create RssItems by watching the file system. They inherit from the class RssFileSystemInfoGenerator simply because they have some logic in common.

Extensibility

So, if you wanted to create an RssDocument that took several rss feeds and merged them into one, you would need to create a new RssChannelGenerator and inject it into the RssDocument. If you really wanted to break the rules you could derive from the RssDocument and create your own JSONDocument, and return the result as JSON instead of XML. There is a lot of freedom here and it comes with the price of readability and maintainability. It is hard to get a clear picture of what is going on in a fully independent system, but it is also very powerful.

In the next chapter we will go deep into Unity and study the configuration.

I have a file sharing service up and running for my friends. It is nothing fancier than a windows share with a web interface tied to it. When I would like to share a new file, I just drop it in the shared directory and it will be available to everyone that has login credentials to that share.

Cool and easy, works like a charm. – Almost.

How should my friends know that I’ve dropped a new file or directory into the share? Of course I could send them an e-mail, but that does not seem very convenient. They would get spammed and I would have to do manual labour.

From the title you’ve already guessed that I would use RSS. Let’s write an application that would monitor the share and publish any additional files as items in an RSS feed. This would only require my friends to subscribe to an RSS channel, if they’re interested in what’s new.

The solution

In the following articles I will discuss how this problem was solved. We will take a detailed look on the source code and how it was implemented. You have the following articles looking forward to.

• Part 1: Design and architecture
• Part 2: Dependency injection with Unity and XML configuration
• Part 3: Structured unit testing
• Part 4: Keep yourself covered with NCover
• Part 5: Easy error handling with ELMAH

Get started with the source

You may of course download the source right now, and start playing with it. Go to the download page and follow the instructions there to get you started.