As I mentioned before, I’m working on a replacement for this blog. I’m going to migrate to Orchard CMS, mostly to learn that content management system. This has been a pleasant experience so far.

Before I can release my new blog I need to move all my content from Wordpress to Orchard CMS. Someone (not me) should really think about writing a module to make this a pleasant journey. I did this manually, because I only expect to do it once.

Export from Wordpress

Wordpress has an export function. You find it in Tools menu. This is cool, except that the format is some weird kind of RSS, that is extended with Wordpress’ own xml elements. Fine. Let’s see what we can do about this.

<?xml version="1.0" encoding="UTF-8"?>
<!-- generator="WordPress/2.9.2" created="2011-05-24 06:45"-->
<rss version="2.0"
	xmlns:excerpt="http://wordpress.org/export/1.0/excerpt/"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:wp="http://wordpress.org/export/1.0/">

<channel>
	<title>Mint</title>
	<link>http://mint.litemedia.se</link>
	<description>building a .NET application</description>
	<pubDate>Tue, 24 May 2011 06:28:55 +0000</pubDate>
	<generator>http://wordpress.org/?v=2.9.2</generator>
	<language>en</language>
	<wp:wxr_version>1.0</wp:wxr_version>
	<wp:base_site_url>http://mint.litemedia.se</wp:base_site_url>
	<wp:base_blog_url>http://mint.litemedia.se</wp:base_blog_url>
	...
</channel>

This is your whole Wordpress exported in one file. You will have draft messages, spam comments and even pages in there. All you’re probably interested in is published pages and accepted comments.

If you want to make some bulk action to your blog data, you should do it now. I’m thinking, changing all the absolute paths from old blog address to the new blog address with a quick, search and replace. I forgot to do that, and now have to go through 180 blog posts manually. Not that I mind very much. I had planned to do that anyway.

Import into Orchard CMS

Go to the modules gallery, find and install the Orchard Team Install Export module. As the name of the module surely reveal, it let’s you import and export data into Orchard. To find out what kind of XML Schema Orchard uses, try writing a couple of blog posts, comments and export it. It should look something like this

<!--Exported from Orchard-->
<Orchard>
  <Recipe>
    <Name>Generated by Orchard.ImportExport</Name>
    <Author>Mikael Lundin</Author>
  </Recipe>
  <Data>
    <Comment Id="/Identifier=6066991b882a488da366d1f64e19d36d" Status="Published">
      <CommentPart Author="Mikael Lundin" UserName="Mikael Lundin" Email="myemail@home.se" Status="Approved" CommentDateUtc="2011-05-22T10:54:33Z" CommentText="This is comment number 1. It has no line breaks." CommentedOn="/Route.Slug=what-to-do-before-release" CommentedOnContainer="/Route.Slug=blog" />
      <CommonPart Owner="/User.UserName=Mikael Lundin" CreatedUtc="2011-05-22T10:54:33Z" PublishedUtc="2011-05-22T10:54:33Z" ModifiedUtc="2011-05-22T10:54:33Z" />
      <IdentityPart Identifier="6066991b882a488da366d1f64e19d36d" />
    </Comment>
    <Comment Id="/Identifier=11376d3721144b2ebb51d4da880592f6" Status="Published">
      <CommentPart Author="Mikael Lundin" UserName="Mikael Lundin" Email="myemail@home.se" Status="Approved" CommentDateUtc="2011-05-22T10:55:00Z" CommentText="Here comes comment number two.&#xD;&#xA;It has several line breaks.&#xD;&#xA;&#xD;&#xA;Saluté!" CommentedOn="/Route.Slug=what-to-do-before-release" CommentedOnContainer="/Route.Slug=blog" />
      <CommonPart Owner="/User.UserName=Mikael Lundin" CreatedUtc="2011-05-22T10:55:00Z" PublishedUtc="2011-05-22T10:55:00Z" ModifiedUtc="2011-05-22T10:55:00Z" />
      <IdentityPart Identifier="11376d3721144b2ebb51d4da880592f6" />
    </Comment>
    <BlogPost Id="/Route.Slug=first" Status="Published">
      <TagsPart Tags="" />
      <CommentsPart CommentsShown="true" CommentsActive="true" />
      <RoutePart Title="New blog on litemedia.info" Slug="first" Path="first" />
      <CommonPart Owner="/User.UserName=Mikael Lundin" Container="/Route.Slug=blog" CreatedUtc="2011-04-16T08:18:28Z" PublishedUtc="2011-04-17T19:59:18Z" ModifiedUtc="2011-04-17T19:59:18Z" />
      <BodyPart Text="&lt;p&gt;I will move all the blog posts from mint.litemedia.se to litemedia.info&lt;/p&gt;&#xD;&#xA;&lt;p&gt;I hope this will result in&lt;/p&gt;&#xD;&#xA;&lt;ul&gt;&#xD;&#xA;&lt;li&gt;More readers&lt;/li&gt;&#xD;&#xA;&lt;li&gt;Easier management&lt;/li&gt;&#xD;&#xA;&lt;li&gt;Better design&lt;/li&gt;&#xD;&#xA;&lt;/ul&gt;" />
    </BlogPost>
    <BlogPost Id="/Route.Slug=what-to-do-before-release" Status="Published">
      <TagsPart Tags="tag1,tag2,tag3,tag4" />
      <CommentsPart CommentsShown="true" CommentsActive="true" />
      <RoutePart Title="What to do before release" Slug="what-to-do-before-release" Path="what-to-do-before-release" />
      <CommonPart Owner="/User.UserName=Mikael Lundin" Container="/Route.Slug=blog" CreatedUtc="2011-04-19T17:46:23Z" PublishedUtc="2011-05-23T19:17:38Z" ModifiedUtc="2011-05-23T19:17:38Z" />
      <BodyPart Text="&lt;p&gt;Things that needs to be done before release of the blog&lt;/p&gt;..." />
    </BlogPost>
  </Data>
</Orchard>

Now we have our data in Wordpress xml format, and we would like to transform it into Orchard xml format to import it into our new blog. For that we will use my favorite tool.

Transforming the export data into import data

We use XSLT to transform from one xml format into another. We could use ordinary scripting, but xslt makes it so easy. Here’s the script that I used. Excuse me for the VBScript part, but I got lazy and took the simple way out when I had to transform date formats.

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0"
	xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
	xmlns:wp="http://wordpress.org/export/1.0/"
	xmlns:msxml="urn:schemas-microsoft-com:xslt"
	xmlns:vb="#VBCustomScript"
	xmlns:content="http://purl.org/rss/1.0/modules/content/">
    <xsl:output method="xml" indent="yes"/>

	<!-- Yes this is ugly but I didn't have the energy to solve it with Xslt 1.0 -->
	<msxml:script language="VBScript" implements-prefix="vb">
	<![CDATA[
	function gmtToUtc(str)
	gmtToUtc = Split(str)(0) & "T" & Split(str)(1) & "Z"
	end function
	]]>
	</msxml:script>  

	<!-- A function to join values together with a seperator
		 Example join(tags, ','
	-->
	<xsl:template name="join">
		<xsl:param name="list" />
		<xsl:param name="separator"/>

		<xsl:for-each select="$list">
			<xsl:value-of select="." />
			<xsl:if test="position() != last()">
				<xsl:value-of select="$separator" />
			</xsl:if>
		</xsl:for-each>
	</xsl:template>	

	<!-- Main entry Point -->
    <xsl:template match="/">
        <Orchard>
		  <Recipe>
			<Name>Transformed export from Wordpress</Name>
			<Author>Mikael Lundin</Author>
		  </Recipe>
		  <Data>
			<!-- Comments: Only approved ones -->
			<xsl:apply-templates select="//channel/item/wp:comment[wp:comment_approved='1']" />
			<!-- Blog items: Only published ones -->
			<xsl:apply-templates select="//channel/item[wp:status='publish']"/>
		  </Data>
		</Orchard>
    </xsl:template>

	<!-- Render a comment -->
	<xsl:template match="wp:comment">
		<!-- Comment publish date -->
		<xsl:variable name="date" select="vb:gmtToUtc(string(wp:comment_date_gmt))" />
		<!-- Parent identifier -->
		<xsl:variable name="parentSlug" select="../wp:post_name" />
		<!-- The comment ID -->
		<xsl:variable name="identity" select="wp:comment_id" />

		<Comment Id="/Identifier={$identity}" Status="Published">
			<CommentPart Status="Approved" CommentDateUtc="{$date}" CommentedOnContainer="/Route.Slug=blog" CommentedOn="/Route.Slug={$parentSlug}">
				<xsl:attribute name="Email"><xsl:value-of select="wp:comment_author_email"/></xsl:attribute>
				<xsl:attribute name="Author"><xsl:value-of select="wp:comment_author"/></xsl:attribute>
				<xsl:attribute name="CommentText"><xsl:value-of select="wp:comment_content/text()"/></xsl:attribute>
			</CommentPart>
			<CommonPart CreatedUtc="{$date}" PublishedUtc="{$date}" ModifiedUtc="{$date}" />
			<IdentityPart Identifier="{$identity}" />
		</Comment>
	</xsl:template>

	<!-- Render Blog item -->
	<xsl:template match="item">
		<!-- Blog post identifier -->
		<xsl:variable name="slug" select="wp:post_name" />
		<!-- Publish date -->
		<xsl:variable name="date" select="vb:gmtToUtc(string(wp:post_date_gmt))" />

		<BlogPost Id="/Route.Slug={$slug}" Status="Published">
			<TagsPart>
				<!-- Render parts -->
				<xsl:attribute name="Tags">
					<xsl:call-template name="join">
						<xsl:with-param name="list" select="category[@domain='tag']/@nicename" />
						<xsl:with-param name="separator" select="','" />
					</xsl:call-template>
				</xsl:attribute>
			</TagsPart>
			<CommentsPart CommentsShown="true" CommentsActive="true" />
			<RoutePart Slug="{$slug}" Path="{$slug}">
				<xsl:attribute name="Title">
					<xsl:value-of select="title" />
				</xsl:attribute>
			</RoutePart>
			<CommonPart Owner="/User.UserName=Mikael Lundin" Container="/Route.Slug=blog" CreatedUtc="{$date}" ModifiedUtc="{$date}" PublishedUtc="{$date}" />
			<BodyPart>
				<xsl:attribute name="Text">
					<xsl:value-of select="content:encoded/text()" />
				</xsl:attribute>
			</BodyPart>
		</BlogPost>
	</xsl:template>
</xsl:stylesheet>

Now you add the following line on the top of the Wordpress xml file, on line 2, under the xml declaration.

<?xml-stylesheet type="text/xsl" href="import.xslt" ?>

At this point it would be pretty simple to create an import from wordpress Orchard module, but I just want to solve my problem and move on. That is why I open my Wordpress xml-file in Internet Explorer and let that transform it for me.

Press F12 and you will have the ability to save the whole transformed file to disc.

Now you can use that file to import your goodies into Orchard. Don’t forget to move wp-content into your  media library also and change all the media links accordingly.

Not very hard, was it?

When you want to create thousands of test cases you don’t use the [TestCase] attribute in NUnit. But did you know that there is a “TestCaseSource” attribute that specifies a method that will generate test case data?

Look at this.

[TestCaseSource("MassiveAmountOfUsers")]
public void ShouldLogin(string username, string password, bool expected)
{
    /* Setup */
    var repository = new UserRepository();

    /* Test */
    var result = repository.Authenticate(username, password);

    /* Assert */
    Assert.That(result, Is.EqualTo(expected));
}

It is an integration test that will try to login a massive amount of users. This is the kind of test where you take a huge diversity of data and try to find out if there is anything that will make it break.

But where does the data come from? The magic string “MassiveAmountOfUsers” holds the answer.

private IEnumerable MassiveAmountOfUsers
{
    get { return GetMassiveAmountOfUsers(); }
}
private IEnumerable GetMassiveAmountOfUsers()
{
    var doc = XDocument.Load("users.xml");
    return
        from user in doc.Descendants("user")
        let username = user.Attribute("username").Value
        let password = user.Attribute("password").Value
        let expected = user.Attribute("success").Value
            .Equals("true", StringComparison.InvariantCultureIgnoreCase)

        select new object[] { username, password, expected };
}

Test cases will be generated by the framework calling the property MassiveAmountOfUsers. This should return an IEnumerable of an array of arguments. We create that array of arguments by reading an XML file. To enable using different types, string and bools, as arguments we create an untyped array of objects. As long as we put strongly typed members into the array, we can use strong types in the test function.

What the xml looks like, is unimportant, but in my test case like this.

<?xml version="1.0" encoding="utf-8" ?>
<users>
  <user username="fsse" password="dj7sihfs" success="true" />
  <user username="hgtd" password="sd122?=s" success="true" />
  <user username="asde" password="!!sf3mff" success="true" />
  <user username="bsfd" password="--sdfj+?" success="true" />
  <user username="aefb" password="!#¤%/(sd" success="true" />
  ...
</users>

You could get the test data from anywhere, the database if you want to. Testing your code with an massive amount of real data, really gives you confidence with your code.

Running this in the NUnit test runner looks pretty much like any [TestCase] suite, but the test cases are generated when the test assembly is loaded.

Today I held a seminar about functional programming at Valtech Tech Day. It’s no coincidense that the title of the seminar is the same as the book by Tomas Petricek, since it inspired me to do the talk. After playing around with F# for over a year, it’s time to get serious and use it in production case.

You will find my slides here. (or download as pdf)

Do we need functional programming?

Yes! Our CPU’s aren’t getting faster, but better at doing several things at the same time. That is why we need to focus on parallel programming, and parallel programming is hard to do with imperative programming. That is why we must rethink our problem definitions and make them much more expressive, solving our problems with functional concepts.

What functional programming language should i learn?

If you’re on .NET today, F# will be the shortest path to functional programming for you. If you’re a hardcore Java developer you should look into Clojure. Erlang seems to be the coolest functional language right now, but Scala is also an alternative.

Is recursion the answer to everything?

Some people does not like recursion because they’re so used to iterative way of thinking. There’s nothing hard about recursive programming, but your brain is not tuned for it. Recursion is not the solution to everything, just as the foreach/while-loop is not the answer to everything, but I would say that its just as important as a looping construct.

Monads, gonads?

Ehhm … yeah.

It’s no secret that I’m working on a replacement for this blog in Orchard CMS, and while doing so, learning to deploy applications to Azure. There is a guide provided by the Orchard Team, but it takes for granted that you’re using Orchard as a plug’n'play CMS and not as a development platform. Here’s a small recollection of my findings.

The Orchard CMS is pointed at the Wordpress audience, comfortable with just installing a blog platform, choose theme, widgets and be up and running without any thought on code, deployment or testing. If you don’t want to author your own theme or write your own modules, you really should download the Azure package and be on your way.

But what if I’m a developer?

Project structure

As a developer I want to have my project setup in Visual Studio. I rename the default Orchard project and add it to a solution together with an Azure deployment project. This is all pretty trivial.

When it comes to compiling you will notice that there are a whole bunch of binaries in the bin-folder that “has to be there”. This is of course to support the xcopy deploy that wordpress people are familiar with (PHP does not have binaries). You should be able to take the package as a whole and dump it on a host to setup your first Orchard site.

I do however want a clean bin-folder that is rebuilt on every compile. How can we accomplish that?

Extract dependencies with NuGet

I use the NuGet package manager to solve out the binary dependencies. Sadly there is no Orchard package yet, but I assume that there will be in the future. Until then we’ll have to sort out the dependencies.

• log4net 1.2.10
• SqlServerCompact 4.0.8482.1
• Autofac 2.2.4.900
• Autofac.Mvc2 2.2.4.900
• Iesi.Collections” 1.0.1
• Antlr 3.1.3.42154
• Castle.Core” 1.1.0
• Castle.DynamicProxy 2.1.0
• NHibernate 2.1.2.4000
• NHibernate.Linq 1.0

Because you need specific versions of each library, you’ll have to add these through the Package Manager Console (View Menu / Other Windows / Package Manager Console). Example

Install-Package log4net -Version 1.2.10

This leaves us a couple of libraries that are not available in NuGet. I just moved these binaries to the package folder in my solution and referenced them as “Copy Always”. Here’s a complete list

• Autofac.Configuration 2.2.4.900
• FluentNHibernate 1.0
• Orchard 1.1.30
  - Orchard.Core.dll
  - Orchard.exe
  - Orchard.Framework.dll
  - Orchard.WarmupStarter.dll
• ClaySharp 1.0
• System.Web.Mvc 3.0
  - Microsoft.Web.Infrastructure.dll
  - System.Web.Helpers.dll
  - System.Web.Mvc.dll
  - System.Web.Razor.dll
  - System.Web.WebPages.Deployment.dll
  - System.Web.WebPages.dll
  - System.Web.WebPages.Razor.dll

Yes, I added the MVC framework as a library, because I don’t know if the host has MVC3 installed in GAC. This would also prevent any collisions with MVC2 because binaries in the bin-folder has precedence over GAC.

The bin folder should now be recreated for you at compilation.

Database deployment

Connecting to your Azure database is easy, just add a firewall rule in Azure and connect with SQL Management Studio using the details available in the Azure management console.

But since we are developers we would like to work locally with functionality in a development database, and deploy to the production database. I prefer to work on a SQL CE4 database, because I can commit the whole thing to source control. This will not scale well with several developers in the team, because the database won’t merge, but as a sole developer I will get the whole database under centralized revision control – and that means a lot to productivity.

The workflow looks like this

1. Dump your SQL CE4 database
2. Import into Azure

Easy!

SQL compact toolbox

You should download the SQL compact toolbox and install it. This will allow you to script your entire CE4 database to SQL that will easily be imported into Azure DB.

Azure connection string

But where is that damn database configuration string? I have no idea why, but in App_Data/Sites/Default is a file called Settings.txt that looks like this. I think the reason has to do with the same Orchard instance should be able to host several “sites” with different databases.

Name: Default
DataProvider: SqlCe
DataConnectionString: null
DataPrefix: null
RequestUrlHost: null
RequestUrlPrefix: null
State: Running
EncryptionAlgorithm: AES
EncryptionKey: 471CAA5...
HashAlgorithm: HMACSHA256
HashKey: 54644AEBB...

Before you deploy to Azure you should change it to something like.

Name: Default
DataProvider: SqlServer
DataConnectionString: Server=tcp:[servername].database.windows.net;Database=[dbname];User ID=[username]@[servername];Password=[password];Trusted_Connection=False;Encrypt=True;

Service configuration

There is  nothing strange going on in the ServiceConfiguration.cscfg. At first I copied the one configuration file from Orchard CMS Azure Deployment Guide but it seems to be a bit outdated. I got warnings that my schema was obsolete.

<?xml version="1.0"?>
<ServiceConfiguration serviceName="OrchardCloudService" osVersion="*"  xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration">
  <Role name="Orchard.Azure.Web">
    <Instances count="1" />
    <ConfigurationSettings>
      <Setting name="DataConnectionString" value="UseDevelopmentStorage=true" />
    </ConfigurationSettings>
    <Certificates />
  </Role>
</ServiceConfiguration>

I changed this to make it a bit more expressive but I have no idea how to get it to conform to the latest version of the schema. I’m still new to Azure and not 100% sure about the configuration, but this works for my website.

<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="LiteMedia.Web.Azure" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
  <WebRole name="LiteMedia.Web" vmsize="ExtraSmall">
    <Certificates>
    </Certificates>
    <ConfigurationSettings>
      <Setting name="DiagnosticsConnectionString" />
      <Setting name="DataConnectionString" />
    </ConfigurationSettings>
    <Imports>
      <Import moduleName="Diagnostics" />
    </Imports>
    <Endpoints>
      <InputEndpoint name="HttpIn" protocol="http" port="80" />
    </Endpoints>
    <LocalResources>
      <LocalStorage name="media" cleanOnRoleRecycle="false" sizeInMB="100" />
    </LocalResources>
    <Runtime executionContext="elevated">
    </Runtime>
    <Sites>
    </Sites>
  </WebRole>
</ServiceDefinition>

Speeding up warmup through precompilation

Oh my god! It takes forever to deploy my website! 5 minutes are spent only starting my website. I’ve seen complaints about this all over the Internet. Some have reported that Orchard CMS takes 15 minutes to boot up on a shared host. This is because a lot of the plugins has to be dynamically compiled at startup. If your site have few visitors, it will sleep and restart even more often making your website sluggish. Can we fix this?

Yes, we can extract projects from the Modules folder and statically compile them into the application. There is really no need for them to be dynamically compiled after each application restart.

You really have to look out for not moving out all the files, but only those that will compile into a binary. So, leave stuff like module.txt, web.config and any other resources like images and cshtml.

This will decrease startup time, but you can trim it even more if you ILmerge the modules prior to deploy. Make it a part of your automated build/deploy script and get a lot of startup improvement.

That’s it for now. Hopefully this will help someone in the same position as I am.

Working with WebForms I often use ASHX handlers for generating stuff. It can be anything from XML to CSV.

This is very lightweight and I like it, but today I ran into a problem where I wanted to generate a CSV for Microsoft Excel, but Excel could not recognize the UTF encoding and tried to parse it as ANSI.

Looks like crap because Excel can’t tell that we’re feeding it UTF8. This can be fixed by explicitly giving the file a BOM (byte order mark).

public void ProcessRequest(HttpContext context)
{
    context.Response.ContentEncoding = Encoding.UTF8;
    context.Response.ContentType = "text/csv";
    context.Response.AppendHeader("Content-Disposition", "attachment;filename=data.csv");

    // Start the feed with BOM
    context.Response.BinaryWrite(Encoding.UTF8.GetPreamble());

    var data = new[]
    {
        "Namn;Land;Poäng", // Name;Country;Points
        "Mikael Lundin;Sverige;1200",
        "John Smith;US;800",
        "Jean-Pierre Bordeaux;Française;600"
    };

    foreach (var rows in data)
        context.Response.Write(rows + "\n");
}

The magic happens at line 8 where we explicitly write the BOM to the stream.