The Software Developer Identity Crisis

This is post tries to explain some of what I was talking about in my talk at weactuallybuildstuff.com, it’s series of two posts, I’ve tried write it as one post but it got too long and icky, so I’ve split it into two.

I believe at the heart of Software Development culture there is an identity crisis. I say this because I believe software development to be an engineering discipline, based on science, but many of fellow developers do not believe this. They say it is an art form or a craft.

To support my argument I think first it would be useful to look at what engineering is. Wikipedia currently defines engineering as:

Engineering is the application of scientific, economic, social, and practical knowledge in order to design, build, and maintain structures, machines, devices, systems, materials and processes

In software, we’re trying build stuff so, it seems clear to me that following engineering practices is necessary. But what are good engineering practices? This is where the science bit comes in. Wikipedia currently defines science as:

Science (from Latin scientia, meaning "knowledge") is a systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions about the universe.

So science is how we organize knowledge, so we can use science and particularly the scientific method to investigate what are good software engineering practices. Accept, I’m left with the overwhelming feeling that we don’t do this. With open source we have become very good at sharing building blocks for software, this is a good thing and I think it’s something our industry can be proud of, but it isn’t enough. To again more insight into best practices we need industry and write more papers and reports on successful and unsuccessful projects (understand the reasons for failure is very import too). Industry also needs to get better about publish statics about projects, and knowing which statistics to collect. We also need a better relations between academia and industry so academics can take up the challenge of performing meta-analysis once this data is available. I not knocking academia, I believe there’s some good working been done out there, but what also needed is a different kind of work. I think more people need to do the for software what epidemiologist do for medicine. In medicine an epidemiologist is someone how looks for health best practices by studying the available data, think computer science desperately needs someone to pull together the available data on software engineering and help tease out what the best practices of our industry really are. (Apologies if you are an academic working in the way I describe, but I don’t feel your voice is being heard)

When I talk about this there are two arguments I often hear people use against software needing science and engineering, so I’d like to deal with them briefly here. The first is that software can’t be engineering as it is an art/craft. I think this is wrong as it false dichotomy. Many artists and crafts folk rely on science and engineering, they often spend years learning the details of how things look and techniques to reproduce them in there chosen medium. If you want to produce giant sculptures as Louise Bourgeois did, then you’ll need to know a thing or two about engineering, though if your Damien Hirst you may outsource this. Interestingly, for perhaps the most famous artist, Leonardo da Vinci, there was no difference between art and science. Leonardo often drew the world in an attempt to understand it better and also to express his ideas about science and technology. The second I often hear against the use of science/engineering in software is that software engineering is simply too complex to be study by science. While it’s true than there are many factors that effect what makes a successful software project and this makes it difficult for scientist to pull part how to build software successful project and give it back to us in easy to understand chunks, not to even try seems a little defeatist. After all, where people have applied the scientific method with rigor we have learn a great deal on many complex subjects.

For what it’s worth I’m not saying current trends in software development, like agile and craftsmanship, are wrong, I merely saying we’re not doing enough to check they are taking us in the right directions. As Scott Hanselman pointed out there’s clearly room for improvement. So, summary, don’t be afraid to follow intuition and draw inspirations form many other unrelated domains, but always fall back on scientific method to check these intuitions are leading in the right direction.

The follow will “How We Fool Ourselves: Why Intuition Isn't Enough” will be published shortly (at my current blogging rate “shortly” means before end of 2015).

(Interesting aside: I found this article from Joe Duffy “Bluring the line between research and development” had and interesting and not dissimilar view of the relationship between industry and academia)

Using the Android GridView as a DataGrid

As part of the app I developed for AngelHack I wanted to display some data in a classic grid style. Android offers a GridView as part of the basic SDK, however if you look at all the samples they only show how to use it as a grid of images, essentially they only demonstrate usages where all columns show the same thing, usually columns of photographs. It’s not too tricky to use the GridView as classic “data grid” but it did require a wee bit of thinking, so I thought I put together a quick example to show how it worked.

In many ways working with Android controls reminds me of working with WFP. They both allow a user to either create controls using an xml based designer or create them directly in code. The way data-binding works is similar too, for Android controls you plug in a “data adapt” which controls the creation of the child controls that will handle displaying the data. To use the GridView we need to create an adapter to which is responsible for telling the GridView how the data source should be rendered. Initially for the data source I created mine own “DataTable like object”, but then after saying d’oh and slapping my head a couple of times I realized there was no need to create “DataTable like object” when I could just use a DataTable.

The first step is to create the adapter, to do this you derive from BaseAdapter<T> where T is the type of element that will appear in the data cells, for simplicity I made T string. Next you need to provide a constructor that takes a Activity (which is an Androidy type thing for accessing the gui context) and the data source itself, in our case a DataTable. This means the top of the adapter class looks something like:

public class DataTableGridAdapter : BaseAdapter<string> {
    private Activity context = null;
    private DataTable itemTable = null;
    public DataTableGridAdapter(Activity context, DataTable itemTable) : base ()
    {
        this.context = context;
        this.itemTable = itemTable;
    }

Next you need to override the count property to return the number of cells (the rows plus one gives room for column headers):

public override int Count
{
    get { return (itemTable.Rows.Count + 1) * itemTable.Columns.Count; }
}

Then you need to override the GetView method, which is the method that creates the control that will be displayed for the cell:

public override Android.Views.View GetView (int position, Android.Views.View convertView, Android.Views.ViewGroup parent)
{
    // find the current row number, need to test we're in the header
    int row = GetRow(position);
   

    // Get our text for position
    var item = PositionToString(position);

    // find if we have a control for the cell, if not create it
    var txtName = convertView as EditText ?? new EditText (context);
   

    // if the row is a header give it a different colour
    if (row == 0) {
        txtName.SetBackgroundColor( new Color(0x0d,0x12,0xf5));
        txtName.SetTextColor (new Color(0x0,0x0,0x0));
    }
   

    //Assign item's values to the various subviews
    txtName.SetText (item, TextView.BufferType.Normal);
   

    //Finally return the view
    return txtName;
}

The important line is '”var txtName = convertView as EditText ?? new EditText (context);” here we see if the control to display the cell has been created or not, if not we create one. In this case we use an EditView, a control build-in to the Android SDK for editing text. This is because we want our grid to be editable, if we wanted a readonly grid we’d use a TextView instead. We we wanted to display complex, more structured data we could return a user defined control instead. The rest of the method is just about setting up the various properties of this control, including it’s content, which we obtain from our data source via the PositionToString method. The position is the identifier of the cell in the grid, the data grid doesn’t provide a row and column id, just one continuous position id, fortunate with just a little maths we can translate a position to a row and column pair. Below shows how we do this and also the implementation of “PositionToString”.

private int GetCol(int pos){
    int col = pos % (itemTable.Columns.Count);
    return col;
}
private int GetRow(int pos){
    int row = pos / (itemTable.Columns.Count);
    return row;
}
private string PositionToString(int pos)
{
    int row = GetRow(pos);
    int col = GetCol(pos);
    if (row == 0) {
        return itemTable.Columns[col].ColumnName;
    }
    if (itemTable.Rows [row - 1][col] != DBNull.Value) {
        return itemTable.Rows [row - 1][col].ToString();
    }
    return "";
}

There’s a few wall-dressing method that need to be implement, but I’m put the code on github, so you can take a look at them here.

To use the “DataTableGridAdapter”, simply add a GridView to one of you’re layouts, a .axml file, then you’ll need to configure the data adapter in the Activity that uses the layout. To do this first recover a reference to the GridView and store it as a member:

gridView = FindViewById<GridView> (Resource.Id.offerGridView);

Once you’ve created your DataTable and populated it, binding to the grid as so:

var itemListAdapter = new Adapters.DataTableGridAdapter (this, dataTable);
gridView.NumColumns = dataTable.Columns.Count;
//Hook up our adapter to our ListView
gridView.Adapter = itemListAdapter;

The slightly annoying thing about this approach is you need to explicitly set the number of columns the grid has and you do need to keep updating the number of columns each time a new column is added or removed.

And that’s basically it, take a look at the full working example here, and feel free to send a pull request if you have an improvements.

Developing For Kindle Fire with Mono C#/Xamarin Studio

As I’ve already mentioned last weekend I took part in AngelHack and took the opportunity to learn how to develop for an Android device (in this case a Kindle Fire) using C# and Xamarin Studio. The experience was a bit mixed, overall I think the folks at Xamarin have done a good job in smoothing the experience when targeting mobile devices, but there’s still a few rough edges. Here’s a run though of what I thought:

The installer for Xamarin Studio works well and does a good job of downloading and installing the large number of different components need to develop for various devices. However, it’s slow, not so much the download which came down surprisingly quickly but the installation took ages with the window’s installer promising “6 seconds remaining” but in reality I was still waiting minutes later.

Once I was past the installer I downloaded the tasky sample and tried to open in Visual Studio but all the projects refused to load. I tried both with VS2012 and the “Shell” version of VS2010. I think this is because I’d installed the Indie version of Xamarin Studio, which doesn’t have Visual Studio integration, so in this case my fault, but one might have reasonable expected a clearer error message about why the projects weren’t loading. This is especially true as the Visual Studio 2010 shell seems to have been installed by the Xamarin Studio installer (at least I have no memory of trying to install VS2010 shell on this machine).

Once I open the tasky project files in Xamarin Studio things started looking up. I soon got tasky compiling and working the emulator. However by default the Android emulator is setup to emulate a phone and I wanted to target a Kindle Fire, there are various other configurations of the emulator but none of them appropriate for Kindle development. Amazon has detailed instructions about how to setup your development environment to target kindle fire, however there only for folks using eclipse. Xamarin Studio installs the same set of tools described in the article but there installed under C:\Users\<username>\AppData\Local\Android\android-sdk in Windows, so it took me a while to figure out where they were (C:\Users\<username>\AppData\Local seems an odd place to install them to).

UPDATE: I’ve just installed XamarinStudio on another box and I noticed the installer asks you were you want to put the Android SDK, obviously this didn’t strike me as important information at the time and I just did “next > next > next”.

Once you’ve found the tools, you need to use the “SDK Manager.exe” to installing the following extra bits for Kindle Fire development (you’ll need to add a reference to “http://kindle-sdk.s3.amazonaws.com/addon.xml” under “Tools > Manage Add-on Sites … > User Defined Sites” before they are available). Then install:

- “Kindle Fire Device Definitions”

- “Kindle Fire USB Driver” (this doesn’t really install the driver it just downloads it, you then need to find the exe and double click it)

- “Intel x86 Emulator Accelerator (HAXM)” (again doesn’t install automatically, you need to double click the exe, and it wasn’t supported by my processor, but well worth doing if your processor does support it)

Before I had installed all of this a I tried “side loading” (directly dragging and dropping my compiled .apk via a USB connection) on to a kindle I’d borrowed. This didn’t work and the app just closed without display an error message. I strongly suspect that this was because I hadn’t uncheck the “Use shared runtime” option, but I haven’t tried “side loading” since. After I installed the “Kindle Fire USB Driver” I just let Xamarin Studio handle the deployment and it just worked.

Working with the Android emulator is slow and the emulator is a little crashy. The emulator seems especially prone to crashing after sleeping/unsleeping my laptop by closing/opening the lid. Although my laptop doesn’t support the “HAXM” accelerator checking the the “Use Host GPU” under the “AVD Options” ( AVD is Android Virtual Device) did seem to speed things up a lot. Starting the emulator and deploying to it is the slowest bit. There is a “Quick deploy” option which is check by default but that just produced strange error messages for me and after a bit of googling the suggested fix seemed to be “if it ain’t working turn it off”. Unchecking the “Use shared runtime” also slows things enormously, so make sure it’s checked during development. I had one other problem with the emulator which turned out to be a user error. When I first use the emulator it would go straight to the start screen of the app. Later, for reason I don’t understand, it started to show the lock screen of the Android OS, along with a “50% Loaded” message. I wasted a lot of time waiting of the “50% Loaded” to disappear, but after several restarts I realized the “50% Loaded” message was a red herring the emulator was fully loaded, I just need to unlock it by dragging the flashy circle over the lock sign.

Another problem that gave me a slow start to my development was that although the tasky project worked fine on both really hardware and the emulator I couldn’t for the life of me get a project I’d created via “File” > “New Project” to loaded into the emulator (I never bothered trying real hardware). When I tried to deploy, the app would seem to deploy but then exit straight away, I tried putting a break point in the first screen, it was never hit. I’m not sure what the problem was, I noticed the app didn’t have a manifest so I tried creating one, but it didn’t seem to make any difference. In the end I just gave up and used a copy of the tasky project to create my app.

The final glitch was related to libraries. I wanted to parse some JSON, easy right? As it turns out a bit of a struggle. I wanted to use newtonsoft.json.dll to do the job, but it you can’t use any old .NET binary to target an Android app, if the binary hasn’t been build specifically for Mono/Android you end up with weird error messages about object being from the wrong mscorlib.dll. I looked around for a version of newtonsoft.json.dll that targeted Android, but only found some old source, no binaries. Next I tried to use “system.runtime.serialization.json.datacontractjsonserializer” but this isn’t included in the Android libs. The one that I found that actually did the job was “ServiceStack.Text”, it’s true that service stack doesn’t provide binaries for Android, but it does at least supply a csproj in the main repo to build the source to target Android, so a big thanks to Demis Bellot for providing this.

UPDATE: Dave Thomas just pointed out that newtonsoft.json.dll is available in the Xamarin “Component Store”: http://components.xamarin.com/

Despite the problems I’ve laid out here overall I was very pleased with the experience of using Xamarin studio to target Android and now I have some Android hardware I may go on to build some apps. It’s true that I did encounter a lot of problems, but I hope to create bug reports and help get these solved. I didn’t add F# into the mix this time around, but I shall definitely be trying it out when it comes to building my next app.

AngelHack Paris 2013

AngelHack Paris is the first “hack day” I have every attended, so I didn’t really know what to except. Early on Saturday morning I found myself in a slightly run down area just outside the “peripherique” road that marks the boundary of Pairs. The event was hosted at “cifacom” a colleague for digital design and art, I was quite sure what to make to their campus, about half seemed to swish lectures theaters and the rest seemed to have been basically left as the old warehouse that it was before it is was converted.

My objectives for the weekend were to learn some new technology, I wasn’t sure what, so before the kickoff presentations I experiment with a little node.js, but became a little frustrated when a tutorial just died with cryptic errors. During the kickoff presentation I learn that the prizes were much better that I had originally thought, I’d just assumed that the weekend objectives were to build something for the fun of it, but I found out 1st prize was a trip for the whole winning team to go to San Francisco, and not just to enjoy the sites, the organizers would arrange meetings with venture capitalist who might be interested in the project. Other prizes were on offer too, the event several sponsored by several large tech firms and most of them were offering prize for the best use of there technology. When I learned to that Amazon was offering a Kindle fire for the best app on it’s Kindle I decided to that it was time to try out Xamarin Studio and make me an Android app. I join a team on the basis that I could do a front end on the Kindle for their web application. The winners of all prizes would be decided by a 2 minute presentation given by the team.

After the kick off presentation we basically found a quiet corner and began to shape our project. I installed Xamarin Studio while we discussed the project specs and who would do what. Setting up the dev environment took a little longer that I hoped and it was only really about 7pm that I was in a position to start coding. I had to head home just after that, but many people left in the early hours of the morning and some stopped around all night. The organizers did a great job in providing things developers like. There was a ready supply of sandwiches, pizza and beer. There was also a masseur providing back massagers and toy helicopters and foam dart guns. Sadly, people were just starting to turn there attentions from their projects to the toys as I was learning, and apparently after mid-night there was a period of full blown urban warfare. I only saw the photos.

Sunday morning was cold and bright and I got up early again to the 7:11 train, only to find that didn’t exist on a Sunday. So I sat coding and shivering on the platform till the 7:41 eventually arrived. I made much more progress Sunday morning and by the 2pm deadline I had a kindle app that I thought at least demonstrated the idea of our project. The rest of the team had done a good job on assembling the web app, datastore and a well polished presentation. At 2pm sharp we assembled in the meeting room to give our presentation and see what the others had done. This was one of my favourite parts of the weekend. There were many projects who’s presentations I enjoyed watching. I was very impressed by the Loopifive app that won the competition, the concept is interesting, a platform that allows musician’s to share and remix there compositions and their presentation was very slick. I did win the best Kindle prize, but it was a bit by default as no-one else built a kindle app.

Here’s a photo of us presenting our project:

 DSCN2458_cropped_small

Then there was still an ample supply of beers for the post event party. I’m glad I stuck around as it provided and interesting opportunity to meet people dissect the weekend, which had been a bit lacking during the competition as everyone was concentrating on there own projects. A fun weekend, I’d definitely do it again.

Alt.Net Coding Breakfast – April 2013 Edition

Last Wednesday I got the chance to attend a “Coding Breakfast” hosted by Damien Thouvenin of CLT Services. The concept is simple, turn up a little before 8:30, enjoy some coffee and croissant, then we attempt a Coding Kata for about an hour and finally we get together to review the results. I was pleased to see the great and the good of the Paris coding scene there, in the form of Yann Schwartz and Jérémie Chassaing.

The chosen Kata was the “Parking Meter” problem. Create a function to calculate how much time the coins you’ve inserted into the machine buy’s you. The parking meter only accepts 10, 20, 50 cents and 1, 2 € coins. Parking costs 1€ 20 min, 2€ 60mins, 3€ 2 hours, 5€ 5 hours 8€ 1 day. You only need to pay for Parking between 9h and 19h and parking is free on Sundays, public holidays and in august. The problem seems fairly straightforward but the various rules mean there’s actually a fair bit of code need to cover all the cases. Myself and Jérémie choose to do the problem in F# and the others coded in C#. I was the only one to finish in the allotted time (although I did notice and fix a small bug later on) but the other weren’t too far from finishing. I didn’t practice strict TDD which the other’s did, but I did produce a test script that I ran from F# interactive.

I the discussion at the end a couple of interesting points came up, how to handle ‘DateTime.Now’ correctly and whether to store current time paid for and start from this when adding new coins. I believe these are both problems caused by trying to model everything as objects, trying to group things into behaviour plus state, rather than just creating functions that make calculations. The correct was to handle ‘DateTime.Now’ is not use it, you’re function should take DateTime representing the start time and all calculations should be based on this parameter. It’s true you’ll need to observe ‘DateTime.Now’ at some point, but this should be at the last possible moment, probably when you’re handling the “AddCoin” event, or whatever. There is no need to store the current time bought, you just need to calculate it and display it or send it to the ticket printer. The only thing you really need to store is the total of the coins entered, I think it’s good practice to try and store the minimum amount of state possible. The function the calculate the amount of time paid for can be called each time the values is needed, the calculation is so quick it’s not going to cause performance issues, plus you’ll always get a version that has been updated for the current time.

I won’t describe the approach I took, but I’ll show you two version of the problem, the first was what I managed during the allotted time, the second is the result of a 20mins tidy and commenting session, and the various steps of the calculation are explained there.

One last thing before for I show you the code, a big thanks to Damien for organizing the event, I’m looking forward to the next one.

Link to the first version (as I can seem to get github to do an embedded version of a previous revision)

Cleaned up version:

Reasons to come to Functional Programming eXchange 2013

The 2013 edition of Functional Programming eXchange is just over a month away, so if you’re not already signed up now the time to do so. When we first started Functional Programming eXchange back in 2009 there were very few functional programming conference’s aimed at developers, over years this has changed quite dramatically and now even if we just talk about about functional programming events in the UK hosted at Skills Matter we have Clojure eXchange, Haskell eXchange, Progressive F#, Scala Days and Scala eXchange. What makes Functional Programming eXchange different from these other conferences is that rather than focus on a specific functional programming language we have the freedom to bring together the talks from a broad range of functional languages, to bring there respective communities together to swap ideas and it gives us a chance to include some talks from languages that do not yet have there own conferences.

I’m very pleased with the way the schedule has turned out, there were other talks I’d loved to have included, but the schedule already is full to busting! I’m very pleased to have both Simon Peyton-Jones and Don Syme the men who are the driving forces behind the research and development of Haskell and F# respectively. Simon will start the day by giving a “Adventures with Types” in which he’ll give us an overview of some of the latest development in the Haskell type system and Don will round the day off by giving a talk “F# in an Open Source World” in which he’ll talk about the changing way the community contributes to F#.

I’m also very pleased to have a talk from Alain Frisch of LexiFi. LexiFi are one of the pioneers of using functional programming industry and have been helping their customers tackle complex problems in the financial domain using OCaml since the early 2000s. Alain has also made many contributions to the OCaml compiler itself and he’ll be covering both these topics in his talk so it really is one not to miss.

David Pollak and Paul Dale will both the be giving Scala related talks, David will be telling us about the latest developments in his popular web programming framework “Lift” and Paul will talking about his experinces using Scala and the Akka framework in successful real world projects. Robert Rees will be giving a talk that covers both Scala and Clojure and will be giving talk that compares both language with a view to helping developers decided why you might chose one or the other for a specific project.

Nic Ferrier will be keeping fans of lisp style languages happy with his talk on ELNode, a node.js style web server that written in Emacs Lisp, so can be hosted inside the famous emacs text editor.

Adam Granicz also has a talk which pushes the boundaries of the web and text editing, he’ll be showing off an IDE in a web browser for the F# language.

Hope to see you there, it’ll certainly be a day pact full of learning and fun!

Science and Software Development

Today I gave a talk ”Science and Software Development” at the weactuallybuildstuff.com. The talk was largely inspired by Ben Goldacre’s “Bad Science” column and book and try to apply some of the ideas relating to the use and abuse of evidence in the software industry. In the talk I tried to build a case for why the software industry might by to pay more attention to scientific techniques, such as “Randomized Controlled Trails”, by talking about availability bias, Confirmation Bias, the placebo effect and overconfidence. The slides are available on SlideShare:

Additional Material came from The Geek Manifesto by Mark Henderson, Irrationality by Stuart Sutherland and the 6th February 2012 Edition of the Guardian Science podcast. The also a large number of quality blogs talking about science scienceblogs.com and The Guardian’s Science blogs seem to be good jumping off points.

A video of the talk should be available soon, I’ll post the link to this blog when it is.

A huge thanks to Greg Young to inviting me to speak and this new and exciting conference!

Functional Programming eXchange 2013

Well we're doing it again!

 

I've been running a conference "Functional Programming eXchange" in London for a few years now. The conference is run by Skills Matter and will take place on Friday 15th Mar 2013.

 

The idea of the conference is to bring together London's diverse functional programming communities to share ideas and war stories. Most previous talks have focus on concepts from functional programming or feedback from projects that have used functional programming. Previous talks can be seen on line:
http://skillsmatter.com/event/scala/functional-programming-exchange-2012
http://skillsmatter.com/event/scala/functionalpx-2011
http://skillsmatter.com/event/design-architecture/functional-programming-exchange-2009

If you'd be interested in speaking please drop us a line with an abstract to fpx@skillsmatter.com

If you're intested in coming long a you can already order your tickets at the conférence's proto-website: http://skillsmatter.com/event/scala/functional-programming-exchange

Sorting Out What .NET 4.5 Bits Come With VS2012

Visual Studio 2012 has been out for a while now, I’ve been using it every day at work and I’m generally very happy with it. The biggest win is the performance, which is some much better than VS2010.

I’ve finally begun to get curious about the new stuff in VS2012. Visual Studio was delivered with .NET 4.5 a shiny new version of the .NET framework, that will allow developers to create “Metro” style apps (which for reason that are too silly to go into are no longer called Metro style apps, but we will continue to call them Metro style apps in this article as suitable name has yet to be found). You can also develop all the apps you used to be able to develop using .NET 4.5, such as WinForm, WPF, ASP.NET etc.

Metro style apps are the new style of application that will only run on Windows 8. They take up the full screen and are design to be used on touch screen tablet devices, though it is possible to use them from a device that isn’t touch screen. Metro style apps do not run on the Win34/Win64 Windows APIs we have come to know and love, they have their own APIs that are completely isolated from the classic Win34/Win64 Windows APIs. These new APIs are called WinRT, confusingly also the name of the version of Windows that runs on ARM based devices. WinRT will not load executable or .dll that target Win32/Win64, code must be compile specially for WinRT and applications cannot simple be download from the internet, they must be installed via the Windows Store (all though inevitably someone will find a way to hack there way round this restriction). The idea is that WinRT is locked down and completely isolated from the Win32/Win64 world to try and prevent the malware issues that plagued Win32/Win64 ‘anything goes’ world.

As we know the .NET framework is built on top of the Win32/Win64, so how does .NET 4.5 allow developers to create Metro style apps, while still allowing them create apps that run on Win32/Win64? The answer is simple, .NET 4.5 is not one framework but two:

-          .NET 4.5 is an in-place update for .NET 4.0 used by Windows 7 applications, Windows 8 classic desktop mode, Mono-Mac, Mono-Linux etc.

-          .NETCore 4.5 is the core implementation of .NET used by Windows Store apps written in .NET languages.

In VS2012 you know have the possibility to target either framework (or both in the case of building a library), it just depends on the project type you choose. So say you decided to create a .NET library, you now have the following choices:

-          ‘Class Library’ project type, this creates a library that will target .NET 4.5 or any of the older ‘classic frameworks’

-          ‘Portable Class Library’ project type, this creates a library that could potentially be used on any version of .NET, that is the ‘classic frameworks’, Silverlight, XBox and the new .NETCore 4.5 for Windows Store.

-          ‘Class Library (Windows Store Apps)’ project type, this produces a class library that can only be used with .NETCore 4.5 for Windows Store.

-          ‘Windows Runtime Component’ project type, this doesn’t produce a .dll like all the other project types, but instead produces a .winmd file. A file .winmd still uses the same Common-IL (CIL) format as a the other libraries, but winmd library can be used from JavaScript and C++ applications that target Windows Store, were as a .dll produced by a ‘Class Library (Windows Store Apps)’ project can’t be.

I wanted to understand the difference between these projects types. I was particularly interested in what was the difference between the files produced ‘Class Library (Windows Store Apps)’ project type and the ‘Windows Runtime Component’ project type. I could seem to find a good explanation of this on the internet. So I open Visual Studio create project of each project type and then used ildasm to decompile each library so that I could compare them in WinMerge. For those who want to play with this for themselves I’ve uploaded by project here.

This is a summary of what I found and few other tip bits about .NET 4.5:

‘Portable Class Library’

These are very similar to the ‘Class Library’ project type. The only difference is that they create ‘retargetable’ reference to mscorlib. It looks something like this in the text version of IL:

.assembly extern retargetable mscorlib

{

  .publickeytoken = (7C EC 85 D7 BE A7 79 8E )                         // |.....y.

  .ver 2:0:5:0

}

I’ve failed to find a good explanation of ‘retargetable’ keyword does in IL. It seems to be an old idea that’s been in available in IL for a good while as I found a reference to it on this site that was put together around 2005, which is actually an old readme file from an old distribution of F#, but it’s on the top results when you google for ‘extern retargetable mscorlib’. The site does offer a little inside what retargetable does:

Retargetable binaries are neutral w.r.t. the publisher of particular referenced binaries.  For example, a truly portable .NET program will be neutral w.r.t. the publisher of mscorlib - rather than picking up a dependency against the publisher of mscorlib used at compile-time. This means the program will bind to ANY assembly called "mscorlib", which is OK for mscorlib but is not necessarily such a good idea for other strong named assemblies.

So there you go.

The programmer also gets a lot of choice about what can be targeted, they could choose to target .NET 4.5 and .NETCore 4.5 but not Silverlight, for example. To make this possible, Microsoft has create different “profiles”. Each profile allows to target a different set of frameworks for example “Profile1” lets you target .NET for Windows Store apps, .NET Framework 4, Silverlight 4, Windows Phone 7, XBox 360. Microsoft provide a set of reference assemblies that have all the methods that are common across all these frameworks, these live under: C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETPortable\v4.0\Profile\Profile1.

To add extra-meta data to the assembly the msbuild project produces a .cs file you’re temp directory and add this into the build. Here’s an example of the generated file:

using System;

using System.Reflection;

[assembly: global::System.Runtime.Versioning.TargetFrameworkAttribute(".NETPortable,Version=v4.0,Profile=Profile4", FrameworkDisplayName = ".NET Portable Subset")]

A full list of all the different profiles can be found here.

‘Class Library (Windows Store Apps)’ and ‘Windows Runtime Component’

These differ hugely from the ‘Class Library’ project type. In .NET Core mscorlib has been renamed to ‘System.Runtime’. So we seem this import at the top of both files:

.assembly extern System.Runtime

{

  .publickeytoken = (B0 3F 5F 7F 11 D5 0A 3A )                         // .?_....:

  .ver 4:0:0:0

}

Confusingly the .winmd file also imports a version of mscorlib, whereas the .dll produced by ‘Class Library (Windows Store Apps)’ does not:

 .assembly extern mscorlib

{

  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )                         // .z\V.4..

  .ver 255:255:255:255

}

This seems to be because a .winmd file uses the definition of System.Object and System.Type from mscorlib whereas the .dll does not. I have no idea why this is.

For the ‘Class Library (Windows Store Apps)’ project type the renaming of mscorlib to System.Runtime accounts for most of the difference, (since the text IL format is very verbose and the assembly name and names is needed each time a type is referenced).

For the ‘Windows Runtime Component’ project there are more changes. I think these changes are to make the expose the managed class via COM to JavaScript and C++. Any public class in a ‘Windows Runtime Component’ project must be sealed, have no new virtual or abstract members and no events. I assume this is to make it COM compatible. We then see that our class also implements an interface:

.class public auto ansi windowsruntime sealed beforefieldinit TestNet45.Class1

       extends [mscorlib]System.Object

       implements TestNet45.IClass1Class

This implementation is completely implicit, no changes to the C# code where necessary. The implantation of the interface is accounts for most of the rest of the differences in the .winmd file and an ordinary .dll.

Another minor difference is that methods in .winmd file have attributes attached that specify the direction:

  .method public hidebysig specialname rtspecialname

          instance void  .ctor([in] int32 a,

                               [in] string b,

                               [in] float64 c) cil managed

If you want to see a picture of the IL diffs you can see one here.

Other Differences in .NETCore 4.5

I’ve been trying to port some existing code to .NETCore 4.5 and keep it compatible with plain old .NET 4.5. There’s a couple of other difference that make this tricky. Firstly, .NETCore 4.5 assemblies are much more fine grain that their .NET 4.0 equivalents. Assemblies tend to only cover one namespace, so were as most of the System.Threading namespace as in mscorlib before it’s now split over several assemblies:

System.Threading.dll

System.Threading.Tasks.dll

System.Threading.Tasks.Parallel.dll

This can be a little annoying, trying to work out where stuffs gone, but I quickly got used to and it doesn’t make a huge difference. The second problem is much more annoying. It seems Microsoft has taken the opportunity to redesign some parts of the framework, System.Reflection has be heavily affect by this. The CLR team has a good blog post explaining the changes. While I understand the design rational, and even think the new API as are better some respect, it’s still very annoying porting code between the two models. Worse is trying to write code that is compatible with the two models, you end up drowning in ‘#if’ definitions. To add insult to injury some of the more powerful features, like the System.Reflection.Emit namespace have been completely removed.

Hope you’ve found this little tour of the difference between .NET 4.5 and .NETCore 4.5 useful. Happy hacking!

The Brave New World of Software Development on the Microsoft Platforms

Things are changing at Microsoft (or maybe they changed a while ago and it took me all this time to notice). The short story is the company no longer cares whether you use their development tools, they only care that you target their platforms: Windows 8 and Windows Azure.
 
The result is this, Microsoft is in the process of opening up Software Development tool chain. They no longer see themselves as the owners of .NET, or at perhaps more accurately some of the technologies around .NET (i.e. ASP.NET, Entity Framework which are now fully open source and parts of ASP.NET are packaged with Mono). Instead they seem themselves as driving contributor with other contributions from the community and other development companies. It can be seen too in the way that Microsoft has contributed to other projects to help bring them to the Microsoft platform, Node.js and the choice of Hadoop for its default Map/Reduce tool are probably the most prominent examples for this. This new way of doing thinks can also be seen in the way they released TypeScript, full open source and integrated with other Javascript open source projects.
 
Why are they doing this? It’s simple. They want to attract as many developers to the Microsoft Platforms as they can, they no longer care whether they’re using C, .NET or Javascript as long as they’re building Apps for the Windows 8 Store or that run on Windows Azure.
 
One interesting question to ask is “is this a good strategy for Microsoft?” I honestly don’t have an answer for this. Clearly the tablets pose a threat to their traditional cash cows of Windows and Office, so getting developers to develop apps for Windows 8 and Azure ‘using any tool’ seems like a good thing for them. Clearly getting Windows Store Apps developed is vital for them, but with so mobile developers already very busy building stuff for iOS and Android, will the ‘using any tool’ be enough to win them over?  As for Azure it’s difficult to see why they’re chasing it so hard, sure the rental model is very attractive to them because it means a nice steady income rather than a one off sales, but the margins on Azure are so very thin. The figures I’ve heard, and admittedly this is rumors rather than hard facts, is that they’re only making a few cents for every dollar spend on Azure, were as with Windows and Office they’re used to 30% or 40% margins. Microsoft make some money from Visual Studio and MSDN licenses, I guess the ‘using any tool’ strategy threatens this but I think perusing this strategy doesn’t necessarily mean they lose much revenue from Visual Studio/MSDN. Developers like Visual Studio so, will continue to use Visual Studio and buy MSDN Licenses. Plus the money they from MSDN licenses goes in the small change jar.
 
Another interesting question to ask is “is this a good thing for Software Developers?” I think it’s mostly a good thing, but there are problems with the model too. .NET being more open means it’s now easier to use from other platforms. Sure, you’ve always been able to use .NET from other platforms thanks to Mono, but now it’s feasible to use more and more libraries that Microsoft have provide themselves. We don’t get the full benefits of open source, such as contributing patches, small incremental releases and features requests as .NET is not full open yet. There’s also a danger of fragmentation. To be fair .NET is already pretty fragmented with different framework version for targeting Windows Desktop, Windows App Store, Windows Phone, Silverlight, and Xbox, and that’s just the Microsoft versions, not even taking account the Mono ones. There’s also a new interesting development, Miguel de Icaza has recently been talking about extending C# to include non-null types. Although, I like the idea of non-null types in C# I’m not completely sure this is a good idea. A Mono C# compiler that supported a superset of Microsoft C# would only add to the fragmentation. Though, I think this shows a growing confidence at Xamarin, this is something the mono team would have never dreamed of in the Novell days.

Another potential problem is that many developers have always looked to Microsoft for guidance about how to develop software. Typically developers on the Microsoft platform like being told what languages, frameworks and tools to use, now they have more choice than ever: C++, JavaScript, TypeScript and .NET. Maybe this extra choice is a good think and will lead to new generation of developers that think for themselves instead of blindly following Microsoft’s guidance. We can only hope.