I read an article on CodeProject about implementing the repository pattern with Linq To SQL.

I tried that approach because it sounded very nice. I did have some problems with performance and after a few debugging sessions (showing the SQL trace or the DataContext log) I found the problem.

By defining the virtual methods of the repository base class like this: public IEnumerable<T> FindAll(Func<T, bool> exp) the Func will not be (what I call) a delayed method, but it will be expanded immediately. I checked the log from the DataContext and it shows that the table is no longer queries using a where clause. Instead, a full table load is performed and the function is evaluated in memory.

However, the solution is very easy, although this mean you'll have to include the Linq assemblies to your testing bed:

protected T Single(System.Linq.Expressions.Expression<Func<T, bool>> exp)

{

    return Items.Single(exp);

}

The trick is to wrap your functions in a System.Linq expression. Client usage is exactly the same. The performance gain on my project was huge, If you consider that tables are no longer loaded completely into memory.

The full code for my repository base class is in the link below.

Repository.cs (4,64 kb)

Yesterday I saw a demo on the new Dynamic Data contribution to Asp.net 3.5. It literally blew me away (really) because that was what I've been doing for most of my time over the last couple of years.

The base

Basically, we develop custom data driven web sites with Asp.net. Most of the time this also involves some kind of back-end content management. What our mini-cms actually did was the following:

• Inspect the database model
• Enrich that model with metadata such as preferred input method, validation and comments and associations,
• Generate a whole lot of Asp.Net controls from the aforementioned properties,
• Inject a lot of role-based security-stuff,
• Present those forms to the end-user to allow basic site management.

So, what we had to build was a configuration file with all the meta data, the database, and of course the web site that got driven by that data.

Not any more. I'm out of a job.

With Visual Studio 2008, Asp.Net 3.5 framework and Dynamic Data the generic application that we wrote can be thrown out of the window, because with the DataContext designer this is not longer something somebody else will pay us to do. As you can see in the demo, by binding your decorated data model to the starter solution, you have an interface in a matter of minutes.

Fortunately the Internet-facing site cannot be generated yet. And, we can embrace the DynamicData technology to our advantage and create sites even faster.

On to the subject of this post!

Silverlight and Dynamic Data

Now this is all nice, but there is a simple problem. If I wanted to use this technology with Silverlight, In the last couple of weeks, I was working on a Silverlight application for our content management concept, and I was wondering if the object model could be easily extracted with the new LINQ To SQL designer. I would have to transfer all meta data and content through a WCF service, right?

Lucky for me, the DynamicData layer isn't even finished and does not yet generate WCF services. So, in order to build a generic Silverlight application you would have to serialize a lot of unknown types into a Silverlight application that was unaware of those types. That is not really what I wanted. I wanted a pre-compiled application that can be configured with just a single configuration property: the data model.

I decided to give it a go, and here's what I came up with. I will try to explain it one step at a time.

Here's my very basic data model:

Just a news item for a particular category that can have zero or more attachments to it. Nothing fancy.

Using the designer, this stuff generates classes for me with the all the properties and data access methods (select, update, delete etc.) while maintaining all the relations.

Here comes the cool bit: Did you know that the DataContext class actually gives us all the meta data without having to use reflection?  Just drill down into DataContext.Mapping and see that you find there:

(edit: since GetTables is a method, it should be written as .GetTables())

Silverlight does not have this nice API, So I created my own WCF contract to serialize these properties in a separate class. Here's a sample of the type specification that I use to transfer to Silverlight (I left the part out that maps the MetaDataMember stuff to my own classes on purpose).

To serialize an item of a particular type I perform a generic LINQ select on the table, I read all the properties of a data item and put them in a dictionary so that Silverlight can read and display those items:

Notice that for this demo I used the Convert.ToString() method on the value to get a nice presentation of the value. In a future version I will need to put the basic values (int, string, DateTime etc.) in the Values collection.

Of course this is not finished by far. I needed a way to decorate the designed classes with validation, descriptions, etc. Because I do not yet have the DynamicData beta installed on my machine I decided to borrow their approach to decorate generated code. Here's a snippet for the NewsItem class:

Because the data context designer generates partial classes, we have the ability to add functionality to those classes. Since I only need to apply some attributes to the generated class I need to define the class again (as partial) and apply my attributes. Since I cannot redefine the properties of that class I have to create an extension class that shadows the properties of the generated class (very nice idea, DynamicData Team!). To use the attributes of the extension class I do need to use reflection, but that is only once. Here's a video of how it works and how it is used. I just copied the concept to there's little more to tell.

A lot of work still needs to be done but I hope you understand that the rest is just more along the same line.

Looks like I still have a job!

I now have been working 12 years on software development and 8 years asp.net. I consider that to be a substational amount of experience. Because of this realisation, I'd like to share some things I've been learning of the years. I summarized my experiences, so check out this list:

1. If you find the problem, look for the answer.
2. If you find the answer, don't stop looking
3. If you stop looking, experiment. 
4. If you stop experimenting, make it work
5. If it works, redesign
6. If you're still not happy, you didn't understand the problem: back to square 1.

These are some mindblowingly simple steps. but they've always worked for me. They help, and not only in software! Number 1 and 2 are easy. That's (1) smart thinking and (2) google. We'll get to number 3 later on. The numbers 4 through 7 are simple development strategies you can read up on in tons of places.

I do like to elaborate on step 3. Most of my coworkers (and a lot of other people i meet) stop after #2. This usually happens after a couple of minutes or hours using google. By the way, if you know the problem, googling for an answer shouldn't last more than a couple of minutes. This is because google is very smart, but it doesn't know your problem. You're actually not googling for articles on the problem - you're googling for articles about the solution to that problem. This sounds strange because most articles talk about 'I have a problem with this and this', so the good google results should point you to an article or a thread that says 'I found the solution for this and this'. The trick here is to include words like 'solution' or 'best practice', 'I finally got it working' etc. to your search query.

Back to #3. Don't stop researching when you think you found the answer. To put it clearly: Number #3 is where the learning phase starts. Start working on a mind-map of the problem space. If you read enough articles on "How do I create custom controls?" for example, you usually cover a lot of the same subject matter, but if you deviate a little from the primary problem, you'll gain tons of slightly related knowledge. To illustrate, check out the class definition of a asp.net master page. I got there recently because I needed to know the base class of a MasterPage. That was easy to find - I also could've looked that up by pressing a help button in Visual Studio. I said 'for example' didn't I? Now as a case of routine I drilled down to the MasterPage Members, where I found out about two very nice properties named ContentPlaceHolders and ContentTemplates. I didn't know these existed, nor would I have found out about them in another way. Especially if you never think you'd need them. But I did need them. I had about a 100 lines of code in my application to find out what regions my MasterPage contains for a CMS i'm working on, but now I can simply refactor and clean up. The lesson I try to tell you is that browsing a couple of non-relevant links from you article *will* improve your knowledge. As you keep reading linked material you'll very soon connect some dots nobody else has ever thought about.

A very wise professor of mine once stated that testing (by definition) is not a way to find defects, but rather a way to determine the quality of the product.

When I read this post, I had an "Ah yes I know what you mean"-feeling. Testing-after-coding is usually regarded as a negative (or 'destructive role' as so nicely put). The reports from automated test suites such as NUnit usually give a success percentage, whilst the reports I usually receive is in most cases no more than a list of defects. Even better, the defects on these lists rarely relate to some or other feature stated in the requirements documentation. 

Granted, a product near completion will contain a lot of cosmetic bugs and annoyances that are not directly related to the requirements document. It is hardly possible to actually deliver a 100% bug-free product. But please focus on the quality please! If the product does what needs to be done, but crashes when you press Shift+Ctrl+Alt+F7+LMouseClick on the 2nd pixel from above, is that considered a critical defect? I think not and whoever was testing that at the end of the project should be sacked, or congratulated on the fact that he had some spare time because apparently all important work was finished. Unfortunately, this is rarely the case.

Instead, if the dev team has its' planning done right there is always some time available for a bug-squat-week.

Do not misunderstand me: I think those bugs should not be there. But the priority should be on quality assertion.

According to this post, they finally decided to pull the plug. Netscape wil cease to exist.

In my early days our company made a lot of custom websites, and of course these sites needed to be crossbrowser compatible. Those were the days when IE 5 and NS 4 were the most recent browsers. Needles to say, the most difficult part of it was to make it work on Netscape. At that time, the implementations of JScript and DHTML were mostly inconsistent (to put it mildly).

The good stuff in this is that it reminded me of 'The Old Days', when building a site was just that.

This week I am attending the Microsoft TechEd Developers in Barcelona. Amongst the first impressions where those of dissappointment. First of all it is not that big as the ones I used to attend in Amsterdam. Secondly these seems to be no real new stuff. There is no new product hype. We've all been playing around with the beta's and CTP's of most products for a couple of months now, including Visual Studio 2008, the 3.5 .Net framework and Silverlight 1.1.

What strikes me most are the number of sessions on Silverlight that show basic stuff in 1.0. I attended a couple of sessions on how to communicate with the webserver from 1.0 Silverlight applications using Javascript and AJAX. Why should I want to do that? It will be phased out anyway as soon as 1.1 'hits the shelves'. Even more so, they already have native webservice support built in and working on SL1.1? I do not intend to spend my time working on code that'll be useless in a couple of weeks. Come on TechEd, bring the good stuff !!

This is of course all very reasonable, but the bottom line was reached when a speaker explained how to do c# a ternary operator on a lvl3 advanced course. This would've been fine if it didn't take him 10 minutes.

But the wheather is nice and the evenings are cool, so what am I complaining about?

In most of our latest Sharepoint projects we need a lot of customization. These customization generally mean that we develop webpparts and custom lists so that we use sharepoint as a sort of database. The list itself is rarely visible to end users and only to a extended sharepoint site that is used as a backoffice application.

The development of the webparts go as follows:

• For every webpart we develop we have:
  • The <webpart>.cs control
  • The <webpart>Control.asc User control
  • The <webpart>.wsp XML file that is used to package the webpart
  • A Test<webpart>.aspx file that is used to test the webpart standalone but is never deployed to the sharepoint application.
• The webparts do not rely on sharepoint, but bind to strongly type objects and collection.
• A Provider pattern is used for retrieving and updating of the data objects.

This provider model allows us to test and verify functionality of the webparts without the need to upload/provision these webparts inside a Sharepoint hosted website. We all know this takes a lot of time. The provider is configured in the web.config of the webpart solution file.

Here's a sample of one of these business objects:

The user control uses this class in codebehind like so:

You see this is basic ASP.Net stuff.

I found this a pretty amazing thing; it cuts down on the time used for each development roundtrip, and it allows us to 'outsource' the development of the web parts to ASP.Net specialist who do not (yet) have the required Sharepoint knowledge. The knife cuts on two sides!

This approach did have an enormous disadvantage however. The provider that implemented the mapping from the hard-coded business object to sharepoint needed a lot of custom code. It only paid of once the business object was used in multiple places. Another problem is that the code that provisioned the custom list was always a one-off piece of custom code. That code has NO relation to the business object.

Now, my experiences with entitypool helped me in writing a library that solves all this:

To create the list in Sharepoint, all that is needed is the following statement:

To retrieve all appointmentobjects we have a factory that converts a SPListItemCollection to AppointmentObject objects:

Easy? I think so.

In my previous post I rambled about a bug in the Web Application .targets file. I found an easier solution which does not involve patching any existing files.

The solution is to add the dependency projects to the msbuild target list like so:

msbuild /t:ResolveAssemblyReferences;_CopyWebApplication

     /p:OutDir=..\Drop\bin\ /p:WebProjectOutputDir=..\Drop\

Wow. Life CAN be easy.

Update: Even more so, it is better to use ResolveReferences, which also copies any project assembly references you have in your project.

 Update 2: You actually need to add a 'bin' directory to the OutDir folder. 

In my search for MSBuild tooling and software complexity analysis, I found the following tool: CCM 

It is a very simple and fast commandline tool. More info later on.