Moved the server yet again, it's been limping and struggling to survive. Expect activity to return.

Technical details, the home server has been upgraded to WHS 2011, now running on another Intel Atom with 4Gb, with Virtual Server 2005 R2 on top.  

Today I finished the build of my new WHS. I used to run a virtual WHS, and I have been extremely pleased with the functionality it provides. However, the virtual drives were getting cramped and I needed to make them redundant a bit, as it is not really wise to have 2 virtual drives on a single physical disk.

After some bad hardware shopping I started again from scratch and this is what I came up with in the end:

• ASRock A330ION mITX Atom with 2Gb DDR3 memory
• CHENBRO ES34096 mITX case with 4 hotswap HDD bays
• Windows Home Server
• 2x 1Tb Samsung Spinpoint 7200RPM
• Licensed OEM Microsoft Windows Home Server

Everything was bought from informatique.

Here’s some pictures:

And now begins the process of moving all the data and setting up the proper shares and permissions!!

public override bool IsHetAlVrijdag()
{
    return true;
}

When you want to run a long-running process in ASP.Net, you are bound to get into time-out problems.

Not when running the process in a background thread. Spawning a thread is easy using the System.Threading.Thread namespace

ThreadStart ts = new ThreadStart(this.PerformWork);
_worker = new Thread(ts);                    
_worker.Start();

…

private void PerformWork()
{
    …do your stuff here.
}

When the website is running under impersonation, you’ll have a problem inside the PerformWork method, because the new thread inherits from the parent thread, which is actually running under the ASPNET account!

To solve this, you need to capture the Windows Identity of the impersonated user, and then change the security context inside the thread like so:

ThreadStart ts = new ThreadStart(this.PerformWork);
_worker = new Thread(ts);                    
_appID = System.Security.Principal.WindowsIdentity.GetCurrent();
_worker.Start();

…

private void PerformWork()
{
     WindowsImpersonationContext wi = _appID.Impersonate();
    …do your stuff here.
}

Problem solved. The background thread now assumes the identity of the person logged in.

1. Avoid alliteration. Always.
2. Never use a long word when a diminutive one will do.
4. Employ the vernacular.
5. Eschew ampersands & abbreviations, etc.
6. Parenthetical remarks (however relevant) are unnecessary.
7. Remember to never split an infinitive.
8. Contractions aren't necessary.
9. Foreign words and phrases are not apropos.
10. One should never generalize.
11. Eliminate quotations. As Ralph Waldo Emerson said, "I hate quotations. Tell me what you know."
12. Comparisons are as bad as cliches.
13. Don't be redundant; don't use more words than necessary; it's highly superfluous.
14. Be more or less specific.
15. Understatement is always best.
16. One-word sentences? Eliminate.
17. Analogies in writing are like feathers on a snake.
18. The passive voice is to be avoided.
19. Go around the barn at high noon to avoid colloquialisms.
20. Even if a mixed metaphor sings, it should be derailed.
21. Who needs rhetorical questions?
22. Exaggeration is a billion times worse than understatement.
23. Don't never use a double negation.
24. capitalize every sentence and remember always end it with point
25. Do not put statements in the negative form.
26. Verbs have to agree with their subjects.
27. Proofread carefully to see if you words out.
28. If you reread your work, you can find on rereading a great deal
of repetition can be avoided by rereading and editing.
29. A writer must not shift your point of view.
30. And don't start a sentence with a conjunction. (Remember, too, a preposition is a terrible word to end a sentence with.)
31. Don't overuse exclamation marks!!
32. Place pronouns as close as possible, especially in long sentences,
as of 10 or more words, to their antecedents.
33. Writing carefully, dangling participles must be avoided.
34. If any word is improper at the end of a sentence, a linking verb is.
35. Take the bull by the hand and avoid mixing metaphors.
36. Avoid trendy locutions that sound flaky.
37. Everyone should be careful to use a singular pronoun with singular nouns in their writing.
38. Always pick on the correct idiom.
39. The adverb always follows the verb.
40. Last but not least, avoid cliches like the plague; They're old hat; seek viable alternatives.

Other Versions:

1.Don't abbrev.
2.Check to see if you any words out.
3.Be carefully to use adjectives and adverbs correct.
4.About sentence fragments.
5.When dangling, don't use participles.
6.Don't use no double negatives.
7.Each pronoun agrees with their antecedent.
8.Just between you and I, case is important.
9.Join clauses good, like a conjunction should.
10.Don't use commas, that aren't necessary.
11.Its important to use apostrophe's right.
12.It's better not to unnecessarily split an infinitive.
13.Never leave a transitive verb just lay there without an object.
14.Only Proper Nouns should be capitalized. also a sentence should begin with a capital and end with a period
15.Use hyphens in compound-words, not just in any two-word phrase.
16.In letters compositions reports and things like that we use commas
to keep a string of items apart.
17.Watch out for irregular verbs which have creeped into our language.
18.Verbs has to agree with their subjects.
19.Avoid unnecessary redundancy.
20.A writer mustn't shift your point of view.
21.Don't write a run-on sentence you've got to punctuate it.
22.A preposition isn't a good thing to end a sentence with.
23.Avoid cliches like the plague.
24.It is wrong to ever split an infinitive.
25.Profanity sucks.

(source)

I’d like to share this piece of code with you all.

Usually, when you use the System.Web.Cache classes, you do something like:

object x = Cache[blabla];
if(x == null) //not in cache
{
    x = …..;
   
    // add to cache
    Cache.Add(x, …);
}
// do something with x here.

Now this is not very thread-safe, because when 2 threads arrive at the ‘if’ at the same time, the object is constructed twice. The second one to complete will just overwrite the cached result of the first one (get it?).

So, in order to avoid this you can add a ‘lock’ statement around this to avoid filling the cache twice.

lock(typeof(ClassName))
{
   …

}

This will essentially avoid that 2 threads will enter the same code at the same time.

The problem with this approach is that there is a significant amount of code for every time that you want to read or write to the cache. The code is very error-prone but it is basically the same pattern over and over.

With generics and delegates you can develop a function that behaves like so:

object x = Cached.Get<object>(“cacheKey”, TimeSpan.FromSeconds(5),
                            () => new object()
                       );

The lambda expression ()=> new object is the fragment of code that is executed to populate the cache. The “cacheKey” is some sort of string that uniquely defines what information you want to retrieve. The key should in theory contain all the parameters for your population method.

Now there is a huge potential problem inside the Get() method. The lock statement (typeof(ClassName)) inside is essentially the same one over and over, so *all* calls through this method will be blocked and serialized, thus reducing concurrency to zero. If you have long-running or expensive population methods (you will, because that is why you are caching) you’ll be slowing the system to a halt.In effect, you’d want a separate lock for every 'cache key’ literal.

To overcome this problem, I developed an implementation that generates a new lock for every key. However, when reading and updating the HttpUtility Cache you should be very careful for race conditions.

CacheUtility.cs (7,11 kb)

This class provides strongly-typed access to the HttpCache. It also provides helper methods to flush a single or all elements that were cached using this method, and also a way to determine groups of cached elements. This way you can flush only a subsection of your cache.

I just saw this article with computer quotes, and I found it to be quite hilarious.

“The Internet?  We are not interested in it.”
– Bill Gates, 1993

“Pessimists, we’re told, look at a glass containing 50% air and 50% water and see it as half empty.  Optimists, in contrast, see it as half full.  Engineers, of course, understand the glass is twice as big as it needs to be.”
– Bob Lewis

“It was a joke, okay?  If we thought it would actually be used, we wouldn’t have written it!”
– Mark Andreesen, speaking of the HTML tag BLINK

“Perl: The only language that looks the same before and after RSA encryption.”
– Keith Bostic

“Yes, we have a dress code. You have to dress.”
– Scott McNealy, co-founder of Sun Microsystems

And finally, since I’m working with UXP professionals:

“We have to stop optimizing for programmers and start optimizing for users.”
– Jeff Atwood

For more great quotes, check here.

Welcome to part 3 of my little series of creating an Asp.net component that allows to upload files of several megabytes without connection timeouts.

In part 1, I explained how the Silverlight component was built and in part 2 the backend service was discussed. In this third and last part I will explain the reusable ASP.Net server control that combines the features into a plug and play component.

The component will solve the following problems:

• How do I host the SilverLight component?
• How do I know when the upload has completed?
• How do I retrieve the contents of the file?

Hosting the SilverLight component

The SilverLightUploader control overrides the CreateChildControls and embeds a System.Web.UI.SilverlightControls.Silverlight component to host the uploader. The control provides for a couple of properties and some are passed on to the silverlight component by providing some initialization parameters. These include:

• The url of the webservice to be called
• The location of the SilverLightUpload.xap package.
• The filter that limits the extension of the files that can be uploaded.

I used a helper function to resolve the passed Url’s to absolute url’s because of some limitations I had in my library, but a simple ResolveUrl should be sufficient in most cases.

Also note the OnPluginLoaded hook. This hook is used to bind the event (that is raised when a file is uploaded) to JavaScript.

When a file upload is completed, the Silverlight component raises an event. This event is then catched by a JavaScript block which saves the token of the upload in a hidden field. The Silverlight component just displayed ‘Upload Completed’ and sits and waits until the page is submitted. (the script block is not shown here).

On postback of the page the control checks if the hidden field contains a valid token, and if this is the case, the asp.net component raises an event that can be caught by the page which contains the control.

The control implements IPostBackDataHandler, which means that on any postback the LoadPostData method of the control will be called.

The GUID of the upload is persisted in a ViewState-backed property. So even if a validation error occurs on postback, the upload is still safe. (Unfortunately, the Silverlight component does not reflect this yet). LoadPostData returns ‘true’ and the asp.net framework calls RaisePostDataChangedEvent after all the initialisation logic of the page have been completed:

The page that is subscribed to the FileUploadEvent can then call the UploadManager.GetUpload() with the ID that is provided in the event to retrieve the file.

Some tricks left

Instead of putting the SilverlightUpload.xap in /ClientBin, I have embedded the package in the web control by adding it as an embedded resource. I added this file to the project using ‘Add Existing Item’ and ‘Add as Link’ to the SilverlightUpload.xap from the web application. This last step saved me from manually copying the xap every time I created a new build.

In AssemblyInfo.cs I have added a declaration so that webresource.axd can be used to retrieve the xap:

Also, I moved the implemementation of the WCF UploadService.svc into the control DLL.

Now, to use this control:

• Add the WebResource.axd handler declaration to the web.config of your web application,
• Append the service EndPoint to the web.config,
• Add the SilverlightUpload.dll to the bin directory (or add a reference)
• Create a file that is a stub for the WCF service (this is the almost-empty UploadService.svc file),
• Create a page that uses the control,
• Add an eventhandler to the FileUploaded event

For your uploading pleasure, I’ve added the project with all code to this post.

That should be it!

SilverlightUpload.zip (73.54 kb)

This is the second part of a multi-part article about implementing large file upload for ASP.Net In my previous post I explained the Silverlight bit and this part will be about the WCF backend.

Curious enough, the WCF backend is actually extremely simple. The are only 3 methods that are exposed as a webservice. There 3 methods cover the 3 phases of the file upload process I explained in the previous post. But before I show the service I need to explain some thing else first.

In summary:

• The client request an upload token.
• The client sends the binary file in small blocks using the token.
• After the file is complete, the client confirms the upload.

We took care of those steps with the Silverlight client. On the server side, we need to respond to those steps.

To maintain a record of all the uploads that are in progress, we need to maintain some sort of ‘memory’.  When an upload starts, a new UploadSession object is created. The UploadSession has a couple of properties:

1. The Filename of the file that is being submitted,
2. The ID or token of the session,
3. The State that the upload is currently in (Created, Streaming, Completed or Failed), and finally:
4. The binary Data of the transmitted file.

UploadManager class

To access a session, a singleton class is implemented.It has 3 methods: CreateSession and GetSession, and finally GetUpload():

CreateSession will create a new session, GetSession retrieves a session. Both are declared internal because they are used by the WCF service (we will get to that in a minute). Third, the public GetUpload() is similar to GetSession but is used to retrieve a completed transfer.

UploadSessions

The UploadSessions class is a helper collection class that is used to store the current uploads. In a first version I was keeping all the uploads in a collection that had Timer events to purge old uploads, but later I decided to use the HttpRuntime.Cache. Here’s the collection class:

In essence, it provides strongly-typed access to the cache using the indexer operator. These getter and setter forwards all calls to the HttpRuntime Cache. If you didn't know, this cache is exactly the same class as the HttpContext.Current.Cache, only shorter.

I set the sliding expiration to 5 minutes. This means that the session will be removed is no block is received within 5 minutes, or when the file is not ‘picked up’ by the consuming client after 5 minutes of the upload. This should be sufficient in most cases.

Also, I use the priority to CacheItemPriority.NotRemovable so the server does not expire the session within the timeout period is it low on resources.

(I just noticed that the Remove method is empty…)

Calling the UploadManager from WCF

The WCF service itself is extremely simple. I just forwards all methods to the UploadManager:

Last, but not least

The last step for the service side is UploadSession class itself:

Again, no rocket science. The UploadSession class is just a wrapper class for a MemoryStream. I intend to expand this class so it flushes the buffer to a temp-file if the file becomes too large, but for now it works perfectly.

The reason I am using a in-memory buffer is a) memory is cheap and fast and b) I’m using the uploaded data in combination with Linq. And Linq stored binary data as a Binary() class, which is just a wrapper class for a byte[]. Therefore it has no use in streaming the file to disk first, and then completely loading it back into memory. The disk buffer approach is only handy when lots of file are uploaded at the same time. If a lot of files are uploaded simultaneously, the total memory usage would have been lower during the time of the uploads, at the cost of requiring temporary files and a lot of disk I/O.

Hosting the Service

Now, to host this WCF service, a basic endpoint must be configured in the web.config of the hosting application:

This configuration was generated automatically when I added the service in my project, so it is completely standard. No magic here. No streaming, no large buffers, nothing fancy.

The Silverlight client needs the Url of the WCF service, but I think that was already covered in the previous post.

Wrapping it all up

This is it for the WCF side of it all. As I mentioned before, we are not finished yet. We still need a way to host the Silverlight application, and tell Asp.net that we have received a file. After that we need to pickup the file from the UploadManager. I’ll be showing you a custom server control that does all three of these steps in a simple, reusable Asp.Net control. But that topic will have to wait until the next episode.

Happy uploading!