A few days ago, Google released their Closure Compiler project for optimizing JavaScript. Here’s what they write about the Closure Compiler:

The Closure Compiler is a tool for making JavaScript download and run faster. It is a true compiler for JavaScript. Instead of compiling from a source language to machine code, it compiles from JavaScript to better JavaScript. It parses your JavaScript, analyzes it, removes dead code and rewrites and minimizes what's left.

The interesting part of the Closure Compiler is that it not only removes whitespace, it also rewrites your JavaScript code to make it smaller and optimizes the code for better performance. My tests show that it can reduce JavaScript files by about 60% - and that’s before HTTP compression! Considering how much JavaScript a modern website uses, this is no less than amazing and highly useful.

The Closure Compiler comes in two flavors – a Java based command line tool and a RESTful API. I’ve been playing around with the API and it works great and very fast.

The code

The C# class I’ve written takes a JavaScript file and passes it through the API and then returns the compressed JavaScript as a string. The class contains one public and one private method and is only 47 lines of code including 16 lines of comments.

public string Compress(string file)

{

  string source = File.ReadAllText(file);

  XmlDocument xml = CallApi(source);

  return xml.SelectSingleNode("//compiledCode").InnerText;

}

 

private static XmlDocument CallApi(string source)

{

  using (WebClient client = new WebClient())

  {

    client.Headers.Add("content-type", "application/x-www-form-urlencoded");

    string data = string.Format(PostData, HttpUtility.UrlEncode(source));

    string result = client.UploadString(ApiEndpoint, data);

 

    XmlDocument doc = new XmlDocument();

    doc.LoadXml(result);

    return doc;

  }

}

How to use it

You can use the class to do various cool things. You can write a MSBuild or NAnt script that automatically compresses your JavaScript files as part of a continuous integration process or, as I prefer, write a HTTP handler to do the same but at runtime. Remember to output cache the compressed result. Here's an example of using the class from ASP.NET:

GoogleClosure gc = new GoogleClosure();

string script = gc.Compress(Server.MapPath("~/script.js"));

Remember that the class doesn't do any exception handling, so you might want to stick that in yourself.

Download

GoogleClosure.zip (905,00 bytes)

Last week a colleague and I gave a talk about scalable architecture and where my colleague talked about databases and application layer scaling, I talked about scaling websites. More precisely, we talked about the upcoming ZYB/Vodafone project

Since there’s still a lot of secrecy about the project, we managed to keep the concepts general. General or not, I’d like to share some thoughts on a different way of scaling websites.

Load balancing

Larger websites are often hosted on multiple web servers under a load balancer that distributes the requests evenly among the servers. This is an old technique for scaling out websites and has been widely used as the de facto scaling mechanism for years.  It’s good, it works and it’s cheap. It’s cheap because web servers often don’t have to be the biggest machines in contrast to e.g. database servers.

So, a load balanced web server setup provides good and cheap scaling possibilities.

Reversed load balancing

Any website, load balanced or not, can also use the vast untapped resources in the visitor’s browsers. Think about it. Quad core CPU’s and 4GB memory is almost standard today – even on laptops. Why not utilize the machine power behind the browsers to do some of the scaling for us?

Traditionally, this is done using browser plug-ins like applets, Flash and Silverlight, but many more sites use JavaScript. Modern browsers process JavaScript very fast and efficient which makes it possible to use JavaScript for scaling purposes.

To utilize the browsers memory we can cache data in JavaScript so we can eliminate chatty communication with the web server. An example would be to load all the data needed behind the scenes after the page is loaded and store it in JavaScript variables.  To utilize the CPU we can make calculations, dynamic rendering and other logic in JavaScript as well.

By pushing some of the load to the browser we are able to scale even more than just using regular load balancing.

It’s not for everyone

There are some problems with this approach that makes it a bad choice for some websites. If enough of the visitors are using old browsers like IE6 then they will get a worse experience because JavaScript runs too slow. There’s also the case where a website just doesn’t have any data to cache like a personal website.

For other types of websites it makes perfect sense. If your visitors have modern browsers and your website is heavily data driven, then it’s a possible candidate. The tests we have done at ZYB shows huge benefits by loading data behind the scenes - both the performance and scalability improves significantly. The load on the web servers dropped drastically with this technique. I hope to be able to show you some real numbers later.