Mads Kristensen

My personal blog about everything tech

About 10 years ago, it was actually possible to look people up by their e-mail address online. You could also find a persons e-mail by searching for his or her name. Back then there where a lot of e-mail directories that acted like the yellow pages but for e-mail addresses. Very handy, but when spam became a problem, no one was willing to publicise their e-mail address and the e-mail search quickly died out.

Years passed and nobody thinks seriously about searching for people by their e-mail address anymore. It was tossed out of our toolbox – abandoned and forgotten.

Then a few years ago, something wonderful started to happen with the web. Community sites, forums, blog platforms etc. stated publishing FOAF and SIOC documents. Both documents contain e-mail addresses of people but not in the traditional sense. They publish SHA1 hashed e-mail addresses.

You can hash an e-mail using the SHA1 algorithm but you can never reverse it. That means the hashed e-mail addresses are secured from spam bots, but they are also left public for all of us to search for. All you need to do is to hash an e-mail address and do a Google search with the hashed value. Try searching for my hashed e-mail address on Google or go hash your own e-mail.

Here is a quick way of using SHA1 algorithm to hash any string value in C#.

public static string CalculateSHA1(string value)

{

  value = value.ToLowerInvariant().Trim();

  return FormsAuthentication.HashPasswordForStoringInConfigFile(value, "sha1").ToLowerInvariant();

}

The limitations of the SHA1 e-mail search is that you can only find people that have an online profile or blog, participate in online discussions or comments on blogs. The number of searchable people will rise as more and more sites start supporting FOAF and SIOC.

I’ve been thinking about how to solve a very simple problem on a website: visitor behaviour tracking. In a sense it is what Google Analytics does, but there are problems with conventional JavaScript based trackers.

They are good at tracking page views, but very bad at tracking actions or behaviour around a website. Some products like Headlight are actually pretty good at tracking actions such as button clicks etc, but at the level I’m interested in tracking, I would have to add JavaScript all over my page. I don’t want to do that.

Also, there are some very good server-side logging products like log4net out there. The problem is that they are not really meant for tracking website behaviour with URLs, user agents and other important metadata.

What I want is a combination of the traditional JavaScript- and server-side methods. So, I’ve played around with a custom HttpModule that logs all page views and custom actions. You add the custom actions yourself by calling VistorLog.AddAction("message", "type"). That way you have page views and actions in a chronologically correct order.

A neat thing is that all page views and actions are kept in session and only when the session expires does it write to the database. That way it can do a batch insert which is much faster than hitting the database constantly. Another neat thing is that the HttpModule is only 100 lines of code.

The code

Basically, three things are going on. The session starts and we add a Visit object to it. 

void session_Start(object sender, EventArgs e)

{

  HttpContext context = HttpContext.Current;

  Visit visit = new Visit();

  visit.UserAgent = context.Request.UserAgent;

  visit.IpAddress = context.Request.UserHostAddress;

  context.Session.Add("visit", visit);

}

Then every page view is registered after an .aspx page is served.

void context_PostRequestHandlerExecute(object sender, EventArgs e)

{

  HttpContext context = ((HttpApplication)sender).Context;

 

  if (context.CurrentHandler is Page)

  {

    Visit visit = context.Session["visit"] as Visit;

    if (visit != null)

    {

      Action action = new Action();

      action.Url = context.Request.Url;

      action.Type = "pageview";

      visit.Action.Add(action);

    }

  }

}

Then the session ends and we need to store the visitor log.

void session_End(object sender, EventArgs e)

{

  HttpContext context = HttpContext.Current;

  Visit visit = context.Session["visit"] as Visit;

  if (visit != null)

  {

    // Log the Visit object to a database

  }

}

Implementation

When you have registered the HttpModule in the web.config, then it starts collection page views in the session. To store them in a database you must add your own code to the session_End method of the module. Now you are also able to store actions just by calling a static method on the VisitorLog module:

VisitorLog.AddAction("Profile picture deleted", "deletion");

Keep in mind that this code is just me playing around in my sandbox. It has never been in a production environment.

Download

VisitorLog.zip (1,11 kb)