November 23, 2012

Tracking Desktop Applications with Google Analytics - what you should know...

If you are developing a software application and have been exposed to Google Analytics, you might find yourself asking a very common question: "Why not use Google Analytics to Track my Desktop Application?" At first glance this might seem like a very attractive option, however if you had to look into the details you will realize there are a number of fundamental problems that you are likely to encounter. The way users engage with a website/browser is very different from the way they engage with a desktop application. This means by simply using the same tracking metrics for websites and desktop applications alike, you cannot effectively measure both. Google Analytics does a good job at tracking website browsing sessions, however a number of deficiencies arise when trying to adapt browsing session metrics directly to track desktop application runtime sessions. Such deficiencies can drastically limit your ability to report on application usage and in some cases also affect your client user experience. In this article we discuss some points you need to keep in mind when looking into such a tracking solution.

When should you use Google Analytics?

If you simply want to track generic activity or software popularity of an experimental application or some non-commercial freeware tool, then relying on a free web-analytics solution might be a good cheap option since through Google Analytics you will be able to answer simple questions such as "How many runtime sessions of my software are invoked daily around the world?". However if you are looking for a more elaborate software analytics or Business Intelligence solution on which you can  rely for strategic business decisions or to increase conversions, then you have two alternatives:

  • Build your own tracking framework - assuming you have the expertise, development time/resources, and server infrastructure to handle big data.
  • Use a third party solution such as Revulytics Usage Intelligence

Below we have outlined a number of points that you should keep in mind when comparing Google Analytics with a specialized Desktop tracking solution such as Usage Intelligence.

How web analytics differ from desktop software analytics

Web analytics solutions (such as Google Analytics) were designed to track user engagement with your website. Essentially this means all metrics are based around browsing sessions where during a session, a visitor enters your website from one of your public pages, clicks around a few pages, possibly generating a few events along the way, until they convert or drop off your site. This gives you enough information to track the count of new vs returning visitors as well as their browsing path through those pages. However as far as Google Analytics is concerned, that is the end of the browsing session and if that user comes back, they will be tracked just like any other new/returning user, irrespective of who they are, how often they return and what happened during their previous visits. Therefore there is no concept of building a user profile to track how their usage patterns vary over their lifetime or throughout their evaluation lifecycle.

Trackerbird Conversion Funnel example

This also means that with Google Analytics  you cannot generate any reports that span over a number of days - such as conversion funnel analysis - since the longest conversion path must start and terminate in a single browsing session. This of course makes sense for website visitors, but it is not always applicable for desktop applications.

In order to track desktop software effectively, whatever solution you choose must keep track of each individual user's story and not a single runtime session. This is an essential building block that will change your reporting abilities from simple generic statistical reports to truly actionable business intelligence reports.

Comparing Google Analytics with Revulytics Usage Intelligence

Below we have listed some specific issues that you will need to cater for if you consider tracking your desktop application with Google Analytics. We also explain how Revulytics manages to solve these issues so the developer does not have to worry about them.

Developer Hacks: Mapping software events to page views

Keep in mind that what you are doing with Google Analytics is a hack, so before you start you need to become familiar with the traditional web-based tracking system using ga.js. The Google Analytics collector  is designed for websites (not applications), so it is up to the developer to determine what constitutes a change of page or clicking of a link inside the application to be able to pass that to Google Analytics. Therefore all the 'events' within your application will have to be manually mapped to page views and URLs in a way which can be reported on later.

- The Usage Intelligence API on the other hand was designed specifically for tracking user engagement with a Desktop Application, therefore it contains all the functions that you need to track runtime sessions starts, stops and other application events such as feature usage and exceptions. The API can be easily integrated in just 30 minutes without the need to worry about the inner workings.

Effect on User Experience

Always-on requirement: To track any event with Google Analytics, you will have to make a web request. This means you will be adding a requirement that your software must always have a live internet connection in order to be tracked.

Web traffic: When using Google Analytics, every tracked feature, button click, program start, etc. will generate an individual web request which means your software will be constantly generating internet traffic, which in extreme cases can slow down your application or raise eyebrows of administrators who might be monitoring internet traffic passing through a corporate gateway.

- Usage Intelligence uses an intelligent caching system where everything is logged in memory and/or to file. The logs are compressed at the most appropriate time before being sent to the server without interfering with user experience.  If a user goes offline, the logs are sent the next time a connection is available and no tracking data is lost. This intelligent caching mechanism will cut down hundreds of HTTP calls to a mere 2-3 calls per runtime session. The system also caters for users who are permanently offline so that log sizes are automatically kept under control.

Protocol Security and Reliability

With web analytics, a single malicious user can simply sniff the connection (using tools like Wireshark) to copy your callhome URL and use this in a script to skew all your reporting data, and unfortunately there is not much Google can do about this since you are simply tracking using anonymous web requests.

- The Usage Intelligence callhome protocol on the other hand implements various security checks including fingerprinting and RSA encryption to prevent replay attacks which make it much more difficult for a malicious users to simply use your callhome URL to skew data on the server.

Check out the Top 5 Google Analytics Reporting deficiencies on the next page...

Reporting Deficiencies

Tracking unique users vs. single sessions: Web analytics solutions are designed to track individual browsing SESSIONS (usually through cookies) and NOT users/installations. Therefore you have no idea if a user launches your application 10 times a day or whether it was 10 seperate users launching it once. This limits all reporting by session rather than by user.

Usage patterns over time: Using Google Analytics it is impossible to report on usage patterns for a particular user group over time because Google cannot identify a user from day1 to day2. Thus for example you cannot know if users are running your application 5 times a week or 5 times a month. You can only tell that a runtime session belonged to either a new or a returning user.

- Usage Intelligence on the other hand tracks a user story from the day they install your application until they 'die'. This means Usage Intelligence reporting can tell you how many times in a day/week/month users are engaging with your application as well as their average runtime duration.  In addition you can even breakdown this data by versions, builds, editions and compare how usage patterns vary between freeware, evaluation users or those having a purchased license.  Since Usage Intelligence keeps track of individual users, you can benefit from detailed conversion funnel and sales cycle analysis reports so you know the amount of time users spent in evaluation before buying/dumping your software. This information cannot be made available with web analytics solutions that rely on single session tracking.

Installation count and Churn rate: With Google Analytics you cannot get an accurate installation count showing you how many unique installations you had over a specific timeframe and how many of these have quit using your software. With Google Analytics you are limited to reporting how much activity (i.e. runtime sessions) you had on a single date. However if you try to look into this daily activity, you don't know which activity came from new users or old established users. Since Google do not really care about building user profiles, there is no easy way to get a distribution report such as how many installations you have running version X or edition Y or how many users you have running an evaluation license as at today. These metrics are simply not required for websites and are thus ignored by Google. Thus to get this information you will have to simulate a page view for every build/version of your software and call this on your application load.

- All these metrics are part of Usage Intelligence's core framework and therefore from a single dashboard you will get access to an accurate installation count, churn rate,  how many users are running each product versions, editions or build number as well as demographic data such as location and OS language. You will also get readily available reports of how users are taking up a new version or product update viewed over a timeline, as well as how your marketing campaigns are affecting the number of evaluation users and conversion rates. Usage Intelligence reports offer you the ability to drill down using real-time filtering criteria, so you can focus on a particular country,version, build number, edition, OS, license type, etc or any combination of those. Data is NOT pre-computed and presented as static numbers but you can investigate any Product Management/Development question that may come up. This of course means you can check specifically if any UI updates in a particular versions/build have an effect on the usage patterns for your product.

No Control Once Deployed

With Google Analytics you need to define beforehand (at development stage) what features and events are important to you and which of them you need to track. Once you deploy a build that tracks a set of events, these events will be permanently tracked and you have no way to switch tracking on/off for any of those events. But why would you ever need to turn off tracking?  Well let's say you want to do an A/B test on some UI wizard to check which version works best with your customers. For this reason you start tracking all the clicks related to this wizard and these will start showing up in all your Google Analytics reports as individual Page views. Once you finish your A/B test, all the event data generated by this wizard are now useless so you don't want your current reports to remain cluttered with such useless data. Unfortunately in Google Analytics you have no way to disable what pages/events are tracked at the clients side, so these events will keep showing up in all your future reports. Given that a number of Google Analytics reports focus on showing just the most visited pages/events, this can possibly hinder your ability to report on other (now more important) events/feature because your reports would be suffocated

- Usage Intelligence on the other hand offers a convenient management console in the Reporting UI which enables you to control exactly which features/events you want to track and which you want to ignore. Therefore you can switch on/off any tracked event or feature without involving any development. In addition to this, when your clients sync with the Revulytics server, the SDK automatically and seamlessly retrieves the new set of whitelisted events/features so that the client itself does not even bother tracking them on the client side. This avoids logging and transmitting useless data, thus keeping your clients working efficiently and the logging as lightweight as possible.

 

One Way Data Collection: Passive vs Actionable

With Google Analytics you are only capable of passively collecting data without having any ability to interact with your customers. In the end, you then need to find individual tools by which you can respond to what you have learnt through your tracking exercise.

- When using Usage Intelligence you do not simply get analytics and reporting but this data is made actionable through a unique Reachout™ Direct-to-Desktop messaging framework. This a powerful in-app marketing tool which can help you increase conversions and collect user feedback through Reachout Surveys. Therefore let's say you discover that most of your users are dropping out from evaluation after 10 days of install. Using ReachOut you can send out promotional offers or other incentives directly to the desktop of these users on the 7th and 8th day of evaluation so that you can experiment with increasing your conversion rate. Likewise you can send out surveys to a particular user-group (say those running a particular build) to collect feedback about your product or their user experience.

Conclusion

As we outlined in this post, there are several points you need to keep in mind before diving into a tracking solution for your desktop software. Of course if you have enough time on your hands to hack around Google Analytics, it may be a good enough and 'free' alternative to get a generic idea of how popular your software is.  However if you have a more mature or commercial software product from which you want to gather more serious user engagement analytics, then you might want to consider a proper Desktop Software Analytics solution.

Post written by Keith Fenech

Subscribe to Our Blog