Analytics Specification

Owned by Adrian Kingston
09 Apr 2018

Background

In order to better evaluate digital experiences we need to collect analytics from all applications running on the floor. The Digital Strategy states that to create the most audience value we must:

  • Use insights from analytics, trend and market analysis, benchmarking and user research to increase empathy with users, communities and partners, continuously improve products and inform business decisions and priorities.
  • Understand our customers, and use all the capabilities of digital technology to build relationships with them through relevant, connected content, products, platforms and services
  • Take a strong user-centred approach to service design that delivers seamless, integrated visitor journeys in any location (physical, digital, blended)

We must also collect observational data to get a sense of overall visitor behaviour as they move through the physical spaces, and how they interact with our digital experiences.

We want to understand visitor behaviour so that we can understand the role of digital within the visitor experience.

The metrics that we collect from each digital experiences may vary, and we will need to define what is useful on a per-experience and per-exhibition basis. Over time we will build up a library of core digital measures, and gain an understanding of what these mean in the broader context.

At a component level there will be consistent metrics, for example all video players will report the same actions and time-based measures, allowing like-for-like comparisons and tracking of longer terms trends across the floor.

At the experience level we will also learn about consumptions of content. This information should be used to inform changes in the digital content strategy, and perhaps even give us insights into the interaction between digital and print text and graphics, and physical experiences.

A critical component of this work will be gaining an understanding of the difference between statistical signals, versus statistical noise. A sound understanding of statistical theory will be vital as we design measures.

Gaining an Understanding of Visitor Behaviour

When we have a full-stack of observational and digital tools in place, we should be able to gain understanding in the following areas:

  • Conversion rate
  • How many people use each experience?
  • How much time is spent on each experience?
  • How deeply do people explore the content?
  • What parts of the experience are popular?
  • Differences between experiences in one exhibition.
  • Differences between experiences across the museum.
  • Differences between components (e.g. video players across the museum).
  • Design patterns that perform well and those that don’t (e.g. A/B testing)

Observational Events

  • Total people though Te Papa door (automated)
  • Exhibition visitor numbers
  • Physical Bounce (looked, but did not stop)
  • Session length
  • Depth of engagement (read, interacted, over-shoulder)
  • Size of group
  • Demographic information
  • Return visit (loop back to something)

Experience Events

  • Content bounce (e.g. stop video before end, scroll page before it could be read)
  • Drop-off vs completion
  • Average reading time on page (vs dwell-time on page)

NB: These may be more difficult to define accurately and track.

Component Events

These events are associated with DEDS components:

Page

  • Page view (including language used)
  • Time spent on page

Navigation

  • Clicked next/prev
  • Swiped next/prev
  • Other gestures
  • Clicked nav (e.g. timeline)

Image Gallery

  • Gallery - opened
  • Gallery - close + time spent in gallery
  • Gallery - image view
  • Gallery – zoom image

Gloss

An interesting point is that if the Gloss is not used, does it mean more people understand te reo Māori.

  • Gloss activated (noting word)
  • Gloss audio played (noting word)

Home

  • Home - clicked
  • Home - automatic return (eg, timed out)

Video

  • Video – Play
  • Video – Stop (Video pause, also?)
  • Video – ended
  • Video – turn on subtitle, English
  • Video – turn on subtitle, te reo Māori
  • Video – turn on subtitle, other

Language

  • Language change
  • Number of pages browsed per language.

Analysis

Overview

The chosen solution should balance the presentation of top-line data, with being able to explore long-term trends.

It should be easily accessed, and quickly and simply present the basic information.

All of the data specified in component events above should be tracked and displayed in a way that makes sense for the metric.

Export

We should be able to export raw data so that we can do more complicated analysis.

Examples of this are:

  • Correlate app data with observational data to draw inferences over time.
  • Track the number of people using interactives (and the time spent) as a percentage of total visitors.
  • Long term trending of data over multiple exhibitions, so we can see if interactive use is increasing/decreasing over time, or if there are outliers.
  • Compare components across multiple exhibits, such as all on-demand video players
  • Determine seasonal trends (holidays) and normalise these

Technical Implementation

We will follow an event driven model. Each component will emit events in an agnostic manner, and these events will collected at the top level of the app and sent on to our analytics system. This ensures that the operation of components is not tied to any specific provider.

At this stage the events will be forwarded to Google Analytics.

Implementation

Event could be send with the native dispatchEvent call, with jQuery’s trigger(), or whatever is the most convenient library call.

Event hits should contain the following data :

{
  hitType: 'string',
  eventCategory: 'string',
  eventAction: 'string',
  eventLabel: 'string',
  eventValue: 'integer',
}

Event fields

The following table summarizes the event fields:

Field Name

Value Type

Required

Description

hitType

Text

yes

The type of event – click, swipe, etc

eventCategory

Text

yes

Typically the object that was interacted with (e.g. 'Video')

eventAction

Text

yes

The type of interaction (e.g. 'play')

eventLabel

Text

no

Useful for categorizing events (e.g. 'Fall Campaign')

eventValue

integer

no

A numeric value associated with the event (e.g. 42)


Example code

Functions to emit event

function emitAnaltyicsPageView(url, title){

  document.dispatchEvent(

    new CustomEvent("deds-analytics-pageview", {

      detail : {

        url: url,

        title: title

      }

    })

  );

}


function emitAnaltyicsEvent(eventCategory, eventAction, eventLabel, eventValue){

  document.dispatchEvent(

    new CustomEvent("deds-analytics-event", {

      detail : {

        eventCategory: eventCategory,

        eventAction: eventAction,

        eventLabel: eventLabel,

        eventValue: eventValue

      }

    })

  );

}


Code for head of document


(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){

(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),

m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)

})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');


// We cannot use cookies running on the file system so use localStorage instead

// ref: https://developers.google.com/analytics/devguides/collection/analyticsjs/cookies-user-id

var GA_LOCAL_STORAGE_KEY = 'ga:clientId';


if (window.localStorage) {

  ga('create', 'UA-XXXXXXX-XX', {

    'storage': 'none',

    'clientId': localStorage.getItem(GA_LOCAL_STORAGE_KEY)

  });

  ga(function(tracker) {

    localStorage.setItem(GA_LOCAL_STORAGE_KEY, tracker.get('clientId'));

  });

}

else {

  ga('create', 'UA-XXXXXXX-XX', 'auto');

}


// Disable file protocol checking as GA normally requires http(s).

// ref: https://developers.google.com/analytics/devguides/collection/analyticsjs/tasks

ga('set', 'checkProtocolTask', null); // Disable file protocol checking.


// Our two event listeners - these match the two types of events captures by GA

document.addEventListener("click", function(e){

  emitAnaltyicsEvent('/testing', 'My Title');

});


document.addEventListener("deds-analytics-pageview", function(e){

  console.log('HI from page-analytics page-view')

  ga('send', {

    hitType: 'pageview',

    page: e.detail.url,

    title: e.detail.title

  });

});


document.addEventListener("deds-analytics-event", function(e){

  console.log('HI from page-analytics event')

  ga('send', {

    hitType: 'event',

    eventCategory: e.detail.eventCategory,

    eventAction: e.detail.eventAction,

    eventLabel: e.detail.eventLabel,

    eventLabel: e.detail.eventValue

  });

});