My PhD is about defining a framework to evaluate the impact of social media and other participatory digital activities. Although the tools and methods are very important, one of the main conclusions stemming from putting the framework into practice is that in order for this data to make an impact on the implementation of changes, the evaluation practice needs to be embedded in the organisation’s internal processes. Having a performance measurement system implies an investment in resources, reporting systems and changes in processes where insights and findings inform the decision-making process.
Although in many cases evaluation is perceived as something that happens once the project has been launched, there is a lot of work that should be done beforehand to make sure that the right data is collected and that an organisational structure is put in place in order to act on this data. In the planning phase of a digital initiative there should be an evaluation plan to gather the objectives, establish the target audiences, select the measures that will be collected during the evaluation and set up the targets for each of the measures. The evaluation plan should also include the roles and responsibilities of the evaluation team, the data collection and analysis time frame, frequency of reporting, stakeholders that will receive the evaluation findings, communication channels, and how the findings will be used.
During the production phase it is crucial to set up the tools that will be needed to gather the desired data. For instance, in order to collect tweets on a particular exhibition, conference or performance, there are very useful tools such as Rowfeeder or a IFTTT recipe, but given the platform’s data access limitations is important to set up the queries in advance. Website production is another case where advanced analytics implementations is often required. It is very common to add Google Analytics code just the day before the launch (or even the day of the launch!) but as this process entails a lot more than simply coping and pasting the code, it should not be left to the last minute. There will be a set of specific configurations based on the particularities of the site at hand, such as creating a funnel report in your ticket purchase process, track clicks on image downloads or count shares of a video on social media… A good example is found on how the V&A is using event tracking to understand users behavior. Apps are another clear case where event tracking is needed to evaluate some user actions or screen views.
Once the project goes live the data is collected and that is when the fun starts. Data is analysed and findings are circulated within the organisation to implement changes based on this information. Reporting can be done in different ways and the detail and level of data will depend on who is the recipient of the information. The internal feedback loop where the findings impact the digital production is key for reaction and success in this rapidly changing digital environment.
