Executive Summary
Without agreed-upon best practices to assess the effectiveness of social media for engendering visitor engagement, museums cannot determine whether their increasing social media presence is actually beneficial or simply a waste of resources. Measuring the quality of visitor participation in social media is difficult. For example, a user has the ability to ‘Like’ a Facebook page, but there is no way to determine if that person is actually interacting with the page on a regular basis. This discrepancy generates misleading numbers on web pages and inaccurate conclusions about how people are engaging with different technologies. The goal of this project was therefore to provide Te Papa with a better understanding of meaningful engagement and recommendations on how to enhance it in the context of social media.
Literature Review
Methodology
Findings & Analysis
Recommendations and Conclusions
Literature Review
Meaningful engagement has been defined in many different ways. Naomi Haywood and Paul Cairns developed three key factors that allow for engagement: participation, narration, and the co- presence of others (2006). Haywood and Cairns argue that all three of their factors must be present in order to develop an environment that encourages meaningful engagement (2006). Morris Hargreaves McIntyre (a company dedicated to organizational development) created a hierarchy of engagement: social, intellectual, emotional, and spiritual (2005). The Morris Hargreaves McIntyre model of engagement describes each level as building from the previous one (2005). Renee Warren continually emphasizes four key terms in her definition: Relationships, Community, Relevance, and Active Involvement. Warren also claims that meaningful engagement cannot occur unless all of the parts are present (2011).
There is a common theme throughout each of these definitions: meaningful engagement can be defined with a group of factors, each being necessary. Whatever form the definition takes, it is clear that meaningful engagement in any context – whether physical exhibits or social media – can be best defined by a set of factors that work together to achieve meaningful engagement.
In the social media world, being able to measure user engagement would allow Te Papa to cater its posts to elicit an ideal response among users. Several tools currently exist to measure the overall success of such social media. These tools can be very useful for monitoring social media and websites alike, but they all have several inherent drawbacks. All of these tools function on the principle of counting ‘Likes’, ‘Comments’, and ‘Shares’, and comparing these to the number of followers. In addition, they all do it in different ways. Therefore there is no continuity across measurement tools. Since the tools are based entirely “on the numbers,” they are blind to factors such as contests that generate a large amount of activity but no real engagement.
Methodology
The goal of this project was to provide Te Papa with a better understanding of meaningful engagement and a set of recommendations on how to enhance it in the context of social media. Our first objective was to understand the museum’s social media presence. The second objective was to define meaningful engagement and apply it to social media. The third objective was to identify characteristics of social media content that enhance meaningful engagement.
We conducted in-depth interviews with five key personnel to accomplish the first objective. These discussions gave us insight into the museum’s social media strategy, its intended audience, and what it was hoping to achieve when posting content. We interviewed staff members who post content to Te Papa’s social media platforms to learn about their individual motivations for posting content.
We drew on several resources to guide our process of defining meaningful engagement, including our literature review and staff interviews. After performing a thorough content analysis on those resources, we held in-depth discussions among ourselves and with staff to set parameters for the term ‘meaningful engagement.’ We synthesized the results of our analyses to form a “pillars of engagement” model that could be used to gauge meaningful engagement.
We applied our model to social media in order to identify trends in meaningful engagement. We narrowed our scope to focus only on Facebook. We compiled a list of characteristics that could be found in a given Facebook post: picture, video, question, link, contest, and casual voice. These became the variables in our survey. We selected previously posted content from the Te Papa Facebook page, one for each of the six characteristics. We duplicated the posts on a mock Facebook page, removing the characteristic we were testing, producing twelve posts. In order to find the changes in engagement, we created an online survey that was posted on Te Papa’s website, Facebook page, and blog. In our survey, we developed a set of questions asking participants to rate the post on each of the pillars of engagement, as well as how they would respond to each post (‘Like’, ‘Share’, ‘Comment’). For each characteristic tested, we were able to compare how participants rated the control post with how they rated the variable post.
Findings and Analysis
Our interviews can be summarized into a few key findings. First, the motivations behind posting content differ depending on who is posting it. When the Collections department posts content, they want to generate public interest and encourage viewers to learn something. When the Marketing department posts content, they want to advertise events to encourage people to come into the museum. Second, we found that opinions on what defines a meaningful response differ from one person to the next. Collections department said that a meaningful response occurs when people talk about the posts, and then create something for themselves. And also defined a meaningful response as a post that is taken offline and into the public. Marketing said that a meaningful response was when there is not only interest in the post, but also in the museum. Finally, we found that there are several different ways the museum measures online success. Information Technology uses Google Analytics for tracking the website, while Collections uses Klout for Twitter, and Marketing uses Facebook Insights for the Te Papa Facebook page.
Through synthesis of our research and analysis of our staff interviews, we were able to produce a list of factors of meaningful engagement. We grouped common factors, as shown below.
This produced three pillars of meaningful engagement: Learning, Personal Interest, and Community. These terms formed our definition. To bring our definition of meaningful engagement to life, we developed a visual presentation of the three pillars (as shown below), known as the HARP Model of Engagement.
Engagement can be displayed on the HARP Model based on how much each of the three pillars is fulfilled. In our data collection approach each pillar was rated on a scale from one to six and then mapped on the HARP Model. The greater the area formed by the resulting triangle, the greater the meaningful engagement. An example of this can be found below.
The left model represents an example of low meaningful engagement with a low level of Community and moderate levels of Personal Interest and Learning. The right model shows how the triangle would change when the pillars of meaningful engagement increased. This change is visually shown by a greater filled-in area of the triangle.
The ratings for Learning, Personal Interest, and Community from our survey were mapped on the HARP Model of Engagement, allowing for a visual representation of the engagement achieved with each variable. The largest increase in the pillars of engagement was seen with the addition of the video. The mapped model can be found below.
A large increase was seen in Personal Interest. This was supported in feedback provided by the participants. The addition of a video also translated to an increase in Community, which is shown by the fact that three times as many participants indicated that they would ‘Share’ the post when the video was added.
Meanwhile, the smallest increase across all pillars was found in the addition of a picture, illustrated below.
Of the three pillars, Personal Interest showed the most noticeable increase. The addition of a picture adds a visual aspect to catch the reader’s eye.
There was one characteristic that decreased engagement: casual voice. The addition of this characteristic showed lower ratings in all three pillars, as illustrated below.
The decrease in Learning can be simply explained. An authoritative tone instills the reader with confidence that the writer is knowledgeable. Several respondents left feedback indicating that they expected Te Papa, as a museum, to present information in its posts with a professional voice. A casual tone undermines the authority of the publisher, which may decrease the amount of learning gained from a post.
Every post had at least one comment stating that it needed more information. These respondents demonstrated that there was an expectation that they would learn something from the posts. There were also several comments from participants who were not from the Wellington region. They pointed out that many posts were very specific to particular exhibits or events at Te Papa. These posts were therefore irrelevant to non-Wellington residents who would be unable to visit the museum.
Recommendations and Conclusions
Based on our analysis, we have developed a set of recommendations for how Te Papa can better engender meaningful engagement with its social media audience. Our survey results show that a casual voice decreases all of the pillars of engagement, representing a decrease in meaningful engagement. We recommend that Te Papa maintain a professional style of writing. The survey data also shows that the addition of a video results in the largest increase in all of the pillars of engagement. For this reason, we recommend that Te Papa use videos in its social media postings wherever plausible.
We also recommend that Te Papa include more information to enhance the Learning pillar of engagement. Additional background information could also be used to help explain the meaning of a social media post as many participants noted that they were confused as to what the post was referring to. However, we also recommend exploring the balance between presenting too little information and overwhelming the viewer with too much text or information.
While our research gives us an idea of how each characteristic affects the three pillars of engagement, the design of our survey prevents us from directly comparing one characteristic against another. We are also unsure of how our rating model behaves. We did not examine whether a change on the lower end of the scale is equivalent to the same numerical change on the higher end of the scale. We also cannot confirm whether a score on one pillar holds the same weight as the same number on another pillar. We recommend that further research be conducted exploring these relationships.
We also recommend that further research be conducted exploring the Community pillar. More extensive research should be performed into discovering what communities are present in social media and what strategies best serve each of them.
The concept of meaningful engagement in social media is vast and we were only able to test one social media platform within the timeframe of our project. We believe that the characteristics and trends we identified can be carried from Facebook into other social media platforms; however, further research can be conducted, exploring these correlations.
Social media is expected to continue to increase in popularity. As organizations and their target audiences take advantage of the quick, easy communication that social media allows for, understanding what meaningful engagement is and how to encourage it will become increasingly important. This project begins to address that growing need, and our hope is that it will not only be used by Te Papa to improve its use of social media, but also provide a strong foundation for future research into this vital field of study.