A framework for interpreting social media interactions in the public sector
Ines Mergel
Department of Public Administration and International Affairs, Maxwell School of Citizenship and Public Affairs, Syracuse University, 436 Crouse-Hinds Hall, Syracuse, NY 13244, USA
a b s t r a c t
Keywords:
Social media measurement Digital interactions Open Government Initiative U.S. federal government
Social media applications are extending the information and communication technology landscape in the public sector and are used to increase government transparency, participation and collaboration in the U.S. federal government. The success, impact and performance of these new forms of bi directional and networked interactions can provide insights to understand compliance with the mandate of the Open Government Initiative.
Many government agencies are experimenting with the use of social media, however very few actively measure the impact of their digital interactions. This article builds on insights from social media directors in the U.S. federal government highlighting their current lack of measurement practices for social media interactions. Based on their articulated needs for measurement, existing rules regulating the extent of measurement practices and technological features of the main social media platforms, a framework is presented that traces online interactions to mission support and the resulting social media tactics. Implications for both researchers and practitioners are discussed.
1. Introduction
Social media use has become an accepted practice in the U.S. federal government and around the world. Following the Open Government Directive agencies are investing resources into working with third party platform providers to harness new technologies to increase citizen participation, collaboration, and transparency (The White House, 2009a,b).
These new forms of digital interaction between government and its stakeholders create the potential to increase democratic engagement and reach online audiences who were not previously involved in policy making activities (Bertot, Jaeger, & Grimes, 2010). At the same time, government opens additional channels for the influx of large amounts of data about digital interactions, content, and expressed online sentiments that need to be analyzed and interpreted to understand to what extent they support government's mission (Lazer et al., 2009).
Many agencies are however reluctant to measure their online interactions, or are even prevented by their interpretation of existing laws and regulations. Investing human and social capital into the use of social media creates on the one hand the potential for government to access innovative knowledge from stakeholders that can help make governmental processes more effective and efficient. On the other hand, merely opening additional channels to broadcast information in order to inform the public, but not reacting to the public's sentiments and feedback loops that are accessible via social media might harm
government's reputation. An important step in using the appropriate metrics therefore includes a deeper understanding of the social and behavioral challenges associated with the interpretation of social media data by public managers as well as social media professionals in government.
Drawing on the existing literature of social media use in government, broader e Government discussions, as well as empirical evidence from in depth, qualitative interviews with public managers in the U.S. Government, this article provides a developmental framework for measuring social media impact in the public sector. First, the article outlines the existing and accepted measurement techniques for e Government services and contrasts them with social media use for online interactions with government stakeholders. The existing rules and regulations that are guiding social media use in the public sector are outlined as the basis for future measurement tactics. Based on the lack of limited measurement attempts by social media directors, a framework for including metrics, procedures, and outcomes is presented that aims to help both researchers and policy makers to measure and interpret social media use in the public sector.
2. Social media in the public sector
Over the past four years agencies and departments in the U.S.
government's executive branch have started to use social media applications, such as Facebook fan pages, Twitter updates, YouTube videos, blogs and RSS feeds. As of May 2012, the 698 departments, agencies, and initiatives of the U.S. federal government have created 2956 Facebook pages, 1016 Twitter accounts, 695 YouTube channels, and 498 Flickr pages to promote their online content and connect to E-mail address:iamergel@maxwell.syr.edu.
Konstanzer Online-Publikations-System (KOPS) URL: http://nbn-resolving.de/urn:nbn:de:bsz:352-0-357029 Erschienen in: Government Information Quarterly ; 30 (2013), 4. - S. 327-334
https://dx.doi.org/10.1016/j.giq.2013.05.015
their stakeholders and audiences (Mergel, 2012a). Some innovators have also ventured into newer platforms, such as the Transportation Security Administration's recent use of the social picture sharing platform Pinterest to publicize confiscated weapons from airplane travelers, or the National Archives and Records Administration's use of the geo location and gaming platform Foursquare to connect citizens with library artifacts and government records.
This development is based on President Obama's“Transparency and Open Government”(OGI) memo in which he highlights three distinct areas of open government (2009): Executive departments and agencies have to increase participation, collaboration and transparency. The OGI specifically urges executive departments and agencies to“harness new technologies”to fulfill the requirements of this mandate.
The use of social media applications by government organizations can be called an extension of the current digitization efforts of government services as a new wave of the e Government era (Bretschneider &
Mergel, 2010). However, it needs to be distinguished from the previous wave of e Government which is still focused on service or program delivery that constitute the core mission of government organizations (Lindgren & Jansson, 2013). Social media applications are used to create additional channels for governments' interactions with its stakeholders (Mergel, 2013). The current tactics do not include individual service delivery, such as e tax filing systems. Instead, government agencies mostly use social media channels to represent artifacts of their core mission, engage the public, or participate in issue conversations and network with stakeholders.
This important distinction between social media use and e Government services hosted on an agency's webserver is also portrayed in the fact that social media applications are provided by third parties, where technological features are hosted outside the direct control of government organizations. The difference to other e Government ap plications (such as static websites as publishing mechanisms for agency information) is a higher degree of interactivity as well as content pro duction by both government and citizens (Cormode & Krishanmurthy, 2008; O'Reilly, 2007). Social media applications allow for multiple authors and authorship rights are distributed to outsiders, including citizens, who are allowed to post blog comments, leave messages, create content on government owned Facebook fan pages, reuse content tweeted by a government agency or forward the content to their own followers. Social media applications are used both internally and externally to reach citizens who are not using the traditional ways of interacting with government. In most cases, social media applications have not replaced existing offline or even e Government services on the contrary all social media applications are used to complement the existing communication mechanisms in government. They allow for two way interaction both within government as intra organizational information sharing mechanisms for example in the form of wikis, as well as inter organizational information sharing tools that allow new forms of interactions with citizens.
Adoption of social media applications is similar to previous waves of e Government applications highly market driven and citizen centric (Lee, Chang, & Stokes Berry, 2011; Mergel, 2012b, 2013). Many agencies are exploring the value of social media to help them fulfill the mission of their own organization and information about the innovative value is slowly spreading throughout the overall government system (see for example Mergel & Bretschneider, 2013; Vonk, Geertman, & Schot, 2007). Informal experimentation enhances the formal top down mandates, but the degrees of innovativeness and extent of adoption vary widely. Generally, social media applications are seen as a new format to respond to citizen or market initiated contacts. So far most initiatives are however of educational and informational values with a very low degree of interaction (replicating what Thomas & Streib, 2003found when they observed citizen initiated contacts in the era of e Government). This study therefore takes the different levels of maturity of social media solutions into account, but then applies the Open Government mandate to understand the development of different
stages of social media interaction efforts and their potential for inter pretation in government.
3. Steps toward institutionalizing social media
The more government agencies use social media, the clearer it became that the existing 2002 E Government Act was limited to email interactions and relative static government website content (United States Congress, 2002). As an example, tracking website traffic or using persistent cookie technology in general is highly regulated in the federal government. A memorandum published in the year 2000 prohibits federal agencies to use persistent cookies to track citizens' visits to a government website (The White House, 2003). Cookies were only allowed with explicit approval. The policy was generally perceived as too constraining and as a barrier (McCarthy & Yates, 2010). An updated policy now states that cookies can be used to help websites deliver personalized versions to their visitors by remembering parts of the customized entries voluntarily contributed by citizens. The OMB M 10 22 guidance on web measurement and customization technologies allows agencies to remember a user's online interactions across a single or multi session in cases when citizens explicitly opt in (The White House, 2010c). Cookies are textfiles that a website places on a visitor's computer so that it can remember the visitor's preferences at future visits. The guidance distinguishes between these tier 1 and tier 2 cookies, which do not collect personal user information. Tier 3 cookies collect users' personal information and require a more extensive review and public comment process, before an agency is allowed to identify individual users. Data retention and access are usually limited to one year or less.
In extension to the existing cookie policy, a recent OMB Memorandum 10 23 provides guidance for government agency's use of third party websites and applications to also include social media and Web 2.0 applications (The White House, 2010a). It directs agencies in their information collection process from third party website:
“If information is collected through an agency's use of a third party website or application, the agency should collect only the information necessary for the proper performance of agency functions and which has practical utility. If personally identifiable information (PII) is collected, the agency should collect only the minimum necessary to accomplish a purpose required by statute, regulation, or executive order.”Especially the following expression:“information for the proper performance of agency functions” leaves room for interpretation. A newer document issued by the General Service Administration's Citizen Engagement office offers an overview of “Digital metrics for Federal Agencies” (The White House, 2012). This new guidance focuses on government owned websites and allows agencies to measure the number and duration of visits, referrals, as well as basic demographic data including network/ISP/organization, country, state/local, or city.
Many federal departments and agencies are therefore still in the middle of navigating the uncertainties of using social media as an extension of the use of their online presence. The initial memo and the subsequent use of social media applications by many early innovators initiated follow up instructions to clarify several aspects of necessary changes. As an example, the U.S. General Services Administration (GSA) developed Terms of Service Agreements with social network services to ensure that the agreements comply with existing government regulations (Aitoro, 2009), NARA provided insights into social media records keeping (Franks, 2010; The White House, 2010d), and the Library of Congress acquired the archive of all tweets ever sent (Library of Congress, 2010).
As the Open Government Initiative evolves and more agencies follow the mandate to“harness new technologies”, it is clear that the executive order requires a set of measures to promote the commitments of transparency, participation, and collaboration. In 2012, the White House therefore released a new Digital Government Strategy that specifically states the need for a coherent set of performance metrics:
“According to the State of the Federal Web Report, only 10% of the 24 major federal agencies use the same performance metrics to con sistently evaluate websites agency wide. But there's a solution for that:“Open web analytics for all .gov websites”, a popular idea sub mitted during the National Dialogue on Improving Federal Websites.” (The White House, 2012). Moreover,“The Social Media Navigator GSA's Guide to Official Use of Social Media”directs agencies to“Monitor what you control: Social media users are responsible for continually checking the pages that they own. The person who has responsibility for approving the page, or their designee, should ensure the information is accurate, timely, relevant, complete, and does not adversely affect the execution of the mission and responsibilities of GSA or the Federal Government. This responsibility includes inactivating the page upon the owner's termination and/or notifying the Office of the Chief Information officer (OCIO) that the page has been transferred to another individual.”(GSA, 2011:11 12).
4. Existing ways of measuring the success of e-Government practices The existing research on e Government measurement techniques focuses mostly on different types of online service delivery in the form of single case studies, or the extent and enumeration of interactive elements of a government agency's website. Measurement techniques include for example Melitski's four step model to evaluate to what degree an agency's website is static, interactive, transactional or transformative (Melitski, 2003). These four dimensions help the author to understand the capacity of e Government performance. A recent study of the Canadian government's website interactions with business owners highlights that those business owners who interact with government through their website are generally more satisfied with government performance (Reddick & Roy, 2013). Another performance measurement model evaluates e Government activities based on input, outcome, intermediate, end or ultimate outcome measures (Stowers, 2004). The results presented in mostly single case studies of specific online interactions, such as e taxfiling, are however mixed. Some studies have found statistical significance as well as other positive outcomes of federal and local governments' use of e Government applications: For example, Tolbert and Mossberger found that providing government internet services increases process based trust by improving interactions with citizens and perceptions of responsiveness (2006). Other studies have found that the mere provision of e Government services does not necessarily lead to greater trust in government operations (Morgeson, VanAmburg, & Mithas, 2010). In turn, providing government infor mation on a website a one directional consumer perspective of the way that citizens interact with government, does not necessarily increase accountability as Pina et al. have shown in their study of local governments in the EU (2010). They conclude that transparency needs to be paired with internal organizational reforms and changes in behavior. Trust in government is however linked to internet use measured as citizen satisfaction with e Government (Welch, Hinnant,
& Moon, 2005). The authors found that citizens are generally satisfied with electronic provision of information as one aspect of transparency, but dissatisfied with electronic transaction and interactivity the service delivery component of e Government. Those citizens who tend to consume information from government website are generally more demanding of interactive services an important insight for the use of even more interactive technologies, such as social media applications.
Other studies declare that e Government itself is still at an early stage and has not obtained many of the expected outcomes (cost savings, downsizing, etc.) that the rhetoric of e Government has promised (Moon, 2002; Norris & Moon, 2005). Government organizations on all levels of governments rush to provide websites, but online government service delivery is still limited. More than a decade later empirical studies hint at potential cost savings in individual services, but not a system wide increase in effectiveness. West claims, “the e Government
revolution has fallen short of its potential to transform service delivery and public trust in government. It does, however, have the possibility of enhancing democratic responsiveness and boosting beliefs that government is effective”(2004, p. 15). Hazlett and Hill emphasize that
“Although there have been examples of very creative use of electronic government in the public sector, there have also been numerous spectacular failures; lack of evidence to support the claim that the use of technology in service delivery results in less bureaucracy and increased quality.”(2003, p. 445). Government agencies do not seem to be able to counteract this trend: A recent PEW internet study found that only 33% of all U.S. citizens trust their government in 2012 indicating that providing static, one directional e Government presences as a form of information provision alone have not made a difference in imposing a positive image or interaction between government and citizens (PewResearch, 2012).
While the measurement of these dimensions is helpful for outside evaluators, the mere provision of numbers of even the most transactional and transformative service does not indicate how useful citizens themselves perceive the service.Suen (2006)suggests that e Government functions, respectively social media interactions can be observed on three different levels of interactivity: one way interactive, semi two way interactive, and two way interactive. One way inter actions focus on informational, educational, government information delivery in the form of email based subscriptions, information accessed on government websites, online voting, opinion polling, or surveys.
Semi two way interactions include online discussion forums, online submissions of comments, online requests for information or services. Finally, two way interactive engagement includes web page personalization, website information search, information query, GIS mapping, live condition or report, as well as virtual city tours (for an overview of a social media engagement ladder based onArnstein (1969)seeNabatchi and Mergel (2010)). So far, the existing literature has not extended its scope to include measurement practices of third party social media platforms and Luna Reyes, Gil Garcia and Romero state that there is“no comprehensive way to evaluate [electronic government] initiatives at the aggregate level” (Luna Reyes, Gil Garcia, & Romero, 2012:324).
As a result, there is a need for a developmental interaction based framework to understand how different levels of citizen interactions with government help to support the mission of a government organization using social media channels. Based on these insights the following empirical research was conducted to extract the need for additional insights as well as the current measurement techniques in the U.S. federal government.
5. Research design
Most of the usage of social media evolved in the form of early experimentation outside of the officially accepted ICT infrastructure and government employees have to abide by the existing rules and regulations that guide their online behavior (Mergel, 2013; Snead, 2013). Many new guidelines were developed in response to the apparent lack and uncertainty to what extent the existing email and web use guidelines cover the use of social media in government.
6. Case selection and context
In order to understand the information needs of government social media professionals when they use social media applications for their government agency, a qualitative approach was used to understand perceptions and internal approaches. For this purpose, social media directors of the departments and agencies in the executive branch of the federal U.S. government were interviewed. The sample includes all those departments in the executive branch who received the formal mandate as part of the Open Government Initiative to harness new technologies. This form of theoretical sampling draws clear boundaries
around the subjects included in this study and focused the data collection efforts toward a full census of individuals responsible for the application of social media in the U.S. federal government (Glaser
& Strauss, 1967; Janelle, 2009).
7. Data collection and analysis
Following Richards' (1996) approach, elite interviews included social media directors responsible for the strategic and managerial efforts of social media implementation as well as managerial oversight of administrative day to day digital interactions with the public.
Their agencies' online interactions are highly visible in the form of social media newsfeeds, however the underlying strategy, day to day management, tactics, interpretation and changes to the existing tactics can only be accessed by extracting the perceptions of the interview partners. None of the 15 departments included in this study and covered by the OGI mandate present their measurement metrics or results publicly on their website. One remarkable exception is the
“CDC.gov Social Media Metrics”monthly reports, that provide charts with daily, monthly page views of the most popular topics, top sites, and pages (CDC, 2012).
Overall 25 interview partners agreed to be included into the study reflecting the full census of all 15 departments, and additional agencies in the executive branch of the U.S. federal government that were frequently named as innovators in the social media space. The semi structured interviews were part of a larger study and therefore contained questions about the target audience(s), goals of the social media strategy, adopted social media tools, managerial day to day processes and social media policies for daily routines, top management buy in, hurdles and benefits, success factors and also the need for social media insights and the currently used measurement metrics.
The interviews included specific questions asking how social media directors implement the OGI executive order to increase transparency, participation, and collaboration. The OGI (The White House, 2009a) provides the following definitions:
• Transparency “provides information for citizens about what their Government is doing.”For the purpose of this study, transparency is therefore interpreted as broadcasting of government information via social media sites;
• Participation should be accomplished by providing “increased opportunities to participate in policymaking and to provide their Government with the benefits of their collective expertise and information.”For the purpose of this study, citizens can engage in preparing policy making decisions by providing their feedback through social media channels.
• Collaboration is defined in the OGI memo as new forms to“solicit public feedback to assess and improve their level of collaboration and to identify new opportunities for cooperation.”For the purpose of this study, social media channels can therefore be used to increase exchanges with citizens or collaboratively work with government stakeholders on innovative ideas to fulfill the mission of government.
The interviews lasted about 60 min, were recorded, transcribed and hand coded paragraph by paragraph in an iterative process to derive the main themes in a grounded theory like approach (Miles &
Huberman, 1994; NVivo 8, 2008). The data analysis of the qualitative interviews started immediately after each interview was conducted:
initial themes and additional innovative concepts that were not mentioned in previous interviews were recorded in memos and used for future inquiry. This review resulted in recurring patterns of social media measurement needs and current practices. After the initial high level coding procedure, the full interviews were coded line by line to extract patterns in the data following Miles and Huberman (1994).
8. Current state of impact measurement of social media activities in the U.S. federal government
At this time, most social media directors interviewed for this study highlighted that they do not actively measure what kind of impact their social media activities have. As one of the interview partners said: “I would call it ‘Return on Ignorance’rather than
‘Return on Investment’ we are not really measuring what we do right now.” Another social media director specifies:“We do web analytics here. The web analytics package we use is not necessarily sophisticated enough to measure [success]. We are in the process of implementing a real time web analytics program that not only measures activity on the website, but also measures thinks like click throughs from social media sites and participation.” These statements show that while social media directors are aware that there might be value in understanding the impact of their interactions, they do not make use of existing measurement instruments, nor do they already have specific metrics or measurement mechanisms in place that help them gain insights.
In addition, it became clear that social media directors have to rely on the reports of third party social media providers and have therefore limited ability to design their own metrics or indicators:
“We do not track any data. Facebook tracks data. As government we are not supposed to drill down too deep into who is using it.”Social media directors rely on the reports of third party social networking applications to provide insights into basic demo graphics, such as gender, or geographic location, such as city and state to understand if their online interactions are actively reaching the target audiences. This quote also hints at the reluctance of changing learned behavior and the limiting legal framework of the cookie policy, which so far limited deep dives into citizen generated use data.
In the following, the measurement practices of the goals of the Open Government Initiative are presented and initial hints at potential interpretations of social media use to support the OGI mandate are provided.
8.1. Transparency
The main reason to engage in social media spaces can be sum marized in one main goal: Representation of the agency on all available online channels. Especially the success of Facebook with about one billion users and Twitter with about 300 million active users in 2012 has convinced social media directors that they want to be where the citizens are and release government information into the newsfeeds that citizens are frequently checking. The following quote is rep resentative to statements of all interview partners: “Why we're on Facebook or Twitter: to be where the people are. So [1 billion] users now on Facebook for example, so you want to be there. When people search for [suppressed by author] stuff on Facebook, they find us.” Respectively, to understand if citizens actuallyfind the information on social media channels, the majority of interview partners noted that an important indicator is the number of followers and viewers of content:“We look at the raw numbers: how many followers did you have when you launched the page, and how many followers you had now. […] Facebook has grown from 0 to 313 in the last three to four months since we have really launched it. It's good it's not fantastic, it's a limited audience. Plus we have not really publicized this. The only way we publicize this is on our homepage. The rest it is all just been through word of mouth.”
The representation objective is clearly to be as inclusive as possible and reach audiences in their social spaces they frequent on a daily basis. Several interview partners recognize the need to reach audiences that do not routinely interact with federal agencies and are therefore excluded from government information. They see the use of additional channels on social networking sites as a way to institutionalize
their interactions and bring government information to citizens. The following quote is therefore representative for many agencies:
“Get the message out to the audiences that might not normally hear it […].”
Social media channels provide some high level insights into the audiences as one social media director points out: “We look at engagements and level of attention: For example, on blogs which are the most popular blogs, how long are people spending on each one? Are they justflipping through, looking at the pictures? Are they spending some time reading? Do they visit multiple blogs when they come, or do they just hit one and then go away? […] Also, where they come from: Is it a Google search, or is it from our [department] page, or from the First Lady's‘Let's move’initiative [as a lead partner]. So seeing that people are coming to us from there helps us more accurately target messages.”
However, a majority of the interviewed social media directors are not sure if they are reaching the audiences their mission statement claims. A detailed analysis of target audiences is not conducted; neither do social media directors know how representative their social media followers are. The goal is to reach maximum transparency by voluntarily releasing government information through channels other than the traditional channels, such as a dedicated website.
8.2. Participation
The second most frequently mentioned objective to maintain social media accounts is citizen engagement. Most social media directors are bound by institutional hurdles they need to overcome and early experimentation has led to the use of social media tools in the same way that traditional static website content is provided and must therefore be classified as broadcasting or pushing out of government information. Some practices indicate that content produced for the core website was duplicated and pushed through social media channels.
While these practices can still be observed, many agencies have recognized the additional value of social networking services:
bidirectional interaction and active networking with the public. Citizens are invited to co produce content that is then replicated on the agencies' websites and used to ask citizens to provide opinions and additional input.
As an example, social media channels are used to pull citizens in to answer online surveys about the content provided:“We do a survey and ask a lot of Yes/No questions, 1 to 10, and allow open ended responses for this. And we do get a good amount of feedback from our most loyal audiences […] in terms of what people are coming to the site looking for.” These customer satisfaction surveys that can be used as quick popup inquiries on an agency's website are subject to restrictions and have to receive clearance from the Office of Management and Budget (OMB) before they can be sent out to citizens or other audience members (The White House, 2010b).
Interactions focus mostly on lower levels of engagement and participation. Beyond providing information to the public, agencies actively seek feedback from citizens through their social media channels. The feedback is then used to increase the quality of thefinal policy, decision, or document as the following quote shows:“We are asking the public specific questions on various topics that are addressed within the planning role [during the draft period of a rule] to get their feedback before we draft it […], so hopefully we will get closer to what is an acceptable and really desirable rule.”
However, practices vary widely: While the majority of agencies uses existing free web analytics tools, such as Google Analytics, and relies on prescribed metrics provided by social media providers, others are more creative and understand how to engage and target audiences on social media. The FCC example provides more detailed insights into these tactics: “We are announcing the Spanish translation
of the National Broadband Plan. We have done a lot of work to reach out to different influential Spanish bloggers to get blog posts translated into English and Spanish, and figure out how we can engage different communities.” This quote shows that government agencies are customizing content to be able to invite comments from very specific audiences, instead of simply pushing out the original version in English.
8.3. Collaboration
Collaboration between government and citizens indicates a higher level of engagement in a reciprocated relationship by allowing the audiences to directly engage with government content and co create government innovations. However, government agencies in this sample were very clear that they do not desire to create a direct, reciprocated relationship with citizens by following citizens back and have creative conversations online. Collaborative engagement is therefore only identifiable in the active interactions of citizens with government provided content:“We do watch retweets [sharing a tweet to a citizen's own Twitter network], when people mention us on Follow Fridays [#FF] and all those different things that show that some messages move better in different channels. But: We don't follow people back [to show] that we don't have an official relationship with [them] if they are not a local, state or federal entity. We don't want to imply that we support or endorse an individual and their opinions and different things we just don't follow unless we have that relationship. So we can't [exchange] direct messages.”While interactions with valuable online content are traceable for government, social media is clearly not used for back and forth conversations that might lead to innovative insights or ideas on how for example government operations can be improved. As a matter of fact true collaboration is channeled to a new platform calledChallenge.gov that provides a place to co create solutions for government problems.
About 50% of the interviewed social media directors recognize that they do not have to play an active role when using social media applications. Instead, a passive strategy to listen and absorb comments provides them with very valuable insights from their audience. While several social media directors mentioned this objective, none of the interview partners was able to point to examples; instead they listed reciprocated feedback and interaction as a desirable goal for their future social media use to become more collaborative.
9. Toward an Open Government framework for interpreting the impact of social media interactions
These findings clearly indicate an increasing paradox: While the Open Government Initiative mandates federal government agencies to engage in higher level online interactions to not only become more transparent, but also increase participation and collaboration, the actual planned interactions are very challenging for social media professionals in government. So far there is limited reflection to strategically plan out engagement activities beyond pushing government information out through social media channels and consequently it is impossible for social media directors to systematically gage the impact they are making.
As one social media director states:“The metrics part is definitely a very important topic that a lot of us are still grappling with. You still need a human being to evaluate that feedback and see if it is really constructive…and we are using the anecdotal evidence we have to show a real impact.”
Looking only at the raw data provided by either third party social media providers or free web analytics tools has limited value to understand and interpret how social media interactions contribute to the mission of each agency or the goals of the Open
Government Initiative. Instead, government professionals need to interpret data in the light of their own agency's mission and the levels of engagement they are aiming to achieve. It would help government agencies to understand to what extent they are reaching the right audiences and on which newsfeeds they are gaining access to citizens' social awareness streams (Naaman, Boase, & Lai, 2010).
The insights provided from social media directors show that government is currently focusing mostly on push techniques and uses social media channels to provide information that is recycled from other government communication channels, such as publications, reports or the website itself. Measures therefore include mostly raw data, such as number of followers and page views. The outcome is similar to static website interactions and is mostly focused on educational or informational purposes (reflecting what Moon, found in his 2002 study on the evolution of e Government services). This one directional way of viewing and reviewing social media interactions however leaves out large portions of the possibilities social media applications provide that can help government understand deeper levels of engagement.
The more government agencies engage in higher levels of citizen participation, measuring the extent to which they are engaging unlikely audiences will help them gain access to innovative knowledge to potentially solve government problems. Asking citizens to submit their ideas or provide media content they have created, such as videos or photos, highlights the potential for bi directional citizen participation:
Citizens' content is actively pulled in through new forms of crowd sourcing and can be reused in government. Examples include comments on draft policies or sentiment analyses of comments by citizens on social media newsfeeds. These features and activities show a higher level of engagement and willingness of citizens to constructively work with the content, add their own ideas and create insights that go beyond the initial postings by government agencies themselves (Halpern & Katz, 2012).
The highest level of engagement however is collaboration supported by social media. Citizens go beyond simply viewing a government agency's online content or commenting on government posts. They are actively interacting with the content provided and seek future engagement opportunities. This can include requests for memberships in a LinkedIn group to participate in problem finding missions facilitated in online discussion groups, subscriptions to a government blog or YouTube channel which shows a continuous engagement with government content beyond a single piece of information or sharing government Facebook updates with their own social network and not just passively absorbing updates. The highest degree of collaboration with government content is reached when citizens are proactively contributing their own content, download videos and documents to work with the content on third party sites or contribute to and even start conversations. As soon as citizens are willing to take offline action based on their previous online interactions with government social media accounts, the highest degree of collaboration is reached. Offline actions can include registering to vote, donating and volunteering time to create a public good, reuse government content to engage with issues, or contributing solutions for government problems.
Overall, it is important to emphasize that government social media professionals can gain important insights by interpreting social media data for decision making. While the raw data high lights attention or at least interest in government information, it is necessary to interpret the data and potentially change social media tactics based on the insights for specific campaigns or routine interactions. The intangible insights are oftentimes difficult to assess and true impact needs to be interpreted in the light of the existing social media goals to support an agency's mission.
The following overview summarizes the framework for measuring social media interactions in the public sector:
Mission Goal Tactics Social media
mechanisms
Outcome
Transparency Information education
One-way push
• Number of followers
& likes/friends (change from start)
• FB likes
• Twitter followers
• Unique visits to blog
• Time spend on pageb30
• Visits only home page
• Views on YouTube & Flickr
• “Read more”
Accountability trust
Participation Engagement Two-way pull
• Click-throughs from social media sites
• Reach: demographic data (gender, location, cities)
• Bookmarking &
digging content
• Twitter retweets, hashtags
• Posting ratings
& reviews
• Spend more than 1 min on site
• Comments on blog & Facebook
• Ratings on YouTube
• Number of links
& trackbacks
• Frequency of check- ins on Foursquare
Consultation, deliberation, satisfaction
Collaboration Cross- boundary action Two-way interactive
Networking Co-design of services
• Request for membership in a LinkedIn group
• Subscriptions to blog, YouTube channel
• Facebook shares
• Twitter direct messages
• Creating their own content
• Downloads of videos, documents
• Conversations
• Volunteering, donations
• Offline actions
Community building Creation of issue networks
10. Implications for social media professionals in the public sector Social media professionals in government have two important tasks in this early experimentation phase with social media metrics. First, they need to understand what data and interactions the current rules and regulations allow government agencies to collect. Second, measuring without goals will unlikely lead to the insights they want to gain.
It is therefore necessary to understand how the use of social media applications as acceptable channels for interactions between government and its diverse audiences might potentially make a difference in the perceptions and sentiments of citizens toward government.
Several important questions can guide the decision making process:
What is the agency's mission and how can social media be used productively to support the mission? Next, it is important to understand the overall composition of an agency's potential audience. Within the (most likely) diverse set of audiences, who are the knowledge hubs and important followers who are willing to distribute government content
through their own issue networks? This will help public managers under stand how content is spreading through the social graph of the network.
Not every type of government content might be suitable for distribution and communication through social media channels. In addition, not every government agency's stakeholders include the public instead they might be other government agencies. Trying to publicly engage with these internal stakeholders won't be appropriate and affordable. It is therefore important for government professional to understand for each update, how it was perceived, how it spread, and how their stakeholders engaged with the information. As a result, the role of government is reduced to a passive listening function without actively being involved in the conversation. Government can then take on the role as a fact checker and merely inject correct and trusted information into the network.
The more government agencies engage on social media, the more citizens will expect responsiveness and real time information sharing in their future interactions with government. However, government agencies rarely have the human resources or the legal freedom to engage in fast and furious exchanges on social media channels, instead every response has to go through a highly restricted information vetting process (Mergel, 2012c). A step wise developmental measurement framework can therefore be helpful to provide guidance for future online interactions.
11. Future research
Social media directors face the ambiguous situation in which only rudimentary measurement techniques are provided by third party service providers and officially approved measurement dashboards are either not available or not affordable. At the same time, citizens are expecting reciprocity and near real time responsiveness from government. In the absence of approved processes and metrics, this study has shown that government organizations revert to existing measurement techniques that are in place for traditional website interactions and are still very reluctant to take the insights into consideration. A majority of the interview partners expressed the need to explore and understand their audience's interactions with the technology better and integrate the lessons learned to improve their standard operating procedures in order to create more efficient and effective government operations.
Social media technologies themselves have not changed citizens' behavior citizens were talking about good and bad government practices with the help of other publishing mechanisms before social media was available. However, social media applications provide new forms of publicness and openness for interactions and content sharing.
Having tools in place that help government to understand and follow issues that are developing can help social media directors to frame their content delivery and change their tactics to create more effective online interactions.
An example is thefirst official sentiment analysis launched by the Singaporean government to analyze how citizens think about their government (Hicks, 2010). Similar to the view expressed but used on an accidental case by case basis the Singapore government focuses on passive listening activities instead of merely pushing information toward their citizens. Other techniques that may help social media directors to understand their actual reach and the qualitative dimensions of their interactions with citizens include Twitter mood indicators (Mislove, Lehmann, Ahn, Onnela, & Rosenquist, 2010), or the analysis of an agency's Twitter network composition with the help of computational social sciences methods (Lazer, Pentland, Adamic, Aral, Barabási, et al., 2009; Naaman et al., 2010). Future research is needed to design and test social media metrics on a large scale quantitative basis and compare the data with agency missions and social media strategies.
References
Aitoro, J. R. (2009).GSA signs deals for agencies to use social networking sites.NextGov, 03/25/2009 (online. Retrieved fromhttp://www.nextgov.com/nextgov/ng_20090325_
5490.php?oref=search)
Arnstein, S. R. (1969).A ladder of citizen participation.Journal of the American Institute of Planners,35(4), 216–224.
Bertot, J. C., Jaeger, P. T., & Grimes, J. M. (2010).Using ICTs to create a culture of transparency: E-government and social media as openness and anti-corruption tools for societies.Government Information Quarterly,27(3), 264–271.
Bretschneider, S. I., & Mergel, I. (2010).Technology and public management information systems: Where have we been and where are we going. In D. C. Menzel, & H. J.
White (Eds.), The state of public administration: Issues, problems and challenges (pp. 187–203). New York: M.E. Sharpe, Inc.
CDC (2012). CDC eHealth metrics dashboard. from.http://www.cdc.gov/metrics/
socialmedia/
Cormode, G., & Krishanmurthy, B. (2008).Key differences between Web 1.0 and Web 2.0.
First Monday—Peer Reviewed Journal of the Internet,13(6), 1–19.
Franks, P. C. (2010).How federal agencies can effectively manage records created using new social media tools. In I. C. f. t. B. o. Government (Ed.),Using technology series (pp. 54). Washington, D.C.: IBM Center for the Business of Government.
Glaser, B. G., & Strauss, A. L. (1967).The discovery of grounded theory: Strategies for qualitative research.New York: Aldine Publishing.
GSA (2011). The social media navigator—GSA's guide to official use of social media.
Retrieved from.http://www.gsa.gov/graphics/staffoffices/socialmedianavigator.pdf Halpern, D., & Katz, J. E. (2012).From e-government to social network government: Towards
a transition model.Paper presented at the WebSci 2012. Evanston, Illinois: USO.
Hazlett, S. -A., & Hill, F. (2003).E-government: The realities of using IT to transform the public sector.Managing Service Quality,13(6), 445–452.
Hicks, R. (2010).Singapore to mine citizen sentiment online.Government analytics (Retrieved 11/28/2010, fromhttp://www.futuregov.asia/articles/2010/nov/22/singapore-mine- citizen-sentiment-social-media/#)
Janelle, R. (2009).GCPedia a success, says Government of Canada CIO. TechVibes.combhttp://
TechVibes.comN, Retrieved onlinehttp://www.techvibes.com/blog/gcpedia-a-success- says-government-of-canada-cio, 11/05/2013
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A. -L., Brewer, D., et al. (2009).
Computational social science.Science,323(5915), 721–723.
Lee, C. -P., Chang, C., & Stokes Berry, F. (2011).Testing the development and diffusion of e-government and e-democracy: A global perspective.Public administration review, 444–545.
Library of Congress (2010). Twitter donates entire tweet archive to Library of Congress.
Retrieved 11/16/2010, from.http://www.loc.gov/today/pr/2010/10-081.html Lindgren, I., & Jansson, G. (2013).Electronic services in the public sector: A conceptual
framework.Government Information Quarterly,30(2), 163–172.
Luna-Reyes, L. F., Gil-Garcia, J. R., & Romero, G. (2012).Towards a multidimensional model for evaluating electronic government: Proposing a more comprehensive and integrative perspective.Government Information Quarterly,29, 324–334.
McCarthy, L., & Yates, D. (2010).The use of cookies in federal agency web sites: Privacy and recordkeeping issues.Government Information Quarterly,27, 231–237.
Melitski, J. (2003).Capacity and e-government performance: An analysis based on early adopters of internet technologies in New Jersey.Public Performance & Management Review,26(4), 376–390.
Mergel, I. (2012a).Social media in the public sector: Participation, collaboration, and transparency in the networked world.San Francisco: Jossey-Bass.
Mergel, I. (2012b).A manager's guide to designing social media strategies in the public sector. In I. C. f. t. B. o. Government (Ed.),Using technology—Special report. : IBM Center for the Business of Government (Washington, D.C.).
Mergel, I. (2012c).Tools help agencies manage social media messaging.NextGov (Retrieved from http://www.nextgov.com/technology-news/tech-insider/2012/10/government- finding-measured-voice-social-media/58923/)
Mergel, I. (2013).Social media adoption and resulting tactics in the U.S. federal government.Government Information Quarterly,30(2), 123–130.
Mergel, I., & Bretschneider, S. (2013).Theoretical model of adoption process with application to social media practices in the public sector: Where the formal and informal organizations meet!Public Administration Review(forthcoming).
Miles, M. B., & Huberman, A.M. (1994).Qualitative data analysis: An expanded sourcebook (2nd ed.)Thousand Oaks: Sage Publications.
Mislove, A., Lehmann, S., Ahn, Y. -Y., Onnela, J. -P., & Rosenquist, J. N. (2010).Pulse of the nation: U.S. mood throughout the day inferred from Twitter 2010. (fromhttp://www.
ccs.neu.edu/home/amislove/twittermood/)
Moon, M. J. (2002).The evolution of e-government among municipalities: Rhetoric or reality?Public Administration Review,62, 424–433.
Morgeson, F. V., III, VanAmburg, D., & Mithas, S. (2010).Misplaced trust? Exploring the structure of the e-government-citizen trust relationship.Journal of Public Administration Research & Theory,21, 257–283.
Naaman, M., Boase, J., & Lai, C. -H. (2010).Is it really about me?: Message content in social awareness streams.Paper presented at the CSCW '10 proceedings of the 2010 ACM conference on computer supported cooperative work Savannah, Georgia, USA.
Nabatchi, T., & Mergel, I. (2010).Participation 2.0: Using internet and social media technologies to promote distributed democracy and create digital neighborhoods.
In J. H. Svara, & J. V. Denhardt (Eds.),Promoting citizen engagement and community building(pp. 80–87). Phoenix, AZ: Alliance for Innovation.
Norris, D., & Moon, M. J. (2005).Advancing e-government at the grassroots: Tortoise or hare?The Public Administration Review,65(1), 64–75.
NVivo 8 (2008).NVivo qualitative data analysis software (version 7th).Melbourne: QSR International Pty Ltd.
Obama, B. (2009).Transparency and open government: Memorandum for the heads of executive departments and agencies.Whitehouse blog (Retrieved fromhttp://www.
whitehouse.gov/the_press_office/TransparencyandOpenGovernment/)
O'Reilly, T. (2007).What is Web 2.0: Design patterns and business models for the next generation of software.Communications & Strategies,65(1), 17–37.
PewResearch (2012). Growing gap in favorable views of federal, state governments.
Retrieved from.http://www.people-press.org/2012/04/26/growing-gap-in-favorable- views-of-federal-state-governments/
Reddick, C., & Roy, J. (2013).Business perceptions and satisfaction with e-government:
Findings from a Canadian survey.Government Information Quarterly,30(1), 1–9.
Richards, D. (1996).Elite interviewing: Approaches and pitfalls.Politics,10(3), 199–204.
Snead, J. (2013).Social media use in the U.S. executive branch.Government Information Quarterly,30(1), 56–63.
Stowers, G. N. L. (2004).Measuring the performance of e-government. In I. C. f. t. B. o.
Government (Ed.),E-government seriesWashington, D.C.: IBM Center for the Business of Government.
Suen, I. -S. (2006).Assessment of the level of interactivity of e-government functions.
Journal of E-Government,3(1), 29–51.
The White House (2003). OMB guidance for implementing the privacy provisions of the E-Government Act Of 2002. M-03-22. Retrieved from.http://www.whitehouse.
gov/omb/memoranda_m03-22
The White House (2009a). The Open Government Initiative. from.http://www.
whitehouse.gov/open
The White House (2009b).The Open Government Initiative.Washington, DC: White House (Retrieved from http://www.whitehouse.gov/sites/default/files/omb/
assets/memoranda_2010/m10-06.pdf)
The White House (2010a).Guidance for agency use of third-party websites and applications— Memorandum for the heads of executive departments and agencies.Washington, DC:
White House (Retrieved from http://www.whitehouse.gov/sites/default/files/omb/
assets/memoranda_2010/m10-23.pdf)
The White House (2010b).Guidance for agency use of third-party websites and applications— Memorandum for the heads of executive departments and agencies.M-10-23 (Retrieved fromhttp://www.whitehouse.gov/sites/default/files/omb/assets/memoranda_2010/
m10-23.pdf)
The White House (2010c). Guidance for online use of web measurement and customization technologies. Retrieved from. http://www.whitehouse.gov/sites/
default/files/omb/assets/memoranda_2010/m10-22.pdf
The White House (2010d). Social media, web-based interactive technologies, and the Paperwork Reduction Act. 7. Retrieved from.http://www.whitehouse.gov/
sites/default/files/omb/assets/inforeg/SocialMediaGuidance_04072010.pdf The White House (2012). Digital government: Building a 21st century platform
to better serve the American people. Retrieved from. http://www.
whitehouse.gov/sites/default/files/omb/egov/digital-government/digital- government-strategy.pdf
Thomas, J. C., & Streib, G. (2003).The new face of government: Citizen-initiated contacts in the era of e-government.Journal of Public Administration Research and Theory,13, 83–102.
Tolbert, C. J., & Mossberger, K. (2006).The effects of e-government on trust and confidence in government.Public Administration Review,66(3), 354–369.
United States Congress (2002).E-Government Act of 2002.107th Congress: Public Law 107-347 (Washington, DC).
Vonk, G., Geertman, S., & Schot, P. (2007).New technologies stuck in old hierarchies:
The diffusion of geo-information technologies in Dutch public organizations.
Public Administration Review,67(4), 745–756.
Welch, E. W., Hinnant, C. C., & Moon, M. J. (2005). Linking citizen satisfaction with e-government and trust in government.Journal of Public Administration Research
& Theory10531858,15(3), 371–391.
West, D.M. (2004).E-government and the transformation of service delivery and citizen attitudes.Public Administration Review,64(1), 15–27.
Ines Mergelis an Assistant Professor of Public Administration at the Maxwell School of Citizenship and Public Affairs and The Information Studies School (ischool) at Syracuse University. She was previously a postdoctoral research fellow at Harvard's Kennedy School of Government, Program of Networked Governance and the National Center for Digital Government. She teaches in the Master of Public Administration program courses on Government 2.0, New Media Management in the Public Sector, Networked Governance, and Public Organizations & Management. Her research interest focuses on informal networks among public managers and their adoption and use of social media technologies in the public sector. In particular, she studies how public managers search, share and reuse knowledge they need to fulfill the mission of their agency.
A native of Germany, she received a B.A. and M.B.A.-equivalent in business economics from the University of Kassel, Germany. She received a Doctor of Business Administration (D.B.A.) in information management from the University of St. Gallen in Switzerland and spent six years as pre- and postdoctoral fellow at Harvard's Kennedy School of Government where she conducted research on public managers' informal social networks and their use of technology to share knowledge.
