Wikipedia Sources: Managing Sources in Rapidly Evolving Global News Articles On The English Wikipedia

Wikipedia
Sources:
Managing sources in rapidly evolving global news articles on the English Wikipedia

An Ushahidi/SwiftRiver Report by Heather Ford, Ushahidi/SwiftRiver Ethnographer, August, 2012
Wikipedia Sources: Managing sources in rapidly evolving global news articles on the English Wikipedia by Heather Ford, Ushahidi is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License (go here http://creativecommons.org/licenses/by-ncsa/3.0/deed.en_US for the full license).
Wikipedia Sources: On the books and on the ground 2012 Heather Ford. Ushahidi 1
Table of contents
1. 2. 3. 4. 5. 6. 7. 8.

Introduction Acknowledgements Executive summary Preface Sources on the books Sources on the ground: The Case of the 2011 Egyptian Revolution Wikipedia article Conclusions and design considerations Bibliography

3 5 6 13 15 22 49 57
Wikipedia Sources: On the books and on the ground 2012 Heather Ford. Ushahidi
Introduction

How is a Wikipedia article different from a news article about the 2011 Egyptian revolution at different points in the storys evolution? What are the roles of social media and other Internet sources in rapidly evolving articles? And what, really, is Wikipedias working perspective on social media sources? This report tells the story of Wikipedia sources through a series of case studies including the 2011 Egyptian Revolution Wikipedia article, highlighting how sources were chosen and categorized, what were the most important variables used in discussing sources, and what this might mean for future tool-building and other projects related to sources. What do we mean by sources? My working definition of sources comes from Wikipedias definition but with one important addition. While Wikipedia refers to the characteristics of sources in terms of the references cited in articles (including characteristics of a) the article or book etc, b) the creator and the c) publisher of the work), I add the Wikipedia editor as an important source of the evolving article. Wikipedia editors, it turns out, are important arbiters of truth in rapidly evolving articles, especially since few secondary sources are available to provide analysis (or summaries etc) of the events so close to its start. And although many claim that Wikipedians play a mere janitorial role, it is clear from at least the examples discussed in this report that a much more fundamental role is being played by Wikipedians, especially in the early stage of an event. Why are sources important? Sources (both the reference and editor variety) are important to understand because they mediate what we read, see and hear on Wikipedia. Wikipedia is built on the three foundational content policies of verifiability, no original research and neutral point of view. This means that, according to policy, Wikipedians may only add information that reflects what it calls reliable sources. Furthermore, Wikipedians must add information in a way that fairly represents all significant views of those sources. Sources are used for two key reasons: the first is to determine the notability of an articles subject, the second is to verify information contained within the article. If we look at broad patterns in the English Wikipedias article content, we can see how articles tend to represent the worldview of Wikipedia editors (the sources that I add to the definition provided by Wikipedia). Mark Graham, for example, has shown how the place in which most editors live corresponds with the places that are represented on Wikipedia. He has found that large parts of Africa remain invisible as a result of these differences (Graham, 2011). If we look at another diagram created by Graham and his colleagues at the Oxford Internet Institute (Graham, Stephens, Hale, & Kono, 2012), we can see how these patterns are broadly similar to patterns in the locations of the worlds academic journals, one of the key types of knowledge that Wikipedians consider reliable. Consequently, the location of editors (as one type of Wikipedia source) is a variable that might help readers understand the completeness of an article. Trying to map out these variables is touched on in this report, but could also be expanded in future research.
Above map from Wiki Space: Palimpsests and the Politics of Exclusion by Mark Graham in Critical Point of View: W ikipedia Reader, 2011

Bar graph to the right entitled The location of academic knowledge from Graham, M., Hale, S. A. and Stephens, M. (2011) Geographies of the Worlds Knowledge. Ed. Flick, C. M., London, Convoco! Edition
How to design to improve source management? Understanding how Wikipedians actually choose, verify, replace and debate sources is important for designing better source management tools. But what exactly is a better source management tool? Although better can mean many different things to different individuals and groups, Ive taken it here to reflect Wikipedias ultimate goal of becoming a globally relevant resource. In this case, source management should be improved for a) greater ease of use and b) greater accessibility, encouraging design patterns that increase the transparency of source origins and other characteristics with the ultimate goal of increasing the diversity of sources being used in the encyclopaedia. In accordance with key Wikipedia principles, I believe design should focus on the ability of a reader to easily check back to see whether the information in a Wikipedia article reflects what are the significant views about a particular subject. Making the characteristics of sources important to decision-making more transparent is therefore an important design goal because we want people to choose and use a source because it is relevant rather than because it will attract the least debate. The report is comprised of three main sections: the first considers what Wikipedia policy says about sources, the second presents findings from a grounded theory study of the 2011 Egyptian Revolution article, and the third outlines key design considerations based on these findings.

Acknowledgements
Id like to thank a number of individuals and organizations who made this research possible. Thanks to Hapee de Groot from Hivos and Janet Haven from OSI for taking a chance by funding their first ethnographic research project and doing so with a newbie ethnographer. Thanks to the Wikimedia Foundation, in particular Dario Taraborelli for continually connecting me to other relevant research projects, and to Erik Moeller from dreaming up the project with former SwiftRiver director, Jon Gosier, and for enabling me to base myself at the Wikimedia Foundation offices in San Francisco for the past eight months. To Professor Jenna Burrell from the UC Berkeley School of Information for her mentorship and assistance, and to Wikipedians, including Dror Kamir who reviewed interview questions. Thanks also to Shilad Sen and Dave Musicant for entertaining regular, fascinating discussions about sources over the past eight months or so, to Rachelle Annechino for her tremendous help editing this document and last but not least, the courageous Wikipedians who I interviewed throughout this research and who shared their perspectives with me.
Executive summary
Almost a year ago, I was hired by Ushahidi to work as an ethnographic researcher on a project to understand how Wikipedians managed sources during breaking news events. Ushahidi cares a great deal about this kind of work because of a new project called SwiftRiver that seeks to collect and enable the collaborative curation of streams of data from the real time web about a particular issue or event. If another Haiti earthquake happened, for example, would there be a way for us to filter out the irrelevant, the misinformation and build a stream of relevant, meaningful and accurate content about what was happening for those who needed it? And on Wikipedias side, could the same tools be used to help editors curate a stream of relevant sources as a team rather than as individuals?
Figure 1 Original designs for voting a source up or down in order to determine veracity
When we first started thinking about the problem of filtering the web, we naturally thought of a ranking system which would rank sources according to their reliability or veracity. The algorithm would consider a variety of variables involved in determining accuracy as well as whether sources have been chosen and voted up or down by users in the past. Eventually the algorithm would be able to suggest sources according to the subject at hand. My job would be to determine what those variables are, i.e. what were editors looking at when deciding whether to use a source or not? I started the research by talking to as many people as possible. Originally I was expecting that I would be able to conduct 10-20 interviews as the focus of the research, finding out how those editors went about managing sources individually and collaboratively. The initial interviews enabled me to hone my interview guide. One of my key informants urged me to ask questions about sources not cited as well as
those cited, leading me to one of the key findings of the report: that the citation is often not the actual source of information and is often provided in order to appease editors who may complain about sources located outside the accepted Western media sphere. But I soon realized that the editors with whom I spoke came from such a wide variety of experience, work areas and subjects that I needed to restrict my focus to a particular article in order to get a comprehensive picture of how editors were working. I chose the 2011 Egyptian revolution article because I wanted a globally relevant breaking news event that would have editors from different parts of the world working together on an issue with local expertise located in a language other than English. Using Kathy Charmazs grounded theory method, I chose to focus on editing activity (in the form of talk pages, edits, statistics and interviews with editors) from the 25th of January, 2011 when the article was first created (within hours of the first protests in Tahrir Square), to the 12th of February when Mubarak resigned and the article name was changed from '2011 Egyptian protests' to '2011 Egyptian revolution'. After reviewing the big picture analyses of the article using Wikipedia statistics of top editors, and locations of anonymous editors etc, I started work with an initial coding of the actions taking place in the text, asking the question What is happening here? I then developed a more limited codebook using the most frequent/significant codes relating to editor action and proceeded to compare different events with the same code (looking up relevant edits of the article in order to get the full story), and to look for tacit assumptions that the actions left out.
Figure 2 Screenshot of my Evernote desktop showing initial coding
I then moved to writing a series of thematic notes on what I was seeing, trying to understand, through writing, what the common actions might mean. I finally moved to the report writing, bringing together what I believed were the most salient themes into a description and analysis of what was happening according to the two key questions that the study was trying to ask i.e. How do Wikipedia editors, working together, often geographically distributed and far from where an event is taking place, piece together what is happening on the ground and then present it in a reliable way? And: how could this process be improved? Ethnographymatters has a great post by Tricia Wang that talks about how ethnographers contribute (often invisible) value to organizations by showing what shouldnt be built, rather than necessarily improving a product that already has a host of assumptions
built into it. And so it was with this research project that I realized early on that a ranking system conceptualized this way would be inappropriate for the single reason that along with characteristics for determining whether a source is accurate or not (such as whether the author has a history of presenting accurate news article), there are a number of important variables that are independent of the source itself. On Wikipedia, these include variables such as the number of secondary sources in the article (Wikipedia policy calls for editors to use a majority of secondary sources), whether the article is based on a breaking news story (in which case the majority of sources might have to be primary, eyewitness sources), or whether the source is notable in the context of the article (misinformation can also be relevant if it is widely reported and significant to the course of events as Judith Millers NYT stories were for the Iraq War). This means that you could have an algorithm for determining how accurate a source has been in the past, but whether you make use of the source or not depends on factors relevant to the context of the article that have little to do with the reliability of the source itself. Another key finding recommending against source ranking is that Wikipedias authority originates from its requirement that each potentially disputed phrase is backed up by reliable sources that can be checked by readers, whereas source ranking necessarily requires that the calculation be invisible in order to prevent gaming. It is already a source of potential weakness that Wikipedia citations are not the original source of information (since editors often choose citations that will be deemed more acceptable to other editors) so further hiding how sources are chosen would disrupt this important value. On the other hand, having editors provide a rationale behind the choice of particular sources, as well as showing the variety of sources rather than those chosen because of loading time constraints may be useful especially since these discussions often take place on talk pages but are practically invisible because they are difficult to find. Analysing the talk pages of the 2011 Egyptian revolution article case study enabled me to understand how Wikipedia editors set about the task of discovering, choosing, verifying, summarizing, adding information and editing the article. It became clear through the study of hundreds of talk pages that editors were:
a) storing discovered articles either using their own editor domains by putting relevant articles into categories or by alerting other editors to breaking news on the talk page,
b)
c)
d)
e)
f)
choosing sources by finding at least two independent sources that corroborated what was being reported but then removing some of the citations as the page became too heavy to load, verifying sources by finding sources to corroborate what was being reported, by checking what the summarized sources contained, and/or by waiting to see whether other sources corroborated what was being reported, summarizing by taking screenshots of videos and inserting captions (for multimedia) or by choosing the most important events of each day for a growing timeline (for text), Figure 3 Wikipedia editor, audes method for storing relevant articles during the initial days of the Egyptian adding text to the article by choosing revolution how to reflect the source within the articles categories and providing citation information, and editing by disputing the way that editors reflected information from various sources and replacing primary sources with secondary sources over time.
It was important to discover the work process that editors were following because any tool that assisted with source management would have to accord as closely as possible with the way that editors like to do things on Wikipedia. Since the process is managed by volunteers and since volunteers decide which tools to use, this becomes really critical to the acceptance of new tools.
10
Figure 4 Work process for sources in breaking news stories
After developing a typology of sources and isolating different types of Wikipedia source work, I identified two sets of design considerations as follows: 1. The first would be to for designers to experiment with exposing variables that are important for determining the relevance and reliability of individual sources as well as the reliability of the article as a whole.
2. The second would be to provide a trail of documentation by replicating the work process that editors follow (somewhat haphazardly at the moment) so that each source is provided with an independent space for exposition and verification, and so that editors can collect breaking news sources collectively. Regarding a ranking system for sources, I'd argue that a descriptive repository of major media sources from different countries would be incredibly beneficial but that a system for determining which sources are ranked highest according to usage would yield really limited results (we know, for example, that the BBC is the most used source on Wikipedia by a high margin, but that doesn't necessarily help editors in choosing a source for a breaking news story). Exposing the variables used to determine relevancy (rather than adding them up in invisible amounts to come up with a magical number) and showing the progression of sources over time offers some opportunities for innovation. But this requires developers to think out of the box in terms of what sources (beyond static texts) look like, where such sources and expertise are located and how trust is garnered in the age of Twitter.
12
Preface

How do Wikipedia editors, working together, often geographically distributed and far from where an event is taking place, piece together what is happening on the ground and then present it in a reliable way? What is the role of social media in this process? These were the central questions prompting this project to analyse the current systems that editors use to track and verify news sources on rapidly evolving pages of .en Wikipedia and to investigate the role of social media in this process. Meeting in the early part of 2011, Wikipedia Vice President, Erik Moeller, and Jon Gosier, then-Director of SwiftRiver, an Ushahidi initiative, discussed the opportunity to collaborate. Wikipedians needed better ways of analysing the reliability of news stories and eyewitness accounts from social media sources during important global events, and SwiftRiver needed a partner to test the new software that we were developing to build intelligence into real time news streams. I was brought on in June last year to research ways that Wikipedians were using to manage social media sources during important global events. The problems we were attempting to tackle with this project are broad and complex. 1. There is no transparent way of seeing how each Wikipedia editor has come to the conclusion to accept one news source as credible while dismissing others. 2. Wikipedia editors have limited means of assessing local notability, especially in media environments dissimilar from those in the region where the majority of editors are based. 3. There is no way for Wikipedians to collaboratively track news sources with each editor using their own system for tracking the news during high volume news events. 4. There is widespread confusion surrounding Wikipedias policy on the use of social media sources, with many editors simplifying what is actually a very nuanced policy by decrying the use of any content encased in a social media wrapper. I started the project thinking that I would be through with the research in no time. But as anyone studying Wikipedia knows, the masses of data available for doing this kind of research make the experience a rich one, but can make the process very time intensive, especially since I was determined to gain a deep understanding of how Wikipedia editors work to construct a truth of what has happened using specific examples.
In order to gain this understanding, I used grounded theory methodology (Charmaz, 2006) to analyse data and build theories out of that data. My data consisted of hundreds of pages of Talk from the first 15 days of the 2011 Egyptian Revolution Wikipedia article, policy pages, policy debates on Meta and on Foundation-l (now Wikimedia-l) mailing lists, as well as interviews with editors and Wikimedia Foundation staff. I began my analysis with an initial coding of these documents, drawing out common themes and action verbs, and then developed a focused coding rubric. Drawing together examples of common themes, I then analysed the data, writing a series of memos as I became more comfortable with the data. The result is an initial report on my findings, with a focus on designers and developers who are interested in developing tools for source management. Ethnography is known for being an expensive research process: it is incredibly time intensive because it aims for comprehensiveness and thoroughness. I hope that this report does the method justice. If there are any errors or weaknesses, they are my own. Im looking forward to hearing the response of Wikipedians and Ushahidi folks to the research in the coming weeks, especially as we continue with the second phase of the Understanding Sources project that consists of tool building and development.

14
A. Sources on the books

a) Wikipedias Core content policies Wikipedias policy on social media sources emerges from its core principles (http://en.wikipedia.org/wiki/Wikipedia:Core_content_policies). The three core content policies include Neutral Point of View, No original research and Verifiability, each of which have an impact on which sources are chosen and which sources endure on the encyclopedia. Core content policies Neutral points of view No original research
Verifiability Neutral point of view (NPOV) is the principle that asks editors to proportionally reflect the current state of knowledge about a subject using independent sources, rather than to strive to reflect their version of what they perceive as the truth. For example, if Wikipedia was around in Greece in the early part of the classical period, it would reflect that the earth was flat, even if some Wikipedians believed that the earth was, indeed, round. At the first publication of significant disputes on the matter, Wikipedians might write that The earth is flat. Some have disputed this claim because with links to reliable sources which might consist of academic papers and peer reviewed journal articles. As opinion gradually tipped towards the current understanding, Wikipedians would have had to decide whether reliable sources collectively indicated that the statement should be changed to The earth is round. Since the decision is not simply a question of calculating the volume of reliable sources in favour or against but rather objectively assessing the quality and dispersion of new ideas, this can be a complex process of debate and wrangling as opposing sources are interrogated, arguments are made on talk pages, edits are refined and consensus is reached, before the process starts up again, iteratively reflecting the current state of play in the relevant field. No original research refers to the policy against using information for which a reliable source does not exist, or publishing what Wikipedians call original thought. At first glance, this rule seems fairly obvious. But disputes arise when Wikipedians synthesize existing sources and others declare that in summarizing, they have created original research. Verifiability is the principle that material (especially information likely to be challenged or being challenged) should be backed up by reliable sources. Verifiability is framed as a way of empowering readers, making transparent the path by which information came to Wikipedia and providing the ability for readers to be able to check whether claims on Wikipedia are backed up by reliable sources. Verifiability is probably the most important policy with regard to source choice but can become problematic when there seem to be
equally reliable sources for opposing views about simple facts and where the reliability (a predominantly subjective term) of certain sources is in dispute. b) Verifiability Verifiability is probably the most important principle defining how sources are used on the encyclopedia. Many of us have seen the citation needed link on a Wikipedia page. According to Wikipedia, anyone may question an uncited claim by inserting a {{Citation needed}} tag. Wikipedia focuses on this requirement as a way to keep Wikipedia content verifiable, enabling any user to check whether what is claimed on Wikipedia is accurate. Figure 5: xkcd cartoon from The geeky cartoonist Randall Munroe, creator of xkcd, illustrates this ability in the https://en.wikipedia.org/wiki/File:Webcomic_xkcd_cartoon to the right. xkcd depicts someone in a crowd asking the politician at the _Wikipedian_protester.png lectern to provide a source or citation for what he is talking about. He may be referring to the fact that politicians are not required to provide citations and perhaps that this might be part of the problem of politics. On Wikipedia, however, editors are held accountable for what they write. According to policy, the burden of evidence lies with the editor who has added the information and any editor may remove information lacking an inline citation (WP: BURDEN) The process of verifiability mirrors one of the key principles of open source software: being able to look under the hood to see how it works. In the same way that free and open source software declares the ability of any programmer to be able to look at the code of a program, the Wikipedia verifiability Figure 6 Example from Wikipedia page on 'Citation needed' (WP:Citation needed) principle declares that ordinary Wikipedia users should be able to check whether a source is being accurately reflected on Wikipedia. Encyclopedic information is not entirely the same as computer code, though, and this is where problems arise on the ground as we shall see later.
c) Reliable sources
16
Reliable sources may be published materials with a reliable publication process, authors who are regarded as authoritative in relation to the subject, or both. (WP:Reliable sources) Wikipedia requires citations for information that may be contested. But not any citation will do. Wikipedia relies on what it calls reliable sources (or RS for short). Wikipedia sources are at the heart of the encyclopedia since it is built on the claim that it doesnt introduce any new knowledge but rather merely represents what experts already know. As you can see from the above core content policies, reliable sources is the central cog in the wheel of Wikipedia work. Understanding how to recognise or reach consensus on which sources are reliable, how to accurately reflect reliable sources in article that they summarize, as well as how to recognise when one source trumps another in a particular circumstance are all critical skills for successful Wikipedians. If a Wikipedia administrator does not recognize any reliable sources in a newly-created article, it could be deleted immediately on the grounds of no indication of importance without any due process (CSD), or put up for deletion by any Wikipedian who questions its importance by virtue of its lack of sources. A 2011 study by Geiger and Ford found that No indication of importance was overwhelmingly the most frequent reason for the speedy deletion of articles and case studies like the Missing Wikipedians (Ford, 2011) indicate that reliable sources was at the center of deletion debates. According to the Wikipedia: Reliable Sources policy, the word source has three related meanings all of which can affect reliability. It is a common misconception that Wikipedia does not allow social media sources. Actually, The word "source" as used on Wikipedia has three related meanings: Wikipedia policy describes a rough hierarchy the piece of work itself (the article, book), of what constitutes reliable sources from self-published sources, sources such as the creator of the work (the writer, journalist), personal web pages and Tweets which and the publisher of the work (for example, Random House or Cambridge should generally not be used, to third-party, University Press). published sources with a reputation for fact- checking and accuracy which should be used Any of the three can affect reliability. Reliable sources may be published materials as often as possible. with a reliable publication process, authors who are regarded as authoritative in relation to the subject, or both. But for each of these media, the policy states From Wikipedia:Reliable sources some exceptions. Self-published sources can
be used to support information about a person or organization being described in articles, and opinion pieces from third-party, published sources might not be used if they are not subject to sufficient editorial oversight. An article about President Barack Obama, for example, might use a Tweet from his verified Twitter account as an example of his unofficial position on a particular issue. Mainstream news reporting is generally considered to be reliable for statements of fact, though even the most reputable reporting sometimes contains errors (WP: Identifying reliable sources) Sometimes news reports are used to back up facts, sometimes they are treated as primary sources and therefore unreliable. Although policy talks about different types of media (e.g. books and academic journals) being more reliable than others, it continually refers to the fact that the reliability of a source depends on context (WP: Identifying reliable sources). Extracting variables in the policy for determining whether a source should be used or not, I created the following list: Variable Rule Exception Number of people involved the more people engaged in checking facts, claims of peer review shouldnt be taken as evidence analysing legal issues, and scrutinizing the that the journal is respected, or that any meaningful writing, the more reliable the publication peer review occurs Peer-reviewed scholarly articles, complete dissertations since they are vetted by the scholarly community Number of citations in Another way of confirming that it has been A corollary is that journals not included in a citation citation indexes vetted and entered mainstream academic index, especially in fields well covered by such discourse (published) is checking the scholarly indexes, should be used with caution, though citations it has received in citations indexes. whether it is appropriate to use will depend on the context. type of article biographies of living persons strictly enforced Contentious material about living persons that is unsourced or poorly sourced whether the material is negative, positive, neutral, or just questionable should be removed immediately and without waiting for discussion
18
published status primary, secondary or tertiary sources
Wikipedia distinguishes between self-published and published by a reputable third party Focus is on secondary sources, although primary and tertiary sources are allowed in some circumstances. Primary sources should be used with care Articles should rely on secondary sources whenever possible. For example, a review article, monograph, or textbook is better than a primary research paper
Specific to news reporting identity of the author opinions of specialists and recognized experts are more likely to be reliable and to reflect a significant viewpoint. But sources that are not authoritative can be used to reflect the opinion of the author rather than as fact. specificity and depth scholarly sources are better than news reports. Articles that deal in depth with specific studies are better than general articles verified not for passing on rumors and gossip circular sourcing using sources that use Wikipedia as their own source context whether a specific story is reliable for a specific fact or statement in a Wikipedia article will be assessed on a case by case basis reputation Primary, secondary and tertiary sources
One way of identifying whether a source is reliable or not is to determine the viewpoint of the source. Wikipedia policies values secondary sources more highly than primary or tertiary sources stating that articles need to be based largely on secondary sources. Secondary sources are third-party sources, one step removed (whether by time or location from an event), whereas primary sources are from eyewitness accounts directly involved in an event. Primary and secondary sources originated in history studies where primary sources were used as artifacts to understand on-the-ground perspectives from a particular time. According to policy, tertiary sources can be used to support broad summaries of topics especially when secondary sources contradict one another. This division of sources can be complicated, though, since journalism can fall into either the secondary source if it provides in-depth analysis but a primary source if it is opinion or eyewitness accounts. Also, Wikipedia can be used as a tertiary source for an article outside of Wikipedia about a topic but it is a primary source in an article about Wikipedia. It seems that there is resistance among many Wikipedians to ever using primary sources, but as the authors of this essay write, "Primary" is not another way to spell "bad"): Primary sources may only be used on Wikipedia to make straightforward, descriptive statements that any educated personwith access to the source but without specialist knowledge will be able to verify are directly supported by the source. This person does not have to be able to determine that the material in
20
the article or in the primary source is True. The goal is only that the person could compare the primary source with the material in the Wikipedia article, and agree that the primary source actually, directly says just what we're saying it does. (Wikipedia: Identifying and using primary and secondary sources) It is therefore the context of the information that determines whether a source is reliable or not, a feature that becomes important when considering design options.
B. Sources on the ground: the case of the 2011 Egyptian revolution article

Background In order to understand how sources are being managed in rapidly evolving articles, I analysed the 2011 Egyptian revolution Wikipedia article on the English Wikipedia. Studying talk pages, edits and others documents, I pieced together conversations, versions and debates from the 25th of January when the article was first created (within hours of the first protests in Tahrir Square), to the 12th of February when Mubarak resigned and the article name was changed from 2011 Egyptian protests to 2011 Egyptian revolution. In these 18 days, editors from the United States, Egypt, Western Europe, the Middle East and elsewhere worked together to build an accurate account of the historic events taking place in Egypt as they happened. Because events were unfolding rapidly, few secondary sources were available to analyse the significance of particular events, or to reflect on which events were most significant. The majority of sources were primary source news accounts by journalists or citizen journalists in the field, or else summarized reports from primary accounts. Because Wikipedians must reflect only what reliable sources (preferably secondary sources) are saying about a particular subject, the challenge became about keeping track of the rapidly evolving news cycle and doing so accurately in one of the most reported global events in the English media in recent years. a) What motivated participation? The PEJ (Pew Research Centers Project for Excellence in Journalism) News Coverage Index indicated that coverage of the Egyptian unrest during the 31 January to 6 February registered as the biggest international story in the past four yearssurpassing any coverage of the Iraq war, the Haiti earthquake and the conflict in Afghanistan (Jurkowitz, 2011). The story was compelling for journalists for a number of reasons that Jurkowitz attempts to explain: One major reason is the number of cameras and journalists (including network anchors) in the country transmitting such riveting scenes as last weeks video of men on camels attacking crowds of protestors in Cairo. Another is the high stakes for the U.S. in one of the worlds most volatile regions as it tries to balance a strategic alliance with President Hosni Mubarak and support of pro- democracy protestors. A third factor may be uncertaintywill Mubarak resign and who will govern after him? And some of last
22
weeks coverage was driven by the fact that the media themselves became part of the storywith journalists being harassed, attacked and detained amid the chaos. Wikipedia editing spikes generally tend to parallel media coverage, but there were two additional motivations reflected in interviews and talk pages that were particular to the Wikipedia community. The first was the unprecedented shut down of the Internet by the Egyptian government, inciting Wikipedia editors who use the Internet and often take part in activities to safeguard its freedoms. In the same way that having journalists attacked made the story more compelling for the media, the fact that the lifeblood of Wikipedia, the Internet, was under attack, made this a more compelling story for Wikipedians. Secondly, some of the key editors who had friends and family in Egypt or who had spent time in Egypt said that editing the article was a way of being involved in this historic occasion and of representing the protesters stories. According to the Wikipedia editor who originally started the page, The Egyptian Liberal: I think I owe the people who are protesting and those who have passed away to tell their story from a NPOV. I am not nor can I be in Egypt due my university so that my way of standing up to Mubarak and tell him I shall not be silenced and I shall tell the world what happened. (2011 Egyptian Revolution: Talk page) The need to be involved in the historic retelling of events motivated hundreds of editors to participate in the editing of the page, with a few editors working night and day to manage the overall direction of the article and its forks as events evolved. When Mubarak finally resigned on the 12th of February, 2011, scores of editors attempted to change the name of the article from protests to revolution before consensus had been reached on the Talk page. According to one of the top editors of the page, User:Aude, this was because Someone always wants to be the first to do that (change the name). (Interview, 4 May, 2012) Having ones username permanently attached to the edit that changed the name to revolution seemed to be a significant incentive for editing, even before general consensus had been reached. b) Who edited the page? Hundreds of editors made edits to the 2011 Egyptian Revolution article, especially in the first days of the protests. Although we are unable to see where all edits originated, we can find out where anonymous users were editing from since their IP addresses are public. Looking up the article on WikiTrip (http://sonetlab.fbk.eu/wikitrip/#|en|2011_Egyptian_Revolution showing that the top editors), an application created by Paolo Massa and others, we can see that the majority of users come from the United States, with significant edits originating from Poland1, Egypt, the United Kingdom and Canada. We can also see that only 0.05% of (anonymous) editors of the page indicated that
1
Probably from the single user from Poland using an IP address who made significant edits before he was banned
they were female and that the majority of edits took place from late January (the start of the revolution was January 25) to early February.
Although hundreds of editors made edits to the main article, only a handful of editors stuck with the page for extended periods of time. These editors were essential for steering the overall direction of the page, since the majority of editors tended to focus on specific details without recognizing factors like article size and length that placed limits on the level of detail that could be accommodated in the main article. These editors also decided when and where to fork the article and weighed in on the Talk page when disputes arose. The Egyptian Liberal, the user who started the article, was the most prolific editor by a significant majority. This user played a critical role in the growth and development of the article in both the English and Arabic versions. From his discussions on the Talk page, it appears that he is an Egyptian living outside the country and while he strongly supported the revolutionaries, he attempted to retain his neutral point of view and ask others to edit sections when he felt that his view was compromised (see an alternative perspective by Kamir, 2011).
24
Responding to requests for an interview, he wrote that he does not do interviews, preferring to retain his anonymity on the Wikipedia platform. Other top editors of the article, such as User:Lihaas, Ocaasi and Wipsenade left the page soon after Mubarak resigned and the articles name changed, while anonymous user: 94.246.150.68 (a Polish IP address) was banned during his tenure on the article. c) How the page evolved The first article (originally called 2011 Egyptian protests) soon forked to provide more detailed information about what happened on each of the first days of the protests, responses to the revolution by different countries, and later, what happened under the rule of the armed forces and the subsequent elections.
New articles branching out from the original Branching out is defined here as articles that started life as part of the 2011 Egyptian revolution article but were later started as their own page. These pages include: http://en.wikipedia.org/wiki/International_reactions_to_the_2011_Egyptian_revolution http://en.wikipedia.org/wiki/Timeline_of_the_2011_Egyptian_revolution http://en.wikipedia.org/wiki/Timeline_of_the_2011_Egyptian_revolution_under_Hosni_Mubarak%27s_rule http://en.wikipedia.org/wiki/Timeline_of_the_2011_Egyptian_revolution_under_Supreme_Council_of_the_Armed_Forces http://en.wikipedia.org/wiki/Domestic_responses_to_the_Egyptian_Revolution_of_2011 http://en.wikipedia.org/wiki/Egyptian_constitutional_review_committee_of_2011 http://en.wikipedia.org/wiki/Trials_and_judicial_hearings_following_the_2011_Egyptian_revolution http://en.wikipedia.org/wiki/Egyptian_constitutional_referendum,_2011 http://en.wikipedia.org/wiki/Egyptian_presidential_election,_2011
New articles related to the revolution that were significantly improved I define new articles here as articles that are related to the 2011 Egyptian revolution page and that, although their subjects were mentioned in the article, werent necessarily significant sections of the original but were started because the article was getting too big and/or because the subject was significant enough to warrant its own page. These include: http://en.wikipedia.org/wiki/Muslim_Brotherhood_in_Egypt http://en.wikipedia.org/wiki/We_are_all_Khaled_Said http://en.wikipedia.org/wiki/Wael_Ghonim http://en.wikipedia.org/wiki/Asmaa_Mahfouz http://en.wikipedia.org/wiki/Sharif_Abdel_Kouddous http://en.wikipedia.org/wiki/Coalition_of_the_Youth_of_the_Revolution http://en.wikipedia.org/wiki/Revolutionary_Socialists http://en.wikipedia.org/wiki/Ghad_El-Thawra_Party http://en.wikipedia.org/wiki/Dignity_Party_(Egypt) http://en.wikipedia.org/wiki/Essam_Sharaf
26
http://en.wikipedia.org/wiki/Hossam_el-Hamalawy http://en.wikipedia.org/wiki/Al-Wasat_Party http://en.wikipedia.org/wiki/2011_Israeli_embassy_attack http://en.wikipedia.org/wiki/Maspero_demonstrations
c) What were the main challenges faced by editors with regard to sources? Editors faced four key challenges as they attempted to edit the rapidly evolving article: 1. Collectively keeping track of the news cycle using available tools; 2. Deciding which photographs and video to use with policy that was designed for text rather than multimedia; 3. Deciding whether to use primary or secondary sources; 4. Citing online sources 1. Collectively keeping track of the news cycle using available tools Editors often separated the tasks of editing an article and searching for news. Edits to the article were performed after news search and consumption for two reasons: either because the editor didnt have the time/inclination to edit the page while they were consuming news, and/or because they were waiting for other sources to verify the information contained in a single source. The mechanism used by editors to solve this problem was to attach the news article with a note on the talk page as a kind of heads up for others. The note functioned as: a) a way to alert other editors to recently-released information relevant to the article (important especially since editors were working in different time zones and the talk page would be one of the first places a collaborating editor would go when they woke up or starting editing); b) a request for others to add the source to the article for them if they didnt have the time or the access privileges, for example, if they were editing as an anonymous IP during the periods when the page was semi-protected (some editors saw the discovery of sources and posting to Talk as their single task rather than editing); c) a place to hold sources while they were waiting for other sources or were unsure as to the relevance of the article for inclusion and were asking for opinions of other editors; d) a way to ask other editors to find references for unsourced or incorrectly sourced information on the article.
In the example to the left, SilverSerenC alerts editors to the release of a new article by Reuters. He summarizes the articles relevance to the article, highlighting the reference to the armys allegiance to the protesters and the demands by the US for Mubarak to end the Emergency Law. Twenty minutes later, SilverSerenC adds another reference, this time to a Guardian report on a protest being planned for the next day. Lihaas responds with thanks to SilverserenC, adding a checkmark that he has Done the edit. In this case, he adds a phrase to reiterate the armys pledge not to use force against citizens (as highlighted below) but does not specify on the Talk page exactly which edit has been done. Looking at the actual edits on the page, we see Lihaass addition, shaded in blue below, as well as the edit summary that refers back to the Talk page.
Example of news alert with just-released important information:
Revision as of 23:10, 31 January, 2011 by Lihaas: The top Egyptian generals, led by Tantawi, held back the military,<ref>{{Cite web|url=http://www.couriermail.com.au/ipad/nationlocked-in-a-deadly-stalemate/story-fn6ck51p1225997761161|title=Nation locked in a deadly stalemate}}</ref> as the army as a whole again pledged not to use force on the citizens of Egypt and said they had a legitimate cause for their rioting.<Ref>http://www.reuters.com/article/2011/01/ 31/idINIndia-54544320110131</ref> Edit summary: (ref from talk --> tv sources WILL come..)
Keeping track
Other editors employed their user pages to keep track of the latest news and categorize articles for later editing. User:Aude
A section of links from Audes Egypt page http://en.wikipedia.org/wiki/User:Aude/Egypt
28
employed one of her user pages to paste citations for relevant articles, photographs, videos etc related to the article as they happened. In an interview, she said that did this because she would be reading the news, rather than editing, and wanted a quick way to store relevant articles for later editing. In the figure to the right is a section of the page containing links related to the Egyptian army that she was storing during the first two weeks of the protests. 2. Deciding which photographs and video to use Criteria for reliable sources didnt work very well for images and video, and editors had to develop a highly sophisticated understanding of bias as related to the placing, representation and choice of image and videos in the article. Video and image sources are different from text-based sources for a number of reasons. The first is that they need to be represented in whole, rather than an excerpt and the policies and laws around using video and image sources as references are not as well defined as textual excerpts. Images and video are more often sourced from social media or unaffiliated individuals rather than from traditionally published reliable sources. This is because, unlike text, video and images must be licensed under a Creative Commons Attribution Share- Alike license in order to be used on the page. Wikipedias policy on multimedia seeks to ensure that copies can be freely and legally made of the articles and that others may republish them. But if images or multimedia contain no license, even if they are meant for republication (such as the raw video footage from protesters uploaded to YouTube) Wikipedia editors must try to contact the authors and ask if they might license the work under CC. This can be a time-consuming (sometimes impossible) task, which is why repositories like Flickr (that explicitly ask uploaders if they want to license their content under CC) are more popular. In the case of the 2011 Egyptian Revolution article, User: Aude was in touch with Al Jazeerah who agreed to license their images and video under a freer CC license than their default so that content could be integrated into the article. She said that Al Jazeerah was receptive of their requests and acted quickly to free up the photographic and video content useful to Wikipedia. But editors still needed to choose from the multitude of photographs and video. Particular challenges relating to images and video were deciding how to present political cartoons and videos from a neutral point of view. a) Presenting video with NPOV An Al Jazeera English video (http://en.wikipedia.org/wiki/File:Al_jazeera_2011_egypt_protests.ogv) was embedded in the article next to the 28th of January section of the timeline (see screenshot to the left captured from the 06:46 UTC 29 January version of the article). The video shows live footage and
commentary from Al Jazeera journalists who were trying to avoid tear gas to bring the images to viewers. The video shows images of protesters gathering outside the Hilton Hotel in downtown Cairo, then tear gas being thrown at them to disperse them while Al Jazeera journalists relate what is happening from a nearby building. On the 29th of January, User: Peter G Werner complains to editors about the video with specific reference to its being embedded in the article, rather than being used as an article reference (i.e. externally linked). He notes that this raises all kinds of NPOV issues (2011 Egyptian Revolution Talk page: 06:49, 29 January 2011 UTC). Ocaasi defends the video writing that it is merely descriptive of local surroundings and events and not editorializing at all and claims that Al Jazeera English is dead center politically (relative to the New York Times and the Egyptian government) (2011 Egyptian Revolution Talk page: 09:04, 29 January 2011 UTC). User: James (T C) writes that It is quite clear who is speaking (not us, them) (09:32) and User: Geni writes the last note in the conversation declaring that Its the only video coverage we have of events and that its under a free license so you can edit the sound/chop up the visual if you want (17:15). None of these responses really get to the heart of Werners complaint, however. It seems that the majority of editors feel that the video merely represents the facts of the situation (as Ocaasi writes: it is merely descriptive) but media scholars would note that camera angles, commentary and a variety of choices made by the journalists mean that no image or video is merely descriptive. In this case, the perspective of the camera is on the protesters rather than the (unseen) forces throwing tear gas canisters. Additionally, were hearing from the journalists who are portrayed as victims of the tear gas, thus eliciting sympathy from the viewer, and although most of the commentary merely relates what is happening in the images and providing context as to where the images are being captured, the narrating journalist invites viewers to empathize with the protesters rather than the government: As you can imagine the tear gas is making it very difficult for us to see You can imagine what it's like for the protesters I mean, we are several dozen feet above the street level from our studio you can see these pictures and now that area has kind of been dispersed as a result of the tear gas. You can see some of the people at the hotel trying to run... Ocaasis comments represent a common misrepresentation of the reliability of sources with his comment that Al Jazeera English is dead center politically, probably because he feels he needs to defend against a general bias against Al Jazeera by many editors. But in doing so, he ignores Werners complaint: Werner isnt complaining that Al Jazeera is biased, hes complaining that the placing of the reference inside the article, almost as a statement in the same voice as the text of the article, is POV. James (T C) does respond to this particular complaint, saying that it is quite clear who is speaking (not us, them) especially since the caption on the video does make it clear whose perspective is being showcased. On the other hand, there are no further sources used in the text of the article to verify what is pictured in this video as occurring around the Hilton Hotel on the 28th.
30
Understanding how the use of video relates to NPOV and verifiability is a challenging issue for editors because it requires new ways of framing and verifying information that differ from textual mechanisms. b) Presenting illustrations with NPOV Knowledgekid87 nominates the Egyptian revolution article for a POV check (which involves a warning tag being placed on the page) because he says that both cartoons used on the page side(d) with protesters (00:26 UTC, 29 January 2011). Others weigh in, debating how to more accurately reflect an NPOV.
Figure 7 Original selection of the page nominated for a POV check by Knowledgekid87 at 00:28, 29 January, 2011
Others make suggestions to resolve the issue or extend the critique: a) Ocaasi suggests (perhaps sarcastically) adding a political cartoon that sides with the government. Ocaasi writes: Political cartoons don't usually side with oppressive regimes. Do you have one that does? We can add it... b) He then suggests Mov(ing) the cartoons to the reception section rather than as illustration for the factual content. The reception section of the article was where editors added summaries of responses to the protests in Egypt and internationally. c) Peter G Werner weighs in to add that justification (needs to) be given that these cartoons are newsworthy in themselves, clearly representative of world reaction. d) The Egyptian Liberal responds with images showing that (Latuffs) cartoons (are) being held by the protesters during their marches. e) Peter G Werner suggests using one of the images of Latuffs cartoons held by protestors instead of the one in the article and adding text that put(s) it in(to) context. f) One the cartoons is subsequently removed.
Figure 8 Latuff cartoon moved from the Protests section to the Domestic responses: Media censorship section (Egyptian revolution article as at 00:56, 31 January 2011) as a result of the POV check
32
Figure 9 Khaled Mohamed Saeed cartoon is removed and caption is edited to include more context.
Figure 10 Figures above showing the 00:28, 29 January 2011 (POV check) and the 00:56, 31 January 2011 versions of the photo captions after the POV check
This example shows that POV checks may be useful for improving content. Editors are incentivized here to review all the content of the page in order to have the POV tag removed so that they are able to prove the quality of the article against one of the most fundamental Wikipedia policies, NPOV. We can also see here that multimedia may require special NPOV guidelines because the context and placement of images or video is critical to its neutrality, unlike textual information. 3. Deciding whether to use primary or secondary sources At 04:32 UTC on the 28th of January, an anonymous user asks other editors to add Joe Bidens comments on Mubarak as reported in the Christian Science Monitor [1] to the article. Eight minutes later, The Egyptian Liberal has summarized the story from the Christian Science Monitor article and added his summary it to the International reactions section as US Vice President Joseph Biden said "...were encouraging the government to act responsibly and and to try to engage in a discussion as to what the legitimate claims being made are, if they are, and try to work them out.", as well as choosing not to refer Mubarak as a dictator and that he should not step down [1] The Egyptian Liberal marks the task as Done on the Talk page. The next day, Abrazame alters the text of the sentence, removing the phrase: as well as choosing not to refer Mubarak as a dictator and that he should not step down and replacing the quote with the following: US Vice President Joseph Biden said Hosni Mubarak "has been an ally of ours in a number of things. And hes been very responsible on, relative to geopolitical interest in the region, the Middle East peace efforts: the actions Egypt has taken relative to normalizing relationship with Israel."[1] In the actual transcript of the interview this line is followed by the words: And I think that it would be -- I would not refer to him as a dictator2.
2 Interview transcript: JIM LEHRER: The word -- the word to describe the leadership of Mubarak and Egypt and also in Tunisia before was dictator. Should Mubarak be seen as a dictator? JOE BIDEN: Look, Mubarak has been an ally of ours in a number of things and he's been very responsible on, relative to geopolitical interests in the region: Middle East peace efforts, the actions Egypt has taken relative to normalizing the relationship with Israel.
34
But Abrazame writes that the Christian Science Monitors headline Joe Biden says Egypt's Mubarak no dictator, he shouldn't step down... is a twisting of Bidens words and that such U.S.-partisan baiting does not belong in this article. Abrazame contends on the Talk page that the actual conversation with Biden was very different from the way it was presented in the headline. User:Missionary disagrees on the Talk page, writing that the story made headlines (in) several unrelated reliable sources pointing to three articles from CBS News (Joe Biden: "Speaker Boehner and I are Friends http://www.cbsnews.com/8301-503544_162-20029872-503544.html), Fox News (Biden: Mubarak should not step down Current (10 May, 2012) citation to a section of the original http://www.foxnews.com/politics/2011/01/28/biden-mubarak-step/) PBS NewsHour interview video from w here the quotes came and the Malaysian Sun (U.S. comes to defense of Egyptian government, http://www.pbs.org/newshour/rundown/2011/01/exclusive-biden- http://story.malaysiasun.com/index.php/ct/9/cid/b8de8e630faf3631/i discusses-unrest-in-egypt-keeping-us-competitive.html d/737115/cs/1/ht/US-comes-to-defence-of-Egyptian-government/), and linking to the transcript of the interview on PBS news (http://www.pbs.org/newshour/bb/politics/jan-june11/biden_01- 27.html). Abrazame replies in an unusually long post that Wikipedia editors should not be swayed by provocative headline writers who have other agendas and that by using this twisted version they would be committing the sin of POV (point of view). Surely you know that headline writers are there to be provocative in order get hits and/or sell copy, and that syndicates and the blogosphere thrives on such provocativeness, preferring to bandy about the broadest and bluntest conclusion-jumping if it fits into their "storyline", rather than actually read and comprehend the words and their context. He also writes that Using this article to amplify this aspect of the VP's interview seems POV and that such a provocative statement should have more than one misreading of a source no matter how many sites mirror this amplified misinterpretation to support it.
And I think that it would be -- I would not refer to him as a dictator. (From PBS.org http://www.pbs.org/newshour/bb/politics/jan-june11/biden_01- 27.html)
32 minutes later, the citation is revised by Sergeo who replaces the Christian Science Monitor article with a link to a video [3] of a section of the interview (not the section that is quoted in the article) with an edit summary claiming that the PBS cite is a Direct source and that it was done for neutrality reasons. Sergeo uses the term direct source perhaps because he would rather not use the less acceptable primary source term. Interestingly, the PBS citation contains only a selection of the entire interview video and does not contain the selection quoted. Sergeo uses the term direct source but the original, full-length source (in which the quote that is used occurs) is located at another URL (http://www.pbs.org/newshour/bb/politics/jan-june11/biden_01-27.html and http://www.youtube.com/watch?feature=player_embedded&v=N-_1FMTMzRg) and a more direct source would be the actual line used in the context of the entire interview. It seems that Wikipedias current policies on sources leave questions relating to appropriate sources here. In terms of the verifiability principle, it seems more appropriate to link to the actual video of the interview so that users are able to check up on what Biden actually said, as well as the context in which it was said. But the verifiability principle also indicates that Wikipedia should merely reflect what reliable sources say about an event rather than the truth of what actually happened. None of the numerous articles published from reliable sources on both sides of the partisan divide about the speech leave out Bidens words: I would not refer to him as a dictator as the Wikipedia article does and it seems that the primary source is being used to prevent having to refer to any secondary source that would contain this information. The question becomes, what should Wikipedians do when news sources twist the truth because of skewed incentives and minimal resources that tend to see other sources merely mirroring these incorrect versions? It may seem better to be able to go directly to the primary source, especially when this is increasingly possible, but the problem is that in many cases, Wikipedians must summarize the primary source and in doing so do not always reflect what the majority of secondary sources are highlighting. This may be a particular problem of news media since news media is particularly subject to partisan bias, recentism and the influences of advertisers and investors, but it is becoming a more common problem. In the end, Abrazame and Sergeo seem to have won this round because the paragraph wasnt edited again, nor was Abrazames (very long) Talk post responded to (the sentence is now on the split off page entitled: International reactions to the 2011 Egyptian revolution (http://en.wikipedia.org/wiki/International_reactions_to_the_2011_Egyptian_revolution). Someone could edit the page now, but it is well known that timing (and consequently the attention that such timing garners) is the most important variable in these stories. Another issue raised by this example is the question of when an editor needs to reach consensus on the Talk page before editing. There are numerous examples where editors revert and reinstate edits without discussion on the Talk page, but here it seems that Abrazame is attempting to use the Talk page as a way to drive his agenda with long speeches -- perhaps too long for editors to read and challenge. Abrazame himself edits the page before making his critique of the previous edit but then reverts Missionarys edit because he writes that he should discuss it on the talk page rather than just reverting. Since Abrazame has the last word on the talk page about the issue, it
36
seems that it has been settled in his favor, when it could be that Missionary has given up due to the length of Abrazames posts and his obvious determination on the topic. It may seem too much of a battle to fight and so there is little oversight. Citations (as they appear/ed in the article): 1: Murphy, Dan (January 26, 2011). "Joe Biden says Egypt's Mubarak no dictator, he shouldn't step down". The Christian Science Monitor. Retrieved January 28, 2011. 2: Biden: Mubarak Should Not Step Down* 3: Murphy, Dan** (January 26, 2011). "Exclusive: Biden Discusses Unrest in Egypt, Keeping U.S. Competitive". PBS. Retrieved January 28, 2011. * The above source is from Fox News.com. Interesting that it is not included. There is significant opposition to using Fox News as a Reliable Source so it may be that the editor is trying to bury the reference. ** It is incorrect that Dan Murphy is the author. Dan Murphy was the author of the Christian Science Monitor article but doesnt appear in the PBS video. Probably an edit error when the citation was changed.
Time Editor 28 January 2011 04:32 76.124.12.112 Edit Edit summary Talk post Notes
Joe Biden's comments on Mubarak and legitimacy of protestors Please add his comments as described in CS Monitor http://www.csmonitor.com/World/Backchannels/2011/012 7/Joe-Biden-says-Egypt-s-Mubarak-no-dictator-he-shouldn-t- step-down
Anonymous user asks editors to add Bidens comments to the article and references the Christian Science Monitor piece [1].
04:39 04:41
The Egyptian Liberal
04:39: US Vice President Joseph Biden said "...were encouraging the government to act
none
04:41:
Done
The Egyptian Liberal summarizes the story from the Christian Science Monitor article
responsibly and and to try to engage in a discussion as to what the legitimate claims being made are, if they are, and try to work them out.", as well as choosing not to refer Mubarak as a dictator and that he should not step down [1] 29 January 2011 02:36- Abrazame 02:40
and adds it to the International reactions section after anonymous user asks for editors to add the information on the Talk page
02:36: US Vice President Joseph Biden said Hosni Mubarak "has been an ally of ours in a number of things. And hes been very responsible on, relative to geopolitical interest in the region, the Middle East peace efforts: the actions Egypt has taken relative to normalizing relationship with Israel."[1]
International: C overage of this statement takes out of context and overstates diplomatic refusal to let an interviewer put words into his mouth
02:51
Missionary
US Vice President Joseph
Undid revision
02:40: His comments "as described in the Christian Science Monitor" are a partisan POV twisting of his words, and such U.S.-partisan baiting does not belong in this article. Biden was asked two or three days ago, "if the time has "come for President Mubarak of Egypt to go?" Biden answered: "No. I think the time has come for President Mubarak to begin to move in the direction that to be more responsive to some... of the needs of the people out there." This is not the same as the title of that smear piece characterizes, that Biden says he "shouldn't" step down. At the time, Obama had not personally made an official comment; now that he has, the CSM's twisting of the Vice President's words, which was never in proper context, are even less necessary or relevant. Responding to questions about "would you say this, and would you say that" as "no, that's not what I would say" doesn't mean you don't think those things, it simply means the Vice President isn't falling into the trap of letting people put words in his mouth at a time that requires diplomacy. That the CSM would put the opposite of those words in his mouth is no more relevant or appropriate to parade as if it were the Vice President's opinion. What is relevant to this situation is the desire for continuity in peace treaties that Mubarak has adhered to as a U.S. ally, so rather than removing the section again in its entirety, I have left in the part that speaks to that legitimately relevant point. Abrazame (talk) 02:40, 29 January 2011 (UTC)
Abrazame alters the text, removing the phrase: as well as choosing not to refer Mubarak as a dictator and that he should not step down and replacing the quote with another.
Missionary undoes
38
03:03
03:33
Missionary
Biden said Hosni Mubarak should not step down, and that "Mubarak has been an ally of ours in a number of things. And hes been very responsible on, relative to geopolitical interest in the region, the Middle East peace efforts; the actions Egypt has taken relative to normalizing relationship with with Israel. ... I would not refer to him as a dictator."[1][2] US Vice President Joseph Biden said Hosni Mubarak "has been an ally of ours in a number of things. And hes been very responsible on, relative to geopolitical interest in the region, the Middle East peace efforts: the actions Egypt has taken relative to normalizing relationship with Israel."[1]
410682826 by Abrazame (ta lk) Edit goes against the unanimous opinion of Reliable Sources
Abrazames revision because he writes that it goes against the unanimous opinion of Reliable Sources and adds the phrase: ... I would not refer to him as a dictator. to the quote from Biden.
Undid revision 410684863 by Missionary (t alk) Please discuss this on the talk page rather than just reverting POV into the article
Abrazame reverts it back to his edit and asks Missionary to discuss this on the talk page rather than just reverting POV.
Disagree This is not a smear piece, it made headlines of several unrelated reliable sources([2][3][4]) It really is important as a reflection of the US view towards Middle East authoritarian regimes from a high authority. There is no broader context to be had here (see transcript), Biden really meant to express his belief that Mubarak is not a dictator and that he should not step down. Surely you know that headline writers are there to be provocative in order get hits and/or sell copy, and that syndicates and the blogosphere thrives on such provocativeness, preferring to bandy about the broadest and bluntest conclusion-jumping if it fits into their "storyline",
04:08
Abrazame
rather than actually read and comprehend the words and their context. It is not relevant to this developing story about Egypt that the Vice President of the U.S. doesn't want interviewers putting words into his mouth. Biden is not asked if Mubarak is a dictator, he is asked if Mubarak should be seen as a dictator, and what Biden replies is that he "would not refer to him as a dictator" (emphasis is mine). Biden does not say that Mubarak "is not" a dictator, it is the headline writer who says that Biden has said this. It is not relevant, then, whether the news agency mistitles it as such, nor that Fox News and the Malaysian Sun run with the headline that claims Biden says something that he does not, particularly when we have the actual transcript to show us this. But this is not merely an argument about semantics, it is one about diplomacy, one that Biden chose to approach diplomatically given the sensitivity of the relationship, given the sensitivity of the then-nascent situation there, and given that he didn't want to get out in front of a story by making clumsily bellicose characterizations of allies. The fact that the U.S. considers Egypt an ally (and a primary reason the U.S. considers Egypt an ally) is arguably relevant to this article (though arguably no less relevant to others who appreciate and wish to maintain the peace Egypt made with Israel), so I thought a good compromise was to restore that aspect of his comments. Would you argue that the use of the term "dictator" is more relevant than the underpinnings of this relationship? It strikes me that this is the approach of a tabloid, rather than that of an encyclopedia. I may be wrong, but I'm not aware of either President Obama or Secretary of State Clinton saying that they would "refer" to him as a dictator (or, indeed, doing so), and the point is that this is what diplomacy is all about. All of them seem to be on the same page that they would like the outcome of this to be decided in a peaceful way by the Egyptian people without the U.S. calling for "dictators" to "step down" as they move toward elections to choose a new leader going forward. It is implicit that leaders who care about preserving lives, much less leaders whose countries have treaties and trade pacts with another country, would prefer a peaceful succession of
40
04:40
Sageo
International: Di rect source - neutrality
government rather than a bloody upheaval that throws all of their mutual interests into question. Stating publicly that you're not going to use disparaging characterizations against such an ally, given that this ally has indicated he will hold such elections (thereby implying that he may be ready to stop being whatever sort of leader one may characterize him as), is a diplomatically appropriate choice. Using this article to amplify this aspect of the VP's interview seems POV. Doing so because it is your misperception that Biden thinks that Mubarak should remain in power for the foreseeable future is a misinterpretation of the interview, one that is fueled by the influence of the POV headlines and not by a nuanced reading and digestion of the actual words and their context, and shows a naive misunderstanding of the responsibility of speech by U.S. leaders. If they say one thing, they may be accused of fomenting another country's civil war or even implying tactical support for such. (Remember Iran not long ago?) If they say another thing, they may be accused of backing a leader over his people. I don't think that any responsible reading of this interview takes away that Biden is saying the latter. Put another way, as it seems to be your assertion that the U.S. wants Mubarak to remain in power rather than hold those elections and have a peaceful transfer of power, or for Mubarak to be unyielding (dictatorial) to the demands of his people for however long he might hold onto his position in advance of such an election, can you cite any other reliably sourced, non-POV-twisting statement to support that? Because selecting this one aspect to misquote out of context is an irresponsible amplification of a position, that Biden and by extension the U.S. government actually wants Mubarak to stay on, and such a provocative statement should have more than one misreading of a source no matter how many sites mirror this amplified misinterpretation to support it. Abrazame (talk) 04:08, 29 January 2011 (UTC)
Changes the citation reference to PBS [3]
4. Citing online sources In the excerpt below, you can find an example of an editor bringing an incomplete source to the talk page. Here, Ocaasi makes a new section on the talk page entitled NEWS DESK (sourcing and verification) and asks editors to Use this section to post requests for sources, statements needing verification. (Talk:2011_Egyptian_revolution/Archive_3)
Reference request:
In this example, Ocaasi notes that the opposition leader has not been named and that a traceable source needs to be provided. The editor who had made the edit was probably watching Al Jazeerahs live news coverage as many were during the events. Other editors respond with inconclusive replies perhaps because Ocaasi hasnt explicitly asked for what he is looking for. About 11 hours later, Silverseren finds an online source referring to the Al Jazeera report and another editor, Lihaas, edits the reference in the article, concluding the collaborative turn.
Interestingly, the text of the article referred to Al Jazeeras report and did not mention the opposition leaders name. The article was from the blogs section of the LA Times and a large logo of the Carnegie Middle East Center on the page indicated that it was written by the staff of the LA Times and the Carnegie Middle East Center. Below youll find the article with phrases referring to other news media reports in bold red. Also notice the original URL probably from an earlier headline that specified the source of the information (i.e. Al Jazeera). EGYPT: Opposition plans to negotiate with military, not president January 30, 2011 | 1:34 pm LA Times
42
http://latimesblogs.latimes.com/babylonbeyond/2011/01/arab-satelite-network-al-jazeera-has-reported-that-egyptian- opposition-leaders-say-that-their-negotiations-for-a-transition.html Arab satellite network Al Jazeera has reported that Egyptian opposition leaders plan to negotiate a transition government with the Egyptian army, which maintained a heavy presence throughout the country on Sunday, and not longtime President Hosni Mubarak. Al Jazeera reporters blogged that protesters still crowded Cairo's Tahrir Square after 11 p.m. Sunday, calling for Mubarak to resign. At one point, protesters spelled out "Down with Mubarak" with their bodies, the network reported. Heavy machine-gun fire could be overheard Sunday night as thousands of protesters marched through downtown Alexandria, CNN's Nic Robertson reported. Army troops were positioned in various parts of the port city, having moved some of their checkpoints over the weekend. Maajid Nawaz, executive director of the Quilliam Foundation, a London-based think tank, told Al Jazeera that Mubarak was a liability to political allies within Egypt and overseas. (An image of the article in the figure to the left) Lihaas added the reference to the article as seen below but he only added the URL, rather than the full citation. The citation was expanded the next day but still contained the question of exactly which opposition leader made the statement. As at 15 May, 2012 the citation had been cleaned up to contain the title of the article, the publication and the retrieval date but is missing important information necessary to understanding the fact that the original reporting was done by Al Jazeera and not the LA Times and even that secondhand
reporting was done by bloggers rather than staff of the LA Times. Original phrase (brought to the talk page by Ocaasi at 09:01, 31 January, 2012): An opposition leader[who?] said that talks would not be held with Mubarak but only with the army.[113] Revision as of 21:34, 31 January 2011 by Lihaas (adding the ref URL) An opposition leader{{Who|date=January 2011}} said that talks would not be held with Mubarak but only with the army.<ref>http://latimesblogs.latimes.com/babylonbeyond/2011/01/arab-satelite-network-al-jazeera-has-reported-that- egyptian-opposition-leaders-say-that-their-negotiations-for-a-transition.html</ref> Later the next day, changed to: ^ "EGYPT: Opposition plans to negotiate with military, not president | Babylon & Beyond | Los Angeles Times". Latimesblogs.latimes.com. Retrieved 2011-01-31. And as of 15 May, 2012 the text reads: Egyptian opposition leaders said that talks would be held only with the army.[84] "Egypt: Opposition Plans To Negotiate with Military, Not President". Los Angeles Times. Retrieved 31 January 2011. The convention of Headline title + publisher + retrieval date (sometimes preceded by the author of the article if known) is used as the format for the majority of references. Editors may use citation templates (see below) to insert appropriate information and format the citation correctly but the templates do not enable editors to reflect complex authorship of forms such as newspaper weblogs where the content is often written by authors outside of the publications core staff or where it may contain information that is unverified (due to the
44
Figure 11 Web citation template and news citation template
Figure 12 Different source templates
d) How was information verified? Information was verified by editors in a number of ways, from simply checking the source to see whether it was accurately reflected in the article text, to counting sources that named events differently. 1. Avoiding single sources When editors brought breaking news to the Talk page, some continuously called for refraining from using the sources to edit the page when only a single news channel was reporting the information. On the 28th of January, Kuzwa brings an update to the Talk page writing that Wikileaks has released a new cable which claims that America has been secretly supporting the pro-democracy movement in Egypt since at least 2008. Also, it looks like major democratic reforms in Egypt were planned for 2011 through some sort of plot. Not sure if this plot counted in Tunisia's collapse or not. See here: [5] --Kuzwa (talk) 21:25, 28 January 2011 (UTC) Ocaasi replies: Think we should wait a bit on it for confirming sources. Nothing against the Vancouver Sun but this should be reported in multiple places if it's accurate and notable. Ocaasi (talk) 22:21, 28 January 2011 (UTC) A few minutes later, other editors discover that the article cited does not refer to the USs support of Egypt. 2. Wait and see This phrase became a constant refrain among edits on the Talk page of the article. Editors would bring breaking news to the page, and when there was only a single source available to verify the information, editors would ask others to wait and see whether other news outlets were saying the same thing. Dont do too much synthesis until reliable secondary sources have reviewed the events and ascertained which are the most important. After some editors call for a split of the timeline section of the article on 3 February, User:Wnt replies that he hopes secondary sources will review and prioritize/summarize events so that editors could then reflect that summary in future versions of the article. Im hoping some secondary source will review events so far, and provide a good outline to follow in terms of how the development of events unfolded. Wnt (talk) 05:43, 3 February 2011 (UTC) Wnts comment reflects one of the problematic issues involving the writing of encyclopedic articles so close to an event. As an article grows, information will need to be summarized in order to keep the article concise. But in order to summarize, editors must prioritize
46
what is important without doing any of their own synthesis (in which case they would be abandoning NPOV and developing what they call original research). Since the news media tends to produce analysis only after producing hard news stories about day to day happenings, these sources can be hard, if not impossible, to come by especially since so many editors would rather edit the page than waiting to see as some users call for. 3. Reading the article/watching the footage/understanding the context of the original source to check whether the information has been accurately represented. On the third of February, IP .68 complained on the talk page that a screenshot of Al Jazeera video footage entitled "Police in civilian cloth beating a protester in Cairo 1.png" had been inaccurately captioned. It was not clear by watching the footage that this particular shot identified police beating up protesters. Other editors chimed in with possible changes to the caption, for example: "A violent scene from the January 28 Cairo protests" suggested by [User:Physics is all gnomes] who said that he had tried mentioning the reports of plainclothes officers beating protesters, but decided it violated WP:SYNTHESIS (the policy ). [Physics] also added a link to the video and the pictured scenes location in the timeline (roughly 1:50) for others editors reference since this was not included in the image information. The caption was altered and then the photo was removed soon afterwards. 4. Finding sources that confirm a single source Figure 13: From the 2011 Egyptian Revolution article Since so many editors were watching live Al Jazeera television coverage of the events, the at 12:32, 1 Feb 2011 core editors had to work together to find online sources to back up information that was cited to television. In the example below, an editor was watching television news coverage and heard an opposition leader saying that talks would only be held with the army (rather than with Mubarak). He edits the page to reflect this with a citation to a TV source. Ocaasi brings this reference to the talk page and asks for more information on the leaders name and the source. Other editors respond with what they suspect it was but with no conclusive answers. Silverseren finds an online source and another editor, Lihaas, edits the reference in the article, concluding the collaborative turn and Ocaasi praises Silverseren for his find. This is an excellent example of the collaborative verification processes that were so successful in this article. Editors can be seen here to be working well together, helping to verify claims using individuals knowledge of unfolding events (The Egyptian Liberal), using
sophisticated search skills (Silverseren) and leadership qualities (Ocaasi) to ensure the quality of the article.
This conversation also provides evidence that the source cited in a Wikipedia article is not always the same as the source of that information. This phenomenon was verified in interviews, where editors stated that they found their information in what they believed was a more controversial source (Al Jazeerah, for example, was cited by many editors as unreliable on other pages) and instead found similar information in a better accepted source and used that as the citation reference. In the case of the 2011 Egyptian revolution article, editors were frequently obtaining their information from television sources and needed to find additional online sources (often released after the live television footage) to back up their claims using agreed-upon sourcing methods. On the one hand, finding additional sources to verify a single source can lead to more accurate curation, but on the other hand, the use of mainstream news sources can render the source of information opaque as readers try to verify claims made in an article.
48
Conclusions and design considerations

A typology of sources In the diagram below, I have outlined a Wikipedia source typology, highlighting each type of relevant source as well as the characteristics that are critical for an understanding of where information on Wikipedia originated. It is important to note here that the characteristics for an editor deciding whether a source is reliable or relevant enough for inclusion in an article is different from the characteristics used by readers to decide whether the article itself is reliable or not. For example, former New York Times journalist, Judith Millers coverage of Iraqs alleged weapons of mass destruction would be notable enough for coverage in Wikipedia (even though the allegations turned out to be false) because the stories played an instrumental role in the USs decision to go to war in 2003. The characteristics of notability (for misinformation or rumor) or validity, verifiability, depth, experience etc (for accurate information) can be used by editors to select sources for inclusion in an article, but information about who has edited the page as well as information about the page itself are some of the characteristics used by readers to determine the accuracy of the article itself.
Wikipedia sources and their related characteristics Source type Characteristics/variables Editors Experience (types of articles); roles (mediation/citations/technical/editing) Article Original version/s of the article: comparisons between the original and the current Reference Work: Author: Publisher: (i.e. book/journal/article/blog Primary/secondary/tertiary previous works/reputation; Third party publisher?; post/Tweet/photograph/video etc) source; specificity and expertise number of people depth; verified or not involved Work: Author: Publisher: Work is notable Author is notable Publisher is notable Goal For readers/users to determine the accuracy/POV of the WP article For editors to determine the accuracy of the source for inclusion in the WP article For editors to determine the relevance for inclusion in the WP article
It is important to note that multimedia sources such as photographs, cartoons and videos, as well as so-called social media sources require different methods of classification regarding whether they are primary or secondary or tertiary. Below are a few potential ways of
characterising new media sources, where a source may be seen as secondary when it is chosen by a traditional publisher or expert author, and where we assume such publishers have done the necessary checks to ensure that information is verified, at least where possible within a short news timeframe. Examples of how new media could be characterised in terms of Wikipedias sources principles Twitter Images Video Primary source Cartoon by Carlos Latuff from Amateur video footage of the 2004 Twitter/Twitpic Indian Ocean earthquake and tsunami
Tweet by Sohaib Athar that became known as the first report of the deadly raid on Osama bin Laden
Secondary source
Tweet by Twitter user re-tweeted by NPR journalist, Andy Carvin
Photograph depicting people Amateur video footage of Japanese in the streets of Cairo tsunami on the New York Times carrying a poster of the Latuff website cartoon during protests
50
Elucidating references by exposing the workflow After developing a typology of sources and isolating different types of Wikipedia source work, I made two sets of design considerations as follows: 1. The first would be to for designers to experiment with exposing variables that are important for determining the relevance and reliability of individual sources including editors, as well as the reliability of the article as a whole. 2. The second would be to provide a trail of documentation by replicating the work process that editors follow (somewhat haphazardly at the moment) so that each source is provided with an independent space for exposition and verification, and so that editors can collect breaking news sources collectively.
1. Exposing variables Traditional encyclopedias (for example to the left) do not tend to focus on the source of the information they provide but Wikipedia has chosen to be different by compelling editors to provide citations for all information that could be questioned. The ultimate goal here is in line with the free and open source software principle of being able to look under the hood, to reverse-engineer a written work, a work of code or piece of hardware in order for an individual to recreate it for themselves or at least make up their own mind about a particular topic.
Figure 14 A page from a traditional encyclopedia (note the lack of citations). Ref: The Encyclopedia of Weather and Climate Change page spread sample from http://www.terrain.org/reviews/26/encyclopedia_of_weather.htm
52
Figure 15 Screenshot from Wikipedia article on icebergs (note the references and external links) http://en.wikipedia.org/wiki/Iceberg
Traditional citation mechanisms dont work particularly well to achieve this goal. Links to online sources often change or are not archived (in the case of Twitter), citation standards do not accommodate newer types of information (blog posts by independent authors on well- known news sites, for example) nor do references expose the variety of variables that could be used by readers and editors to better understand where their information is coming from. Articles could, for example, provide user-friendly information showing where the majority of editors are editing from, how much experience they have, as well as which parts of the article were edited by the original author. Some work has been done on showing readers how often parts of an article have been edited and by how many editors, but although the number of edits may be one variable for readers to decide whether they trust content on Wikipedia, this single variable doesnt necessarily equate with trust (Adler et al., 2008). Innovative source visualisations could focus on features that go beyond what a user would expect from a page. For example, a visualization might show that an article about Zimbabwe is edited by a the majority of people from outside the country, or whether an original authors contributions survive longer and suffer fewer modifications than later contributions by other authors to the same page (along the line of research by Vigas, Wattenberg, & Dave, 2004). In so doing, readers and editors would be provided with tools to assess the reliability and relevance of sources and articles using a variety of lenses.
2. Documenting the workflow Editors perform tasks of discovering, choosing, verifying, summarizing, adding information and editing the article in ways that, if formalized, can work to provide a paper trail that can be adapted as the article matures and grows. Practices that editors used to perform the following actions included:
a) b) c) d) e) f) storing discovered articles either using their own editor domains by putting relevant articles into categories or by alerting other editors to breaking news on the talk page, choosing sources by finding at least two independent sources that corroborated what was being reported but then removing some of the citations as the page became too heavy to load, verifying sources by finding sources to corroborate what was being reported, by checking what the summarized sources contained, and/or by waiting to see whether other sources corroborated what was being reported, summarizing by taking screenshots of videos and inserting captions (for multimedia) or by choosing the most important events of each day for a growing timeline (for text), adding text to the article by choosing how to reflect the source within the articles categories and providing citation information, and editing disputing the way that editors reflected information from various sources and replacing primary sources with secondary sources over time.
The majority of posts in the Talk pages studied related to editors posting a reference in order to achieve the following goals: a) alert others to breaking news and ask for comments on importance/relevance to the article; b) ask others to add the source to the article; c) ask for more sources to corroborate a source. Editors worked together to perform these micro-tasks, with some editors working primarily on collecting/verifying/citing sources. Since so much of a talk page is related to the tasks around sources, particular source workflow pages (separate to Talk pages) could be created where editors could focus activities around source work using a decision tree similar to the one below:
54
Figure 16 Work process for sources in breaking news articles
In this way, items can be entered into the workflow from discovery (solving the problem that some editors had to use their own talk pages to collect news items) through to citation in the article, with documentation tied to each source regarding the rationale for its inclusion or exclusion. On ranking By exposing source characteristics and by collecting usage information as editors move through the source workflow, we can replace the original ranking idea with something more practical and in keeping with principles of citizen-driven cooperative work like Wikipedia and Ushahidi. Editors and mapmakers are working to curate a meaningful collection of items by making important editorial judgements about the content that they receive, therefore it makes sense to use that data (location of source, corroborating sources, relevant categories of
the article where it might fit etc) to build a database of sources and their context, rather than implement a new action of voting up or down when it is unclear what editors are evaluating in their votes, and when this is an extra action rather than based on actual use.

56
Bibliography
Adler, B. T., Chatterjee, K., de Alfaro, L., Faella, M., Pye, I., & Raman, V. (2008). Assigning trust to Wikipedia content. Proceedings of the 4th International Symposium on Wikis - WikiSym 08 (p. 1). New York, New York, USA: ACM Press. Retrieved from http://dl.acm.org/citation.cfm?id=1822258.1822293 Charmaz, K. (2006). Constructing Grounded Theory: A Practical Guide through Qualitative Analysis (Introducing Qualitative Methods series) (p. 224). Sage Publications Ltd. Retrieved from http://www.amazon.com/Constructing-Grounded-Theory-Qualitative- Introducing/dp/0761973532 Graham, M. (2011). Wiki Space: Palimpsests and the Politics of Exclusion. In G. Lovink & N. Tkacz (Eds.), Critical Point of View: A Wikipedia Reader (pp. 269-282). Amsterdam: Institute of Network Cultures. Graham, M., Stephens, M., Hale, S. A., & Kono, K. (2012). Geographies of the Worlds Knowledge. Jurkowitz, M. (2011). Events in Egypt Trigger Record Coverage. Journalism.org. Retrieved from http://www.journalism.org/index_report/pej_news_coverage_index_january_31_february_6_2011 Kamir, D. (2011). Parallel online and real world Egyptian revolutions, or Wikipedias Tahrir Square. Drors Blog. Retrieved July 30, 2012, from http://anduraru.wordpress.com/2011/06/25/parallel-online-and-real-world-egyptian-revolutions-or-wikipedias- tahrir-square/

Wikipedia Sources: Managing Sources in Rapidly Evolving Global News Articles On The English Wikipedia

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Wikipedia Sources: Managing Sources in Rapidly Evolving Global News Articles On The English Wikipedia

Uploaded by

Copyright:

Available Formats

Wikipedia

An Ushahidi/SwiftRiver Report by Heather Ford, Ushahidi/SwiftRiver Ethnographer, August, 2012

Figure 2 Screenshot of my Evernote desktop showing initial coding

Figure 4 Work process for sources in breaking news stories

A. Sources on the books

published status primary, secondary or tertiary sources

http://en.wikipedia.org/wiki/Hossam_el-Hamalawy http://en.wikipedia.org/wiki/Al-Wasat_Party http://en.wikipedia.org/wiki/2011_Israeli_embassy_attack http://en.wikipedia.org/wiki/Maspero_demonstrations

Example of news alert with just-released important information:

A section of links from Audes Egypt page http://en.wikipedia.org/wiki/User:Aude/Egypt

The Egyptian Liberal

US Vice President Joseph

International: Di rect source - neutrality

Changes the citation reference to PBS [3]

Figure 11 Web citation template and news citation template

Figure 12 Different source templates

Conclusions and design considerations

Tweet by Twitter user re-tweeted by NPR journalist, Andy Carvin

Figure 16 Work process for sources in breaking news articles

You might also like