Writing History in the Digital Age

The Wikiblitz (Graham)

The Wikiblitz: A Wikipedia Editing Assignment in a First Year Undergraduate Class (2012 revision)

by Shawn Graham

See 2013 final edition from the University of Michigan Press

See 2011 open peer review version, with many more comments

¶ 1 Leave a comment on paragraph 1 0 In this paper, I describe an experiment conducted in the 2010 academic year at Carleton University, in my first year seminar class on Digital History. This experiment was designed to explore how knowledge is created and represented on Wikipedia, by working to improve a single article. The overall objective of the class was to give students an understanding of how historians can create “signal” in the “noise” of the Internet, and how historians create knowledge using digital media tools. Given that doing “research” online often involves selecting a resource suggested by Google (generally one within the first three to five results)¹ this class had larger media literacy goals as well. The students were drawn from all areas of the university, with the only stipulation being that they had to be in their first year.

¶ 2 Leave a comment on paragraph 2 0 The positive feedback loops inherent in the Web’s structure greatly influence the way history is consumed, disseminated, and created online. Google’s algorithms will retrieve an article from Wikipedia, typically displaying it as one of the first links on the results page. Somewhere, someone will decide that the information is “wrong” and he (and it is typically a he)² will need to “fix” the information, clicking on the “edit” button to make the change. To Google’s algorithms, this is one of many signals that this page is more valuable, more relevant and so worth a higher ranking. In this way, Wikipedia and Google feed one another and the loop is strengthened.³

¶ 3 Leave a comment on paragraph 3 0 We as historians need to teach our students to understand how this all works, and how this creates historical knowledge. Digital media make all history public history (whether we like it or not)⁴, and we need to get our research into that positive feedback loop. While Google is a closed service, its workings only dimly perceived through its effects, we can at least engage with the other part of that positive feedback loop: Wikipedia.

¶ 4 Leave a comment on paragraph 4 0 Using Wikipedia in teaching is not a new idea; Roy Rosenzweig made that argument in 2006.⁵ Wikipedia itself now has a page for “School and University Projects” that lists over 50 formal collaborations with Wikipedia.⁶ This experiment was my first foray into using Wikipedia editing as a formative assessment exercise. While it was by-and-large a successful experiment, it did have one unexpected element: pushback and resistance from one significant element in the class, my declared History majors.

¶ 5 Leave a comment on paragraph 5 0 FYSM1405a, Digital History
We took some time to get to Wikipedia in this course. The first section of the course looked at the sheer mass of historical materials available on the Internet, asking, how do we find our way through all of this? How do we identify what is important? The structuring readings during this module were reflections by the seminal author Roy Rosenzweig (founder of the Center for History and New Media at George Mason University).

¶ 6 Leave a comment on paragraph 6 0 We also looked at how the “doing” of history was itself an “unnatural act,” in Sam Wineburg’s felicitous phrase.⁷ This led to a second module where the students explored the idea that we never observe the past directly; we must build models to fit what we “know” into a system of explanation. In digital work, these models are explicitly written in computer code. Understanding how the code forces a particular worldview on the user is a key portion of becoming a “digital historian.” Computer games are another kind of model of the world; historical computer games are some of the best selling games on the market today. A consideration of gaming and “playing” with history led to a module focused on crowdsourcing history, and to this particular assignment. Wikipedia can be thought of as a kind of game where competing visions of common knowledge vie for dominance.⁸ I introduced the related idea that since Wikipedia involved complex interactions between hundreds of thousands of autonomous individuals, who interacted according to a small set of rules,  it could be considered a kind of complex system. In this way a coherent Wikipedia entry is an emergent property of decentralized, undirected cooperation and competition.⁹ We spent two sessions before the Wikiblitz looking at crowdsourcing and ways that small changes/additions can add up to substantial revisions.¹⁰ We discussed Wikipedia’s ‘Neutral Point of View’ (NPOV) provisions by looking at political blogs, and contrasting them with other resources.¹¹ We looked at the history of Wikis more generally, and that of Wikipedia itself specifically.¹²

¶ 7 Leave a comment on paragraph 7 0 Figure 1: The Ottawa Valley on Wikipedia (September 20, 2011)

¶ 8 Leave a comment on paragraph 8 0 The assignment prompt was as follows:

At your computer, examine the article at http://en.wikipedia.org/wiki/Ottawa_Valley ¹³ (Figure 1, a screenshot of the page as it exists today). Identify areas that are logically weak or poorly written, or areas (especially related to its history) that are otherwise incomplete. Using a pseudonym, log into Wikipedia and make a substantial improvement to the article. Email me with your pseudonym and a brief description of the changes you made. All changes are to be made within class time.¹⁴

During a subsequent class, you will review how the article evolved during your blitzing of it, and the subsequent changes made by the wider Wikipedian community. You will be asked to reflect on how much of their contribution survived the interval. Why did those parts survive? Why did some parts get reverted or deleted? How does the Wikipedian community deal with citations and points of view? Your reflection will be written before the class, taking the form of a short paragraph, and will form the starting point for the class discussion.

¶ 11 Leave a comment on paragraph 11 0 Part one of this assignment, the Wikiblitz itself, was conducted on November 26, 2010. Part two, the reflection and discussion, took place on December 1, 2010. On December 1, the students were shown a time-lapsed video illustrating how the wiki page changed over the course of the blitz and the subsequent week. They shared their observations with their seat mates to either side, before sharing with the class as a whole. Their written reflections were taken in for grading as per the rubric, Figure 2 (noting that the majority of the points concerned their actual engagement with the Wikipedia page).

¶ 12 Leave a comment on paragraph 12 0 Figure 2. Rubric for the Wikiblitz exercise.

¶ 13 Leave a comment on paragraph 13 0 My desired outcome was that the students should see how knowledge creation on Wikipedia is as much about style as it is about substance. I wanted them to see that writing for Wikipedia constitutes a kind of peer-review. Finally, I hoped that they would perceive how the NPOV provisions could lead to particular kinds of rhetoric and judgments regarding knowledge credibility and suitability (and so situate this kind of writing firmly within the continuum of historiographic writing).¹⁵ What I did not do in preparing for this exercise was engage in any explicit debate over whether wiki writing was an appropriate activity for an historian. Given the trajectory of the class content and conversation, I assumed that the rationale was by this point self-evident (encapsulated in the opening to this essay). In hindsight, that was perhaps an error.

¶ 14 Leave a comment on paragraph 14 0 Resistance and Surprise
I had made it clear to my students that I felt that Wikipedia was a valuable resource, when students understood how it worked and used it appropriately. Curiously however there was pushback from an unexpected quarter: my actual history students. As a first year seminar at my institution, the majority of the students come from other majors. My history students themselves were actually in the minority. In conversation, it became apparent that these students already had quite clear ideas about authority, authorship, and intellectual property that fit in quite well with established ways of writing history.¹⁶ They had internalized the main strength of a Wiki, that it may be edited by anyone, as a challenge to “their” work, and thus something to be avoided: “I did the work. I don’t want somebody screwing it up”. Others have noted this phenomenon.¹⁷

¶ 15 Leave a comment on paragraph 15 0 Clay Shirky wrote in 2003:

And this [the speed with which changes can be reverted], mirabile dictu, is why wikis can have so little protective armor and yet be so resistant to damage. It takes longer to set fire to the building than put it out, it takes longer to graffiti the wall than clean it, it takes longer to damage the page than restore it. If nearly two hours of work spent trying to subtly undermine a site can be erased in minutes, that’s a lousy place to hang out, if your goal is to get people’s goat.¹⁸

¶ 17 Leave a comment on paragraph 17 0 The idea that one has to monitor a site also produced pushback in my core group of history students. It seems to me that trained by years of launch-and-forget, where a paper or assignment is written, graded, and then never revisited, has made it difficult for students to entertain the idea that scholarship is conversation; that what you write can have an impact and you should respond to that impact.¹⁹

¶ 18 Leave a comment on paragraph 18 0 We discussed these issues in class, and I felt that I was making progress. However, when the day arrived to do the Wikipedia assignment in-class, a large proportion of that minority of students were “sick.”

¶ 19 Leave a comment on paragraph 19 0 The Day of the Wikiblitz
While the students were making their edits to the page, I observed the edits page and commented on what I was seeing via instant messaging to the class as a whole.²⁰ The class period was one hour and 15 minutes. Many of my comments concerned the intricacies of editing and formatting the page, and guiding the secondary research going on in the background (or at least, trying to guide it). Below are certain key observations:

Great to see some changes being made already. But a question for you — many of the recent changes are focusing on the City of Ottawa itself: is that appropriate for an article on ‘the Ottawa Valley’? Shouldn’t the focus be elsewhere? Perhaps this is a change that needs [to] be made…? (n.b. You can of course make edits to somebody else’s edits, from this class!)

[some time later:] Folks, this is an article about the Ottawa Valley, not the city of Ottawa!

[some time later again:] Seems to be a lot of energy focused on the tourism aspect… has anybody corrected any obvious errors in the text yet? What about the fact that a valley has two sides…? where’s the info on the Quebec side?

¶ 23 Leave a comment on paragraph 23 0 This perhaps was one of the hardest lessons for the students to absorb, that Wikipedia articles are “spare” in the sense that they contain no fat. If an article loses its focus, other users will either delete that fat or remove it to its own wiki page. In the subsequent discussion of the exercise, the students were evenly split on whether or not I should have intervened during the exercise to remind them about scope. Was a paragraph on the Ottawa Valley’s largest city warranted? By and large, the class ultimately decided that it was not, since the city is now culturally (at least, in the students’ point of view) and legally distinct from the other jurisdictions in the region. We explored the pattern of links that connected (or not) these two articles, noting that a person who landed on the “Ottawa (City of)” page (or even the “Ottawa (disambiguation)” page) would not be directed to the “Ottawa Valley” page, nor would a visitor to the “Ottawa Valley” page be directed to the “Ottawa (City of)” page.²¹ As in life, so in art: the two concepts were distinct, reflecting and reinforcing that distinction. It is important to remember and to make clear to students that in Wikipedia, it is not just the content of a given page that matters, but also the network structure of links that connect all these together.²² (Perhaps a few rounds of the Wikigame, a variation of the party game, “six degrees,” could be useful to make that point). ²³

¶ 24 Leave a comment on paragraph 24 0 The energy that the students expended on the tourism industry was interesting. In the discussion, it transpired that this was because it was “easiest.” Aside from the Wikipedia link, a basic Google search for “Ottawa Valley” returns nearly 9 million results, the first few pages of which are tourism related. We were on campus and had full access to library resources while we did this blitz, and we had already had numerous discussions about best practices in research. That it became apparent quite quickly, and it was publicly demonstrable, that the students were not even approaching basic expectations for research was an important outcome.²⁴

[A student] has just made some edits to the site… but a wikipedia automated vandalism ‘bot has reverted them!

¶ 26 Leave a comment on paragraph 26 0 This event was a great surprise to the entire class, me included, as we did not realize that these even existed. Wikipedia of course has a page explaining how wiki bots work.²⁵ Much of the tedious work of editing Wikipedia pages (correcting link formatting for instance) can be automated within the wiki framework. Currently, there are well over 1000 distinct tasks that are approved for bots. Some of the earliest bots were created to upload massive amounts of material into Wikipedia quickly (this was how major portions of the 1911 Encyclopedia Britannica on the Project Gutenberg site were uploaded into Wikipedia, apparently).²⁶

¶ 27 Leave a comment on paragraph 27 0 As we discussed this incident, we surmised that our small class’s activities, a concentrated stream of edits, all from more or less the same place, at the same time, must have triggered the bot to revert our changes. The student whose edit finally triggered the bot was greatly upset by this. How could a bot decide that her work was somehow malicious? It was a prime teachable moment on the way humans and computers interact.

[final comment by SG:] Hi everyone – in the space of a class, we’ve made 30 substantial edits to the page (and many minor ones); increasing its size from 13.8 kb to 23.4 kb – that’s the equivalent of about four pages of text. Now – until Wednesday [the following week’s class], keep an eye on the page. Let’s see how long this version lasts; don’t make any more edits.

¶ 29 Leave a comment on paragraph 29 0 A year after this class ended in 2010, there have been about 40 edits to the page. Clearly, this page is not one that attracts a lot of attention from the contributors to Wikipedia. But, our burst of activity did attract others to the site, and some changes and reversions were made by other users. Wikipedia users and editors might often operate under pseudonyms, but activity draws attention. Many of the students were quite surprised by this, since it undermined the idea of the anonymous troll making malicious changes undetected.

¶ 30 Leave a comment on paragraph 30 0 The following week, I put together a time-lapsed video of the edits to the page from its one-line birth in 2005 to the end of November 2010, following the example of Jon Udell’s Heavy Metal Umlaut video.²⁷ Visualizing the evolution of a Wikipedia page is very instructive. The interests and early structure that emerged in the article’s first few months seem to set the skeleton for all subsequent revisions. Once a structure emerges, it seems it takes a lot of energy to over-rule it or otherwise make substantial changes. For instance, the political history of the Ottawa Valley was quickly expunged but a section on First Nations’ land claims in the area resisted all efforts to remove it (by other non-class Wikipedia authors).

¶ 31 Leave a comment on paragraph 31 0 The exercise was for the most part, successful. In their written feedback, I was particularly heartened to read the following:

The fact that many of the changes made by the class were reverted [by other Wikipedians] means that even an ‘any one can edit’ site like Wikipedia is in fact conservative and resistant to change. Why is that? Perhaps it’s because people take ownership of particular pages… I also thought it was quite amazing how the anti-vandalism bot reversed some of our changes…this feature designed to preserve the presentation of fact has the effect of preserving misinformation as well…

The fact that the people writing and editing Wikipedia pages could in fact be just like us – first years with little in-depth knowledge – is actually rather frightening…

¶ 34 Leave a comment on paragraph 34 0 I tabulated the content of my students’ feedback:

¶ 35 Leave a comment on paragraph 35 0 That students need to understand how knowledge can be crowd-sourced, produced and disseminated on the web is I think not a radical conclusion. What this small exercise demonstrates for writing history in the digital age is one small way of confronting the more important issue: that our history students can be reluctant to engage with this mode, this way of writing. There will be pushback, and we need to explore it, understand where it comes from, and think carefully about how to address it. If we want to raise the quality of public discourse about history, we have to begin with our students and show them how what they do can have immediate impact, given the feedback loop that exists between Google and Wikipedia. In some ways my experiment failed, in that I did not achieve the buy-in of all of my ‘official’ history students; but in other ways it succeeded in that I reached my other students who did not normally (as a part of their regular course of study) have to confront the ways knowledge is socially constructed. For one brief moment, they were digital humanists.

¶ 36 Leave a comment on paragraph 36 0 About the author: Shawn Graham is Assistant Professor of Digital Humanities in the Department of History at Carleton University in Ottawa, Canada. The Wikiblitz was part of the course work for FYSM1405a, ‘Digital History,’ and a brief reflection on this assignment was first posted on Graham’s blog, ‘Electric Archaeology: Digital Media for Learning and Research’ (http://electricarchaeologist.wordpress.com).