a born-digital, open-review volume edited by Jack Dougherty and Kristen Nawrotzki

The Wikiblitz: A Wikipedia Editing Assignment in a First Year Undergraduate Class (Fall 2011 version)

1 Leave a comment on paragraph 1 4 In this paper, I describe an experiment in my first year seminar class on Digital History where we explored ideas about how knowledge is created and represented on Wikipedia, through actively working to improve a single article.

2 Leave a comment on paragraph 2 0 Thanks to the Internet, positive feedback loops are changing how history is consumed, disseminated, and created. Search for any topic using the Google search engine, and its algorithms will typically retrieve an article from the Wikipedia, presenting it on the most valuable real estate on the Internet: a listing in the first few links on the results page. Sometimes that information will be incorrect, or slanted, or otherwise in need of change. Somewhere, someone will decide that he (and it is typically a he)1 will click on the ‘edit’ button and make the change. To Google’s algorithms, this is one of many signals that this page is more valuable, more relevant than some other page – and thus worth a higher ranking. In this way, Wikipedia and Google feed one another.2 That this feedback loop was able to be established is thanks to the design principles adopted by Wikipedia where every article would be free and would be created using open-source software (and thus allowing its content to be used for other purposes).3

3 Leave a comment on paragraph 3 0 A new facet to this feedback loop has recently been launched. Called “Google+”, it represents Google’s latest foray into so-called social networking. Google+ focuses on an individual’s circles of friends and acquaintances. It pushes two different streams of information at a user, “the stream” and “sparks”. The stream is the continuously updating scroll of information coming from individuals in a user’s various circles. “Sparks” is a continuous stream of information based on a user’s specific interests.4 Steve Levy writes,

This mother of all streams would be the equivalent of an intravenous feed of information, with inclusion of all the vital content from our social graph and the world at large (Google calls this the “interest graph”). It would scroll forever, and everything would be relevant. If Google’s original goal was to expeditiously dispatch us elsewhere, with this near-clairvoyant stream, Google could turn us into search potatoes who never leave.5

5 Leave a comment on paragraph 5 9 If Google+ succeeds, it is entirely possible that the results returned by its streams could be vastly different from those from a standard search.6 We as historians then will need to understand, and to teach, our students to understand how this all works, and how this creates historical knowledge, and historical consciousness. Digital media make all history public history (whether we like it or not), and we need to get our research onto that interest graph, into that positive feedback loop. While Google is a closed service, and its workings can only be dimly perceived through its effects, we can at least engage with the other part of that positive feedback loop: the Wikipedia.

6 Leave a comment on paragraph 6 0 Historians are already trained in how to do “proper” research. It is our students who need to understand the perils and potentials of Wikipedia. We all of us need to engage and work to make Wikipedia better, more representative, and more diverse. Until some new disruptive technology comes along, for the time being Wikipedia has become our distributed memory, our starting point (and for many, the end point) of research. Using Wikipedia in teaching is not a new idea; Roy Rosenzweig made that argument in 2006. 7 A search of Google for “use Wikipedia for teaching” produced over 4 million results as of July 6th 2011. Wikipedia itself now has a page for “School and University Projects” that lists over 50 formal collaborations with Wikipedia.8 This experiment with Wikipedia was my first foray into using Wikipedia editing as a formative assessment exercise. While it was by-and-large a successful experiment, it did have one unexpected element: pushback and resistance from one significant element in the class, my declared History majors.

7 Leave a comment on paragraph 7 2 FYSM1405a, Digital History
In 2010-11, I taught FYSM1405a, a first year seminar designed to give students an understanding of how historians can create “signal” in the “noise” of the Internet, and how historians create knowledge using digital media tools. Given that many individuals when doing “research” online will select a resource suggested by Google – and generally one within the first three to five results9 – this class had larger media literacy goals as well. The students were drawn from all areas of the university, with the only stipulation being that they must be in their first year.

8 Leave a comment on paragraph 8 1 We took some time to get to the Wikipedia in this course. The first section of the course looked at the sheer mass of historical materials available on the Internet, asking, how do we find our way through all of this? How do we visualize or otherwise identify what is important? The structuring readings during this module were reflections by the seminal author Roy Rosenzweig (founder of the Center for History and New Media at George Mason University.10

9 Leave a comment on paragraph 9 4 We also looked at how the “doing” of history was itself an “unnatural act”, in Sam Wineburg’s felicitous phrase.11 This led to a second module where the students explored the idea that we never observe the past directly; we are always building models to fit what we “know” into a system of explanation. In digital work, these models are explicitly written in computer code. Understanding how the code forces a particular worldview on the consumer is a key portion of becoming a “digital historian”. Computer games are another kind of model of the world; historical computer games are some of the best selling games on the market today. How do they represent history? Can we subvert or challenge these representations? A consideration of gaming and an ethic of “playing” with history led to the a module focused on crowdsourcing history, and to this particular assignment. Wikipedia is, in a certain sense, a kind of game where competing visions of common knowledge vie for dominance. McGonigal suggests that Wikipedia shares many traits with massively multiplayer online games.12 Wikipedia can also serve as a platform for casual games which “race” to find the shortest paths between random articles.13

10 Leave a comment on paragraph 10 0 I introduced the idea that since Wikipedia involved complex interactions between hundreds of thousands of autonomous individuals, who interacted according to a small set of rules, that it could be considered a kind of complex system. That is, knowledge production – decentralized, undirected cooperation to write an encylopedia- was an emergent property to these interactions.14 We looked at agent based modeling simulations of termites as a metaphor for how crowdsourcing can create knowledge.15 In the simulation, the termites interact in a world with the simple instruction “pick up a piece of wood when you find it; put it down when you encounter another piece of wood”. From an initial random scattering of wood chips, a single pile emerges in the center of the world.  We looked at the history of Wikis more generally, and that of the Wikipedia itself specifically.16

11 Leave a comment on paragraph 11 0 Figure 1: The Ottawa Valley on the Wikipedia (September 20, 2011)

12 Leave a comment on paragraph 12 5 The assignment prompt was as follows:

Instructions to students: At your computer, examine the article at http://en.wikipedia.org/wiki/Ottawa_Valley (Figure 1, a screenshot of the page as it exists today). Identify areas that are logically weak or poorly written, or areas – especially related to its history- that are otherwise incomplete. Using a pseudonym, log into Wikipedia and make a substantial improvement to the article. Email me with your pseudonym and a brief description of the changes you made. All changes are to be made within class time.17

During a subsequent class, you will review how the article evolved during their blitzing of it, and the subsequent changes made by the wider Wikipedian community. You will be asked to reflect on how much of their contribution survived the interval; why did those parts survive? Why did some parts get reverted or deleted? How does the Wikipedian community deal with citations and points of view? Your reflection will be written before the class discussion, taking the form of a short paragraph, and will form the jumping off point for the class discussion.

15 Leave a comment on paragraph 15 3 Part one of this assignment – the Wikiblitz itself – was conducted on November 26, 2010. Part two – the reflection and discussion – took place on December 1, 2010. On December 1, the students were shown a time-lapsed video illustrating how the wiki page changed over the course of the blitz and the subsequent week. They were then given the prompt to take 15 minutes to write down their reflections on their experience creating knowledge on Wikipedia. They shared their observations with their seat mates to either side, before sharing with the class as a whole. Their written reflections were taken in for grading as per the rubric below, Figure 2 (noting that the majority of the points concerned their actual engagement with the Wikipedia page).

16 Leave a comment on paragraph 16 0 Figure 2. Rubric for the Wikiblitz exercise.

17 Leave a comment on paragraph 17 5 My desired outcome was that the students should see how knowledge creation on Wikipedia is as much about style as it is about substance. I wanted them to see that writing for the Wikipedia constitutes a kind of peer-review. Finally, I hoped that they would percieve how the “neutral-point-of-view” (NPOV) provisions could lead to particular kinds of rhetoric and judgements regarding knowledge credibility and suitability.

18 Leave a comment on paragraph 18 0 Resistance and Surprise
When I assigned this exercise in class, I was quite excited. I had already made it clear to my students that I felt that Wikipedia was a valuable resource, when students understood how it worked and used it appropriately. My students were quite surprised by that revelation, since they (being first year students) had had several years of prohibitions against using Wikipedia, courtesy of their secondary school teachers (and indeed were still being told, in other areas of the university, that Wikipedia was not to be engaged with).

19 Leave a comment on paragraph 19 2 Curiously however there was pushback from a certain number of the students, in an unexpected quarter: my actual history students. As a first year seminar at my institution, the majority of the students come from other majors. My history students themselves were actually in the minority. In conversation, it became apparent that these students already had quite clear ideas about authority, authorship, and intellectual property that fit in quite well with established ways of writing history.18 They had internalized the main strength of a Wiki — that it may be edited by anyone — as a challenge to “their” work, and thus something to be avoided: “I did the work. I don’t want somebody screwing it up”. Others have noted this phenomenon.19 Wikis have been a major technology on the Internet for at least a decade now (though they are approaching their 20th anniversary)20 and so one would imagine that this attitude should be diminishing. That this attitude persists is perhaps in part a testament to the power of frequent admonishment in the high schools. Though it rarely stops students from using the Wikipedia, or even citing it, it certainly appears to stop students from contributing to it.

20 Leave a comment on paragraph 20 0 Clay Shirky wrote in 2003:

And this [the speed with which changes can be reverted], mirabile dictu, is why wikis can have so little protective armor and yet be so resistant to damage. It takes longer to set fire to the building than put it out, it takes longer to grafitti the wall than clean it, it takes longer to damage the page than restore it. If nearly two hours of work spent trying to subtly undermine a site can be erased in minutes, that’s a lousy place to hang out, if your goal is to get people’s goat.21

22 Leave a comment on paragraph 22 6 The idea that one has to monitor a site also produced pushback in my core group of history students. It seems to me that trained by years of launch-and-forget, where a paper or assignment is written, graded, and then never revisited, has made it difficult for students to entertain the idea that scholarship is conversation; that what you write can have an impact and you should respond to that impact.

23 Leave a comment on paragraph 23 0 We discussed these issues in class, and I felt that I was making progress. However, when the day arrived to do the Wikipedia assignment (an in-class assignment), a large proportion of that minority of students were “sick”.

24 Leave a comment on paragraph 24 1 The Day of the Wikiblitz
While the students were making their edits to the page, I observed the edits page and commented on what I was seeing via instant messaging to the class as a whole.22 The class period was one hour and 15 minutes. Many of my comments concerned the intricacies of editing and formatting the page, and guiding the secondary research going on in the background (or at least, trying to guide it). Below are certain key observations:

Great to see some changes being made already. But a question for you- many of the recent changes are focusing on the City of Ottawa itself: is that appropriate for an article on ‘the Ottawa Valley’? Shouldn’t the focus be elsewhere? Perhaps this is a change that needs  be made…? (nb you can of course make edits to somebody else’s edits, from this class!)

[some time later:] Folks, this is an article about the Ottawa Valley, not the city of Ottawa!

[some time later again:] Seems to be a lot of energy focused on the tourism aspect… has anybody corrected any obvious errors in the text yet? What about the fact that a valley has two sides…? where’s the info on the Quebec side?

28 Leave a comment on paragraph 28 4 This perhaps was one of the hardest lessons for the students to absorb, that Wikipedia articles are “spare” in the sense that they contain no fat. If an article loses its focus, other users will either delete that fat or remove it to its own wiki page. In the subsequent discussion of the exercise, the students were fairly evenly split on whether or not I should have intervened during the exercise to remind them about scope. Was a paragraph on the Ottawa Valley’s largest metropolitan center warranted? By and large, the class ultimately decided that it did not, since the city is now culturally (at least, in the students’ point of view) and legally distinct from the other jurisdictions in the region. We explored the pattern of links that connected (or not) these two articles, noting that a person who landed on the “Ottawa (City of)” page (or even the “Ottawa (disambiguation)” page) would not be directed to the “Ottawa Valley” page, nor would a visitor to the “Ottawa Valley” page be directed to the “Ottawa (City of)” page.23 As in life, so in art: the two concepts were distinct, reflecting and reinforcing that distinction. It is important to remember and to make clear to students that in the Wikipedia, it is not just the content of a given page that matters, but also the network structure of links that connect all these together.24 (Indeed, perhaps a few rounds of the Wikigame — a variation of the party game, “six degrees” — could be useful to make that point).

29 Leave a comment on paragraph 29 2 The energy that the students expended on the tourism industry was interesting. In the discussion, it transpired that this was because it was “easiest”. Aside from the Wikipedia link, a basic simple Google search for “Ottawa Valley” returns nearly 9 million results, the first few pages of which are tourism related. We were on campus and had full access to library resources while we did this blitz, and we had already had numerous discussions about best practices in research. That it became apparent quite quickly that the students were not even approaching the basics – and it was publicly demonstrable that this was the case – was an important outcome.25

30 Leave a comment on paragraph 30 2 [A student] has just made some edits to the site… but a wikipedia automated vandalism ‘bot has reverted them! This event was a great surprise to the entire class, me included, as we did not realize that these even existed. Wikipedia of course has a page explaining how wiki bots work.26 Much of the tedious work of editing Wikipedia pages (correcting link formatting for instance) can be automated within the wiki framework. Currently, there are well over 1000 distinct tasks which are approved for bots. Some of the earliest bots were created to upload massive amounts of material into Wikipedia quickly (this was how major portions of the 1911 Encyclopedia Britannica on the Project Gutenberg site were uploaded into Wikipedia, apparently).27

31 Leave a comment on paragraph 31 2 As we discussed this incident, we surmised that our small class’s activities — a concentrated stream of edits, all from more or less the same place, at the same time — must have triggered the bot to revert the changes we were making. The student whose edit finally triggered the bot was greatly upset by this: how could a bot decide that her work was somehow malicious? It was a prime teachable moment on the way humans and computers interact.

[final comment by SG:] Hi everyone – in the space of a class, we’ve made 30 substantial edits to the page (and many minor ones); increasing its size from 13.8 kb to 23.4 kb – that’s the equivalent of about four pages of text. Now – until Wednesday [the following week’s class], keep an eye on the page. Let’s see how long this version lasts; don’t make any more edits.

33 Leave a comment on paragraph 33 0 Since the day we worked on this article, there have been only 10 edits. As I write this in July 2011, there have been none at all since March. Clearly, this page is not one that attracts a lot of attention from the contributors to Wikipedia. But, our burst of activity did attract others to the site, and some changes and reversions were made by other users. Wikipedia users and editors might often operate under pseudonyms, but activity draws attention. Many of the students were quite surprised by this, since it undermined the idea of the anonymous troll making malicious changes undetected.

34 Leave a comment on paragraph 34 3 The following week, I put together a time-lapsed video of the edits to the page from its one-line birth in 2005 to the end of November 2010, following the example of Jon Udell’s Heavy Metal Umlaut video.28 Visualizing the evolution of a Wikipedia page is very instructive. The interests and early structure that emerged in the article’s first few months seem to set the skeleton for all subsequent revisions. Once a structure emerges, it seems it takes a lot of energy to over-rule it or otherwise make substantial changes. For instance, the political history of the Ottawa Valley was quickly expunged but a section on First Nations’ land claims in the area resisted all efforts to remove it (by other non-class Wikipedia authors).

35 Leave a comment on paragraph 35 1 The exercise was for the most part, a successful exercise. In their written feedback, I was particularly heartened to read the following:

The fact that many of the changes made by the class were reverted [by other Wikipedians] means that even an ‘any one can edit’ site like Wikipedia is in fact conservative and resistant to change. Why is that? Perhaps it’s because people take ownership of particular pages… I also thought it was quite amazing how the anti-vandalism bot reversed some of our changes…this feature designed to preserve the presentation of fact has the effect of preserving misinformation as well…

The fact that the people writing and editing Wikipedia pages could in fact be just like us – first years with little in-depth knowledge – is actually rather frightening…

38 Leave a comment on paragraph 38 1 I tabulated the content of my students’ feedback:

Gist of comment # of mentions by students
ease of use 1
the way Wikipedia ‘self heals’ 3
lack of professionalism 3
content is contested 5
fact that it is ‘in public’ compels professionalism 1
authority lacking – these people could be just like us! 2
futility of trying to improve articles 2
where do wikipedians get their sources? 1

39 Leave a comment on paragraph 39 3 That students need to understand how knowledge can be crowd-sourced, produced and disseminated on the web is I think not a radical conclusion. What this small exercise demonstrates for writing history in the digital age is one small way of confronting the more important issue: that our history students will be reluctant to engage with this mode, this way of writing. There will be pushback, and we need to explore it, understand where it comes from, and think carefully about how to address it. If we want to raise the quality of public discourse about history, we have to begin with our students and show them how what they do can have immediate impact, given the feedback loop that exists between Google and Wikipedia. In some ways my experiment failed, in that I did not achieve the buy in of all of my ‘official’ history students; but in other ways it succeeded in that I reached my other students who did not normally (as a part of their regular course of study) have to confront the ways knowledge is socially constructed. For one brief moment, they were digital humanists.

40 Leave a comment on paragraph 40 0 About the author:  Shawn Graham is Assistant Professor of Digital Humanities in the Department of History at Carleton University. A brief reflection on the ‘wikiblitz’ assignment was first posted on Graham’s blog, ‘Electric Archaeology: Digital Media for Learning and Research’. The Wikiblitz was part of the course work for FYSM1405a, ‘Digital History’, at Carleton University, Ottawa, Canada.

  1. 41 Leave a comment on paragraph 41 1
  2. Ruediger Glott and Rishab Ghosh, “Analysis of Wikipedia Survey Data. Topic: Age and Gender Differences”. March 2010 http://www.wikipediasurvey.org/docs/Wikipedia_Age_Gender_30March 2010-FINAL-3.pdf 6, accessed September 20, 2011.
  3. Shawn Graham, “Signal versus noise: Why academic blogging matters, a structural argument,” Electric Archaeology, April 2, 2011, http://electricarchaeologist.wordpress.com/2011/04/02/signal-versus-noise-why-academic-blogging-matters-a-structural-argument-saa-2011/ accessed September 20, 2011.
  4. Roy Rosenzweig, “Can history be Open Source? Wikipedia and the future of the past,” The Journal of American History 93.1 (2006) 117-146, online edition http://chnm.gmu.edu/essays-on-history-new-media/essays/?essayid=42 accessed September 20, 2011.
  5. Stephen Levy, “Inside Google+ – How the search giant plans to go social,” Wired June 28, 2011 http://www.wired.com/epicenter/2011/06/inside-google-plus-social/ accessed September 20, 2011.
  6. Ibid
  7. Ibid
  8. Rosenzweig, “Can History be Open Source?”, 2006
  9. Wikipedia: School and University Projects, http://en.wikipedia.org/wiki/Wikipedia:School_and_university_projects, accessed September 20, 2011.
  10. A generalized pattern; see for instance Jansen, Bernard and Amanda Spink, “How are we searching the World Wide Web? A comparison of nine search engine transaction logs,” Information Processing and Management 42.1 (2006) 248-263. Google currently has around two-thirds of the US search market, according to comScore, a digital marketing research firm. comScore, “comScore Releases May 2011 U.S. Search Engine Rankings”, http://www.comscore.com/Press_Events/Press_Releases/2011/6/comScore_Releases_May_2011_U.S._Search_Engine_Rankings, accessed September 20,2011.
  11. With his passing, the center has been renamed The Roy Rosenzweig Center for History and New Media” in his honor, http://chnm.gmu.edu/
  12. Sam Wineburg Historical thinking and other unnatural acts: Charting the future of teaching the past. (Philadelphia: Temple University Press, 2001).
  13. Jane McGonigal, Reality is Broken. Why games make us better and how they change the world. (Penguin Press: New York, 2011), 228-231.
  14. Alex Clemsha, The Wikigame http://thewikigame.com/ accessed September 20, 2011.
  15. cf Melanie Mitchell, Complexity: A guided tour (Oxford: Oxford University Press, 2009), 215-220
  16. Uri Wilensky. NetLogo Termites model. (Center for Connected Learning and Computer-Based Modeling, Northwestern University, Evanston, IL. 1997). http://ccl.northwestern.edu/netlogo/models/Termites accessed September 20, 2011. “Termites” may be run using the Netlogo platform, Uri Wilensky, NetLogo. (Center for Connected Learning and Computer-Based Modeling, Northwestern University, Evanston, IL. 1999). http://ccl.northwestern.edu/netlogo/ accessed September 20, 2011
  17. Robert E. Cummings and Matt Barton, eds. Wiki writing: Collaborative learning in the college classroom (Ann Arbor: University of Michigan Press, 2008). Digital edition: http://hdl.handle.net/2027/spo.5871848.0001.001 ; Rosenzweig “Can history be open source?” 2006
  18. Hence my naming the assignment ‘The Wikiblitz’.
  19. cf Stephanie Vie and Jennifer de Winter, “Disrupting intellectual property: Collaboration and resistance in wikis” in Robert E. Cummings and Matt Barton, eds. Wiki writing: Collaborative learning in the college classroom (Ann Arbor: University of Michigan Press, 2008), 109-110, on the challenge Wikis present to established patterns.
  20. For instance, Lamb “Wide Open Spaces: Wikis, Ready or Not,”  Educause Review, September–October 2004, 37–48  http://connect.educause.edu/Library/EDUCAUSE+Review/WideOpenSpacesWikisReadyo/40498 accessed September 20, 2011; Martin and Dusenberry “Wiki Lore and Politics in the Classroom” in Robert E. Cummings and Matt Barton, eds. Wiki writing: Collaborative learning in the college classroom (Ann Arbor: University of Michigan Press, 2008) 213-214; Blau and Caspi noted a similar mood in collaborative writing spaces like Google Docs, where students reported their own edits improved the draft while others’ made it worse. Ina Blau and Avner Caspi, “Sharing and collaborating with Google Docs: The influence of psychological ownership, responsibility, and student’s attitudes on outcome quality,” Chais Research Center for the Integration of Technology in Education, Open University of Israel, 2011 http://www.openu.ac.il/research_center/download/Sharing_collaborating_Google_Docs.pdf, accessed September 20, 2011.
  21. http://en.wikipedia.org/wiki/History_of_wikis accessed September 20, 2011 For the history of the Internet, the Wikipedia is often peerless.
  22. Clay Shirkey “Wikis, Grafitti, and Process” Corante August 26, 2003 http://many.corante.com/20030801.shtml#50187 accessed September 20, 2011.
  23. See http://en.wikipedia.org/w/index.php?title=Ottawa_Valley&action=history for the page history.
  24. While true at the time of the exercise, this is no longer the case, see http://en.wikipedia.org/wiki/Ottawa
  25. Capocci et al., who write, “… preferential attachment may be the consequence of the intrinsic organization of the underlying knowledge; alternatively, the preferential attachment mechanism emerges because the Wiki technical capabilities are not fully exploited by Wikipedia contributors: if this is the case, their focus on each specific subject puts much more effort in building a single Wiki entry, with little attention toward the global efficiency of the organization of information across the whole encyclopaedia.” A. Capocci, V.D.P. Servedio, F. Colaiori, L.S. Buriol, D. Donato, S. Leonardi, and G. Caldarelli, Preferential attachment in the growth of social networks: The internet encyclopedia Wikipedia,” Physical Review E, 74.3  036116-1:6 http://pre.aps.org/pdf/PRE/v74/i3/e036116, accessed September 20, 2011.  There is also a network structure to the patterning of author collaboration on Wikipedia that should also be scrutinized. Ulrik Brandes, Patrick Kenis, Jurgen Lerner, Denise van Raaij, “Network analysis of collaboration structure in Wikipedia” in Proceedings of the 18th International Conference on World Wide Web, Madrid, Spain. ACM: New York, 2009, 731-740
  26. One that Gabriel Bodard noted on the Stoa.org blog in a discussion on using Wikipedia in Classics courses, in 2007, “Wikipedia as teaching tool”, March 25th http://www.stoa.org/archives/600, accessed September 20, 2011.
  27. http://en.wikipedia.org/wiki/Wikipedia:Bots
  28. http://en.wikipedia.org/wiki/Wikipedia:History_of_Wikipedia_bots
  29. Jon Udell. “The Heavy Metal Umlaut” http://jonudell.net/udell/gems/umlaut/umlaut.html accessed September 20, 2011.

Source: http://writinghistory.trincoll.edu/teach/the-wikiblitz-graham/?replytopara=14