Wikipedia:Wikipedia Signpost/2016-01-06/Recent research: Difference between revisions

Recent research

Teaching Wikipedia, Does advertising the gender gap help or hurt Wikipedia?

By Piotr Konieczny, Tilman Bayer, and Max Klein

A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

Does advertising the gender gap help or hurt Wikipedia?

Reviewed by Tilman Bayer

A working paper in economics provides several novel results shedding light on Wikipedia's much discussed gender gap, focusing on three aspects: The causes of the gender gap in contributors, its impact on Wikipedia's content, and how outreach measures that highlight the gender gap influence participation on Wikipedia.

It uses several sources of data, including the edit histories of all registered English Wikipedia users who have stated their gender in the user preferences, a survey and experiment with 1000 Amazon Mechanical Turk users (from the US only, who were paid $1.50 for a 20 minutes task), and a dataset of biographical articles with the subject's gender obtained from Wikidata (excluding "celebrities like actors, athletes, and pop stars", focusing on "professionals", e.g. politicians and scientists, and cultural figures like writers and composers), together with pageview data.

Regarding causes of the gender gap, the author provides an overview of existing research, for example dismissing the so-called second shift as an explanation ("There are no gender differences in the amount of free time", p.3) and pointing out that "women contribute no less than men to another example of online public good provision, writing user reviews for products and services".

From the survey, the author concludes that "almost half of the gender gap in Wikipedia writing is explained by gender differences in two characteristics: frequency of Wikipedia use and belief about one’s competence ... The gender difference in the belief about competence could be due to women being less competent or due to women underestimating their competence. The survey data does not allow to distinguish these." (While the paper is otherwise well-informed about pre-existing research, it would have benefited from connecting this result to the work of Shaw and Hargittai; see our review of their paper "Mind the skills gap: the role of Internet know-how and gender in differentiated contributions to Wikipedia").

Moving on to the effect of the editor gender gap on Wikipedia's content, the paper finds "that women are about twice as likely as men to contribute to Wikipedia articles about women", based both on the edit histories dataset and the Mechanical Turk survey. Intriguingly, "the number of readers per editor is higher for articles about women, and the share of articles that no one reads is larger in the case of articles about men". In other words, readers prefer articles about women, editors prefer articles about men. The author indicates that the readership discrepancy mostly comes from the tail end of low-traffic biography articles:

"On a typical (median) day in September 2014, no one read 26 percent of the biographies of men versus only 16 percent of the biographies of women."

The third part consisted of an experiment designed to "test whether providing information about gender inequality in Wikipedia changes editing behavior". Mechanical Turk respondents were divided into two groups that were provided with different introductory information about Wikipedia:

"Wikipedia has been criticized by some academics and journalists for having only 9% to 13% female contributors and for having fewer and less extensive articles about women or topics important to women." (a quote from the article Gender Bias on Wikipedia)

vs.

"Wikipedia started in 2001. English-language Wikipedia has over 4.5 million articles."

They were then "asked to imagine a hypothetical situation in which they edit a person’s Wikipedia page. Respondents were asked to look at Wikipedia articles and find some relevant information from the web that is missing from a Wikipedia article. ... In the end, they were also asked how likely they are to edit Wikipedia in the future."

The first version, highlighting the criticism of Wikipedia's gender gap, is "associated with a 35 percent decrease in the likelihood of editing Wikipedia in the future", i.e. discouraged rather than encouraged respondents from contributing, which the author calls "somewhat unexpected". This negative effect is concentrated among men: "The information that the majority of Wikipedia editors are men, leads men to reduce their editing effort, but it does not change the behavior of women." As summarized by the author:

"The result provides an example where encouraging gender equality can partially backfire. Wikipedia has set a goal to increase the share of female editors. One way to achieve this is by discouraging male editors. However, this might not be desirable ... The implication for Wikipedia and other forms of media is that it is important to balance the efforts of attracting new contributors and keeping the current ones."

She also points out that "there are other examples in the literature where informational treatment has backfired".

The paper is highly innovative and adds several novel results (with direct relevance for Wikipedians' work to combat this kind of systemic bias), some of which are not mentioned in this summary. The author seems justified in calling it "the first comprehensive study of gender inequality in a new media environment such as Wikipedia". A weakness of the part of the paper that studies the effect of editors' gender on their contributions might be its partial reliance on the gender as stated in their accounts' user preferences. The author stresses that her methodology is robust against potential under-reporting by one gender (for examples, female editors being less willing to publish their gender in this way because of concerns about harassment). However, she adds that the validity of the results rests on the assumption "that editors don’t systematically report wrong gender. Since the default option is not specifying one’s gender, I would not expect that they are massively reporting wrong gender." In contrast, a 2011 paper by other authors ("WP:CLUBHOUSE", see Signpost summary) that used the same methodology (and concluded that e.g. women vandalize Wikipedia more often than men) explicitly pointed to the possibility that their results might be affected by deliberately wrong reporting (although this might mostly concern vandals with few edits overall, i.e. less relevance to the questions studied here). The paper also falls victim to a survivor bias fallacy when interpreting an otherwise interesting result as "female editors [having] increased from 3.7 percent in 2002 to a peak of 11.5 percent in 2011. In 2013, 10.4 percent of the active editors were female." The option to state a gender in one's user preferences was only introduced in 2009, so it is possible that, for example, there was a much higher percentage of women editing Wikipedia in 2002 who however left before they had the opportunity to state their gender seven years later.

Teaching Wikipedia: The Pedagogy and Politics of an Open Access Writing Community

Reviewed by Piotrus

This dissertation looks at the opportunities for writing pedagogy offered by the Wikipedia:Education program. It provides an interesting, though not comprehensive, overview of the literature in the field, and then proceeds to describe and analyze a number of educational assignments that the author has carried out on Wikipedia through their 2011 course. The author concludes that the "teaching with Wikipedia" approach is generally beneficial to students in a number of ways, from improving their writing and research skills, to an increase in student's rhetorical skills, and understanding of topics relating to knowledge creation. The main limitations of the study, acknowledged by the author, is that it is based on a small sample of students (the course seems to have only about seventeen participants). Nonetheless, it is a useful addition to our still limited understanding of the practice and benefits of the use of Wikipedia in an educational setting.

"Wikipedia, sociology, and the promise and pitfalls of Big Data"

Reviewed by Piotrus

This paper, or perhaps an essay or an Onion piece (2,500 words, with little original research), entitled "Wikipedia, sociology, and the promise and pitfalls of Big Data", is a strange beast. Published in the journal Big Data & Society, it doesn't really address the topic of big data; instead presenting a sociologically-informed and critical discussion of a number of aspects of Wikipedia that, while interesting, seems out of place in an academic journal, and reads more like an academic blog entry. The authors display a reasonable familiarity with Wikipedia, though they make a few factual mistakes (such as suggesting that Wikipedia:WikiProject Sociology was formed with the assistance of the American Sociological Association in 2004; in fact ASA has not been aware of WP:SOCIO until late 2000s and its support for it has been limited to linking to the WikiProject from their Wikipedia Initiative Page).

Based on their literature review, the authors don't hesitate to make some strong claims about Wikipedia, primarily in the vein of Wikipedia becoming less friendly to new editors, though most of those claims are more or less supported by the sources cited. The authors' research question is how the discipline of sociology is framed on Wikipedia, with special attention to the concepts of notability of academics (WP:PROF) and the gender imbalance of the Wikipedia biographies of sociologists. Unfortunately, as this is not a proper research piece, the authors' findings are rather sparse, and primarily concern the fact that topics covered by the WikiProject Sociology and its related portal are poorly structured, that Wikipedia's biographies of sociologists are mostly about male subjects (the article omits, however, the question of gender bias in academia - aren't most sociologists male anyway...? ), and that WP:PROF guideline may not be enforced too strictly for sociological biographies. It was an enjoyable reading, but overall, as seen in the article's sections which are entitled Abstract, Declaration of conflicting interests, Funding and Notes, there is something important missing - the article proper. As the authors make a point of stressing (twice) the chaotic and unorganized nature of Wikipedia's coverage of sociological topics, I can't help but feel that the article, which also fails to drive home any particular and well organized point, could well fit that description too.

See also our earlier coverage of the authors' research project: "Gender imbalance in Wikipedia coverage of academics to be studied with 2-year NSF grant"

Briefly

Wikipedia and the Stock Market

Reviewed by Max Klein

Wikipedia may effect the stock market in a "governing" way, says Crowd Governance: The Monitoring Role of Wikipedia in the Financial Market. It looks at how the stock market and insider trading reacts to the creation of a Wikipedia article about a traded firm. Using a sample of 413 articles on S&P500 firms, it was found that stock prices significantly drop on the days their Wikipedia article is created. Furthermore prices drop further for companies that have more insider traders, or which are more institutionally owned. This goes to show, the authors say, that Wikipedia governs the stock market by "reducing information asymmetry". Firm information on Wikipedia would seem to benefit the public more than information in newspapers, that is bad news for Wall Street.

Other recent publications

A list of other recent publications that could not be covered in time for this issue – contributions are always welcome for reviewing or summarizing newly published research.

• "Understanding the ‘Quality Motion’ of Wikipedia Articles Through Semantic Convergence Analysis"' From the abstract: "This study aims to check if Wikipedia’s [quality] ratings really reflect its stated criteria. According to Wikipedia criteria, having abundant and stable content is the key to article’s quality promotion; we therefore examine the content change in terms of quantity change and content stability by showing the semantic convergence. We found out that the quantity of content change is significant in the promoted articles, which complies with Wikipedia’s stated criteria."
• "Wikipedia's Politics of Exclusion: Gender, Epistemology, and Feminist Rhetorical (In)action" From the abstract: "In this article, I explore how Wikipedia functions as a rhetorical discourse community whose conventions exclude and silence feminist ways of knowing and writing. Drawing on textual analysis of Wikipedia's editorial policies, as well as interviews with female users, I argue that Wikipedia's insistence on separating embodied subjectivity from the production of knowledge limits the site's ability to facilitate any substantial, subversive feminist rhetorical action."
• "Knowledge Quality of Collaborative Editing in Wikipedia: an Integrative Perspective of Social Capital and Team Conflict" From the abstract: "Despite the abundant researches on Wikipedia, to the best of our knowledge, no one has considered the integration of social capital and conflict. Besides, extant literatures on knowledge quality just pay attention to task conflict, while relational conflict is rarely mentioned. Meanwhile, our study proposes the nonlinear relationship between task conflict and knowledge quality instead of linear relationships in prior studies. We also postulate the moderating effect of task complexity."
• "Collective remembering of organizations: Co-construction of organizational pasts in Wikipedia" From the abstract: "The authors analyze 1,459 edits of Wikipedia pages of ten organizations from various industries. Quantitative content analysis detects Wikipedia edits for their reputational relevance and reference to formal sources, such as corporate communication or newspapers. Furthermore, the authors investigate to which degree current corporate communication in form of 177 press releases has an influence on the remembering process in Wikipedia. ... The analysis of press releases shows that current frames provided by corporate communication finds only little resonance in the ongoing remembering processes in Wikipedia."

References

← Previous "Recent research"

Next "Recent research" →

In this issue

6 January 2016 (all comments)

News and notes

In the media

Arbitration report

Featured content

Recent research

WikiProject report

Technology report

+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try .

• Does advertising the Gender Gap... I would have thought it is pretty obvious that saying "Wikipedia wants, and needs, more women and girls to edit" will help, whereas the message "Wikipedians hate women, that's why women hate editing there." will have the opposite effect. I have never seen why some of my colleagues have thought the latter narrative would be the beneficial one to promote to the media. All the best: Rich Farmbrough, 06:53, 7 January 2016 (UTC).
  
  • Actually regardless of what the message is, we know both from WLM and fundraising that a positive wording is less likely to deter participation, while negative wording (in whatever context) has significant negative impact. The trick with canvassing is to try to make things sound fun, and gendergap is not fun (yet). On the whole, though, fascinating findings. Jane (talk) 07:38, 7 January 2016 (UTC)
    • Agreed with the above sentiments whole-heartedly. Thanks to the authors for a great write-up here. I, JethroBT ^{drop me a line} 11:18, 7 January 2016 (UTC)

+1 great report! --Atlasowa (talk) 14:40, 7 January 2016 (UTC)

It says right there in the article that they found negative wording reduced the gender gap in their test population. As a bonus it's easier and a lot more satisfying than positive wording. DPRoberts534 (talk) 03:46, 8 January 2016 (UTC)

Actually, the difference was between negative information about Wikipedia (i.e. the gender gap in editorial content and editors themselves) and a neutral description (neither positive or negative and not mentioning the gender gap at all). Secondly, the study didn't find that negative wording reduced the gender gap except in the sense that those receiving it were more likely to choose articles on women for the hypothetical improvement task. The second and more important finding is that the negative wording decreased the likelihood of actually contributing amongst both men and women, but was strongest for the male participants. As I have pointed out in the past, the narrative of the gender gap that was started by Sue Gardner and since developed via more and more hyperbole has proved both counterproductive and destructive. It infantilized and stereotyped women and turned our male colleagues from people into an equally stereotyped "other" from which we must be protected. With Gardner's 2011 blog post "Nine reasons why women don't edit Wikipedia", the negative narrative was set firmly on its course. Unsurprisingly, the gender gap in the number of women editors (to the extent it existed back in 2011) had really not improved at all by early 2015. So, instead of applying a bit of lateral thinking, the solution was to ramp up the negativity even more to a level where all perspective was lost. In any case, a very interesting article, and many thanks to Tilman Bayer for highlighting it. Voceditenore (talk) 13:15, 8 January 2016 (UTC)

• Intriguingly, "the number of readers per editor is higher for articles about women, and the share of articles that no one reads is larger in the case of articles about men".

that does not imply:

In other words, readers prefer articles about women, editors prefer articles about men.

Lets say for example there are 100 articles about men, 10 articles about women. The men's articles have on average 100 editors and 100 readers per article, the women's articles 10 editors and 20 readers per article. Women's articles have more readers per editor (2:1 vs 1:1), yet, readers prefer articles about men (10000 vs 200 readers)... Prevalence (talk) 12:05, 10 January 2016 (UTC)

• No mention of stereotype threat? czar 07:18, 13 January 2016 (UTC)

• Is there a statistic available for the percentage of established editors who don't state their gender? I'd like to find out if that's a small minority or a majority of the population. Thanks. Praemonitus (talk) 20:56, 15 January 2016 (UTC)
  • You can see the stats as of September 2015 here: Wikipedia:Database reports/User preferences#Gender. That's about 615K editors who declared a gender in Special:Preferences, out of all 27 million accounts (perhaps 8 or 9 million of which have ever made an edit here) at the English Wikipedia. The percentage of editors declaring a gender is likely to be higher at languages that use different words/endings for male and female editors. WhatamIdoing (talk) 23:08, 29 January 2016 (UTC)
    

The Signpost needs your help putting together the next issue.

Home

About

Archives

Newsroom

Subscribe

Suggestions