Wikifying scholarly canons

Francis Bacon’s New Atlantis, published in 1627, introduced the bold idea of a world in which humans developed technology and accumulated knowledge in support of the common good. His book helped inspire the Age of Enlightenment and the idea that we could look to the present, and not only the ancients, to better understand ourselves and the world. It particularly inspired Robert Boyle (1627-1691), who in the mid-17th century began a practice of publicly demonstrating his findings in his then new “(e)laboratory”, a large venue designed to allow others to observe the apparatus he used and validate his results [1].

Before Boyle, most of those who tinkered with the natural world were alchemists who hid the details of their work and then made grand claims about the phenomena they had privately “discovered”.  What Boyle realized, though, was that the kind of progress found in New Atlantis required the establishment of commonly accepted facts. Work done in secret contributed little, and without a detailed explanation of the methods involved, any results were viewed with suspicion and were nearly impossible to verify or reproduce. By having respectable witnesses (at the time literally aristocratic “peers”) scrutinize his experiments and vouch for his results, Boyle initiated a departure from the alchemist’s closet [2].

As an alternative to requiring actual witnesses to validate every scientific result, Boyle developed conventions of modesty and detail in scientific reporting. These conventions were widely adopted and succeeded in describing methods and results with enough clarity to inspire trust and permit replication, thereby allowing scientific knowledge to build on itself. But standardized scholarly communication also had secondary benefits: thanks to official archival publication, researchers could claim credit for their discoveries; papers themselves took on a more standard form, making them more useful and accessible to readers; and editorial review helped to filter out redundant or unimportant reports.

Ever since the British Royal Society‘s founding of the Philosophical Transactions in 1665, the scholarly article and monograph have continued to serve as the basic units of contribution, persisting even as science has professionalized and media turned electronic. Like the works of the classical period that so preoccupied the pre-Enlightenment academy, the cultural institutions of scholarly articles and monographs themselves have stood the test of time. Indeed, they have proliferated: since 1665 the number of peer-reviewed journals has increased at a relatively steady compound annual growth rate of between 3.2% and 4.4% [3], with 24,000 of them in existence in 2010 [4]. As of 2012 more than 1.5 million refereed articles are published yearly [5], and since most journals have impact factors of 1.0 or less, the majority of articles published today will, within two years, have been cited at most once [6].

“Nullius in verba”

Many academics would report that a love of research and sharing knowledge is what motivated them to become scholars and scientists. They nearly always give away the articles that result from their work, and are rewarded by the benefits to their career (notably hiring and tenure) from increased prestige.  Given that the purpose of scholarship is the advancement of human knowledge, one might imagine that scholars would jump at the opportunity to make accurate information on a topic they care about easily accessible. Yet for the most part, academics continue to publish original research within journals that restrict access, and have thus far not engaged substantively with, for instance, Wikipedia.

When academics have been asked why they do not contribute to Wikipedia, or why they do not make their data more easily available, or why they continue to avoid new “open access” publication venues, one of the most common explanations is “not enough time” [7,8]. Academic hiring, tenure, and grant review boards preference most strongly the publication of scholarly results in prestigious outlets. In the sciences these are journals such as Nature and Science, and in the humanities a book published by a selective university publisher [7]. By communicating through such media, an academic can be sure that his or her peers will see (or at least, should have seen) the researcher’s contribution. By contrast, contributions to Wikipedia, or to writing popular blog posts, generally accord no academic recognition to a contributor. This may help to explain why, for those without access to institutional journal subscriptions, the claims of today’s academics may at times appear as trustworthy as those of the pre-Enlightenment alchemists.

An approach

A potential solution is the creation of a scholarly wiki-based encyclopedia, one that recognizes the existing imperatives of academic communication. As a wiki, every entry would be a “living review”, subject to further revision as the field advances. Unlike Wikipedia, however, contributions to a scholarly wiki-based encyclopedia would be targeted toward a scholar’s peers rather than the general public. Each entry would undergo peer review and archival publication. Once published, entries would be subject to curation by a recognized and established scholarly authority on the topic. Though the articles themselves would not constitute original research, site-based scholarly discussion regarding potential revisions would no doubt make reference to recent (and perhaps even original) arguments and data.

By making each review article a public resource and providing tools for scholarly commentary and annotation, the articles themselves would become central venues in which scholars could be assured that any contributions would be visible to their peers. When even the most minor contribution has an immediate global impact, each contributing act brings with it strong intrinsic rewards, a fact that is responsible for motivating the hundreds of thousands of volunteer editors that built Wikipedia.

A central scholarly encyclopedia would also serve to empower the academy in the eyes of the public by making obvious the extent of academic consensus on a range of topics. This might be especially helpful in the social sciences and humanities, which can seldom deploy the power of demonstration to convince a broader audience. By consolidating vetted, trustworthy scholarly opinion, the academy might unify online and thereby retake ground currently claimed by special interest groups and public relations firms.

Yet perhaps the greatest advantage of a scholarly wiki-based system is the ratchet-like nature of its development: each article need only be written once, and thereafter serves as a permanent, living part of an online canon. Because contributing provides an immediate benefit to authors in the form of a published, citable, peer-reviewed scholarly article, such a resource need not rely on pure altruism for its growth. While career benefits to junior faculty contributors might be minor early in the encyclopedia’s development, advanced undergraduates and graduate students would nonetheless have strong incentives to team up with senior faculty co-authors, especially if this meant that a student could derive a peer-reviewed publication from an article that had already been written for course credit.

Why would scholars pay any particular attention to the content of wiki-based scholarly reviews and the commentary that accompanies them? In a word: prominence. Scholarpedia, an attempt to create precisely the kind of venue described here, demonstrates excellent performance in search engines. When titles from its articles are searched for, Scholarpedia entries typically appear on the first page of results, often ranked within the top five (e.g., at the time of writing, Geoffrey Hinton’s review of “Deep belief networks” appears in the top position in Google when its title is queried). It is thus in a scholar’s self-interest to, in the near term, use Scholarpedia as a venue for publishing reviews because it is a reliable means of gaining global visibility.

To predict the longer term impact of developing a large number of wiki-style review articles it is informative to look to Wikipedia. Because of the high prominence of Wikipedia results, and because of the limitations of the software Wikipedia uses, the Wikipedia community has needed to develop norms that discourage conversations about the article’s contents [9]. While the audience for scholarly review articles is smaller than the audience for lay Wikipedia articles, the immediacy and reach of article commentary is identical, creating strong incentives for living scholarly review articles to additionally serve as venues for informal scholarly discussion. Through better structuring and community filtering of conversations, each online forum should prove at least as informative as the content of the article itself.

One potential outcome of marrying the dynamic collaborative revision of online review articles with ongoing goal-directed scholarly discussion is the provision of an alternative, and more incremental, venue for presenting and vetting a range of scholarly contributions. For instance, in the course of a discussion regarding a revision to a review article’s main text, a scholar might marshall computer code, post charts, reference online databases, and even present raw data. As in the popular collaborative programming website GitHub, simple visualizations of the extent, significance, and frequency of contributions could then provide alternative metrics of impact.

Challenges

Perhaps the greatest concern related to the development of an online wiki-style scholarly encyclopedia is who has authority — a concern that, fortunately, has already been successfully addressed in the development of printed encyclopedias. Scholarpedia’s approach is simple: it begins with an invited base of Editors and article Curators that have established reputations within their field. By combining this base with a system of public vouching for both authors and titles, authority is delegated to scholars themselves, adopting a distributed structure akin to that of the current academy. In effect, if respected scholars are willing to stake their reputations on the existence and content of a scholarly review article, then the review article merits inclusion.

There are additional epistemological and ontological issues concerning what should qualify as a potential scholarly review. As in Wikipedia, certain entries will no doubt lead to “disambiguation” pages (e.g. “language”). One might ask, for example, if there should be one article on “depression” (as a medical condition) or if instead “depression” should be an entire category of articles. There will no doubt be many other such challenges — here we simply defer to the judgment of the community as the site develops.

The largest challenge, however, is a practical one: the sooner scholars and scientists each publish a summary of an area of expertise online, the sooner we all receive a permanent benefit. Scientific societies and universities might encourage the rapid production of such articles by offering lucrative prizes, awarded after publication to the best review article published, as judged by a jury of experts or the appreciation of the public. Health-oriented foundations might fund similar competitions, motivating expert academics to create a persistent information resource that would accelerate the search for treatments. Such competitions might even be employed by corporations that have an interest in determining scholarly consensus on a topic of interest to them.

If not now, when?

Scholars are often asked to write and present similar treatments of the same information in different contexts. An online wiki-based scholarly encyclopedia, however, only requests the contribution of an article once. If every established scholar were to write one brief review on a topic of their expertise, we could produce a dynamic, permanent, accessible, scholarly, and authoritative online encyclopedia in a matter of months. It would instantly become one of the most useful resources ever created, and its benefits to humanity would only increase as its entries were revised and its contents elaborated.

After Gutenberg built his printing press it took more than 200 years before the first scholarly journal was established. The chief obstacle to learned inquiry in those times was not inadequate media, but instead the practices and conventions of researchers. Boyle solved the problem of his age by introducing new modes of scholarly communication that encouraged greater openness and trust, and in doing so helped spark the Age of Enlightenment. Perhaps the solution to our challenge is similar.

If you’re an established researcher interested in summarizing an area of your expertise, or if you would like to write an article in collaboration with someone who is, we’d love to see you propose an article.

  1. Shapin, S, and Schaffer, S. “Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life.” Princeton, N.J.: Princeton University Press, 1985.

  2. Mason, Stephen F. “A History of the Sciences.” New Revise. New York, New York, USA: Macmillan Publishing Company, 1962.

  3. Mabe, Michael. “The Growth and Number of Journals.” Serials: The Journal for the Serials Community 16, no. 2 (2003): 191–197.

  4. Campbell, Robert, and Alice Meadows. “Scholarly Journal Publishing: Where Do We Go from Here?” Learned Publishing 24, no. 3 (July 1, 2011): 171–181.

  5. Ware, Mark. “The stm report: November 2012”. Int’l Assoc. of Sci. Tech. & Med. Pub., 2012.

  6. Althouse, Benjamin M., et al.Differences in impact factor across fields and over time.” Journal of the American Society for Information Science and Technology. 60.1 (2009): 27-34.

  7. Harley, Diane, Acord, Sophia Krzys, Earl-Novell, Sarah, Lawrence, Shannon, & King, C. Judson. (2010). Assessing the Future Landscape of Scholarly Communication: An Exploration of Faculty Values and Needs in Seven Disciplines. UC Berkeley: Center for Studies in Higher Education. Retrieved from: http://escholarship.org/uc/cshe_fsc

  8. “Research:Expert Participation Survey.” Wikimedia, 2011. Accessed 16 October, 2012. http://meta.wikimedia.org/wiki/Research:Expert_participation_survey.

  9. “What Wikipedia Is Not.” Wikipedia, Accessed 17 November, 2012. http://en.wikipedia.org/wiki/Wikipedia:What_Wikipedia_is_not#FORUM.

By: Leo Trottier and Eugene Izhikevich

Acknowledgements: John Jacobson

Leave a comment

13 Comments

  1. So do you think that scholars should change their behaviour, or that a new online encyclopedia should be created?

    Both Scholarpedia and Wikipedia could potentially fulfill the role of “wikifying scholarly canons”, but neither did so far. Maybe we need something in between, a Wikipedia where academics could get credit and prevail over laypersons, or a Scholarpedia where everyone could more easily contribute. (After all, most edits need not be done by specialists.)

    Reply
    • Scholarpedia is designed to be editable by everyone, not just experts. However, the expectation is that only established scholars and scientists can assume the role of Curator for an article.

      I love Wikipedia, but it is not of the academy, at least not yet.

      Reply
  2. I am surprised you did not mention “Living Reviews in Relativity,” http://relativity.livingreviews.org/ – “Living Reviews in Relativity is a peer-reviewed, full open access, and exclusively online journal, publishing freely available reviews of research in all areas of relativity.”

    In places, it almost sounds like you are describing it…

    Reply
  3. While I have written that I doubt any such “one site to rule them all” approach could work (http://www.ideationizing.com/2012/02/dsrci-distributed-scientific-research.html), I do wish you luck.

    Perhaps you might make the best contribution to the expansion of world knowledge by promulgating your system rather than promoting your site. By this, I mean encouraging others to create a multiplicity of “sister sites” which all follow the same rules but which may specialize in different topics or merely reside on a particular university’s server. If a standardized set of rules was created for the naming of servers and the creation of links, then it would be possible for all these different sites to behave as if they were one. This would create a world-wide community of Scholarpedias rather than a single ivory tower. As I have written in my blog (and as mentioned my Michael Nielsen in his TED talk on the topic (http://www.ted.com/talks/michael_nielsen_open_science_now.html)), scholars may be hesitant to put all their eggs in one basket, owned by someone they don’t know, such as you propose here, but they may be willing to put their eggs in certain baskets owned by their employers. As long as all those baskets have a matching and compatible system, then the net effect is the same but with much more participation. And, in the end, isn’t that what you are really after?

    Reply
    • Thanks! Great comment! We indeed tried to write the article as a way of advocating for Scholarpedia’s approach, if not Scholarpedia itself, but many people took it as the latter.

      I think basing these things within the universities themselves might, indeed, be a smoother way to transition. But (1) this fails to provide the level of prestige that a more central venue would, (2) it doesn’t help deal with the problems of redundancy and the over-abundance of scholarly niches, and (3) university-based topics won’t likely survive their curators.

      Reply
  4. Thanks for the insightful post. One of the most fulfilling things I’ve done as an academic was to update the Wikipedia entry relating to my dissertation work. A while ago I set up a website which academics can use if they are interested in putting their scholarship up on Wikipedia. Of course there is no reason not to put it up on Scholarpedia as well.

    Reply
  5. Reblogged this on the Assault of Nimbosa and commented:
    this is the Great Work!

    #GreatWork

    Reply
  1. Could We “Wikify” Scholarly Canons? | Tech247news
  2. MathEd.net wiki | The Aperiodical
  3. Friday Morning JETLawg: E-Currency, E-Discovery, E-Law, and Tech Industry | JETLaw: Vanderbilt Journal of Entertainment & Technology Law
  4. hypertext | Learning Document

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 820 other followers

%d bloggers like this: