Using Google Scholar in Anime/Manga Studies

Since its launch ten years ago, in October 2004, Google Scholar, Google’s “free service [that] helps people search scholarly literature” has made a major impact on how we search for and access academic publications. Some things about Google Scholar are certain. Students at all levels from high schools to graduate departments – rely on it. Educators and professional researchers accept it. And library and information science scholars work to understand, analyze, and describe it. So, how does Google Scholar hold up as a research tool for anime and manga studies?

One more certain thing about Google Scholar that it does have a quite a few strengths:

  • First and foremost, it is free to use and open to any Internet user, without the need to log in through another website or a paid subscription.
  • Google Scholar is also intuitive and familiar – it’s not particularly different from the basic Google search page, and does not require learning any new search techniques.
  • Google Scholar is “comprehensive” – it covers a wide range of content, from different sources, in different languages. It is not limited to the holdings of a particular library, the publications of a particular publisher, or materials on a particular area, subject, theme or topic.
  • If and when possible, Google Scholar offers access to the full text of articles, not just to records for them on a journal website or in an institutional repository.
  • It is integrated with several other Google products.
    • One of the most obvious is Google Books – Google Scholar searches the actual texts of many of the titles that are available in Google Books, not just the titles and their associated bibliographic records. This is in sharp contract to searching library catalogs, that traditionally only represent some details about particular books, and standard bibliographic databases, which often do not cover books (and especially essay collections) at all.
    • Another is the Google Scholar Author Profile, a way for readers to access all of the writing indexed by Google Scholar that is written by a particular author, view the author’s citation statistics, and most importantly, access some details about the author, such as his or her academic affiliation and areas of interest. A link to the author’s personal website may also be available if the author chooses to provide one. So, for example, searching in Google Scholar for Srividya Ramasubramanian, the author of Japanese anime heroines as role models for U.S. youth: Willful identification, parasocial interaction, and intercultural entertainment effects, in Journal of International and Intercultural Communication, 5(3), 189-207, returns not only links to her publications (and publications that she is mentioned in), but also, a way for readers to find out more about the author and contact her directly. This again is unlike the standard practice in other academic databases, where it’s usually possible to simply retrieve all the records that share a common author, but no way to find out any details about the author themselves.
  • Finally, despite it being a free-to-use product, Google Scholar provides several “value-added” features, such as a way to access materials “related” to any particular publication that it indexes, and at least some of the publications that cite to it.

At the same time, librarians who compare Google Scholar to other academic databases have identified many of its major shortcomings. Gail Herrera (2011) notes many of them in Google Scholar users and user behaviors: An exploratory study, College & Research Libraries, 72(4), 316-330, as do Andrew Asher, Lynda Duke, and Suzanne Wilson (2013) in Paths of discovery: Comparing the search effectiveness of EBSCO Discovery Service, Summon Google Scholar, and conventional library resources, College & Research Libraries, 74(5), 464-488. Some of these include “limited advanced search functionality, incomplete or inaccurate metadata, inflated citation counts, lack of usage statistics, and inconsistent coverage across disciplines”, as well as a lack of any concrete information about Google Scholar’s actual scope of coverage.

So, taking these shortcomings into account, how useful is Google Scholar in locating scholarly publications about anime and manga.

A simple and straight-forward search in Google Scholar for the keyword ‘anime’, limited to results published in 2013 and 2014, returns an overwhelming “about 19,800 results”. Looking at these in-depth, though, some things are obvious right away. Of the 10 results on the first page, two are for actual scholarly books on anime (although one of these was published in 2012), and three more for articles in established peer-reviewed journals. Two more are to non-academic/popular titles, including an essay collection that was actually first published in 2010, One result is for an article in the obscure at best Online Journal of Communication and Media Technologies, and another is a record for a conference presentation in a Japanese university’s institutional repository. Finally, one more item is a “false positive” for the purposes of anime and manga studies – the word “anime” in the title of an article referring to something else other than Japanese animation.

For the next ten, the break-down is much worse – one false positive, three book reviews, one “citation” – a mention of a book chapter, without a direct link, and four citations (again without links) to books by HMA Erotic, published by “Wreckreation Publishing” – I think it’s safe to say that none of the four qualify as scholarly publications. The final result is an abstract of a conference presentation.

Narrowing the search to the top ten results of the “about 13,600” published this year does not improve the result – two books (one of them non-scholarly, and the other published earlier than 2014), three book reviews, two presentations, the same citation to a book chapter, and two mentions of HMA Erotic titles.

Changing the scope of the search to manga, again for 2013-2014, returns “about 17,200” results. The very first one is relevant – but incorrect – it’s for a collector’s edition of Fred Schodt’s Dreamland Japan: Writings on Modern Manga – which was published in 2011. And none of the next nine are useful – “Manga” is also a last name shared by several authors in fields that have nothing to do with Japanese comics. The same breakdown holds on the next page – nine false positives, one book on manga that was published in 2011.

Finally, removing the chronological limitations entirely and searching for ‘anime’ as a keyword across all publication dates produces significantly different results – “about 308,000”. All ten on the first page are relevant – seven to books, and three to journal articles. However, one of the results is for a link to a pirated copy of a scholarly monograph, and two are to non-academic titles.

Would the results be different with a more sophisticated search, for example, one that introduced an additional keyword? Searching for ‘anime’ and ‘Japan’, limited to items published since 2013, returns approximately 2,470 results. Of the first 10, one is an actual article on anime in a peer-reviewed scholarly journal, one more is a citation to an article, four are book reviews, two results are non-academic books, one, a book that was published outside the search range, and one more, a book review of a title on Japan, but not on anime/manga.

For comparison, searching for “anime” in the ProQuest Research Library, a standard electronic resource that is available in most academic (and many public) libraries, the same search – anime, restricted to results published after January 1, 2013, returns 199 items. Of the first twenty, 7 are articles on anime in peer-reviewed scholarly journals, and 11 are reviews of scholarly books on anime. Two are “false positives”, and one result is a short non-peer-reviewed item in a scholarly journal.

Search summaries:

Google Scholar

Search term: Anime
Range: 2013-2014
Total results: 19,800+

Results 1-10:

  • 3 articles on anime in established peer-reviewed journals (+1 in an online-only open access journal)
  • 2 scholarly books on anime (one published before 2013)
  • 2 non-scholarly/popular interest books on anime (one published before 2013)
  • 1 conference presentation on anime
  • 1 “false positive” (different meaning of the term “anime”)

Results 11-20

  • 4 citations to titles identified as e-books published by “Wreckreation Publishing”
  • 3 reviews of books on anime
  • 1 citation to a chapter on anime in a scholarly/edited essay collection
  • 1 false positive
  • 1 abstract of a conference presentation on anime

Range: 2014
Results 1-10

  • 3 reviews of books on anime
  • 2 books on anime (one non-scholarly, one published prior to 2014)
  • 2 citations to Wreckreation Publishing titles
  • 2 presentations on anime
  • 1 citation to a chapter on anime in a scholarly/edited essay collection

Search term: Manga
Range: 2013-2014
Total results: 17,200+

Results 1-10:

  • 9 false positives
  • 1 non-scholarly book on manga published in 2011

Search term: Anime
Range: all years
Total results: 308,000+

Results 1-10:

  • 7 scholarly books on anime
  • 2 non-scholarly books on anime
  • 1 link to pirated PDF of scholarly book

Search terms: Anime AND Japan
Range: 2013-2014
Total results: 2,470+

Results 1-10

  • 4 reviews of scholarly books on anime
  • 2 reviews of non-scholarly books on anime
  • 1 article on anime in a peer-reviewed/scholarly journal
  • 1 citation to article
  • 1 article on Japan (not anime-related) in a peer-reviewed scholarly journal
  • 1 book published before 2013

ProQuest Research Library

Search terms: anime
Range: 2014
Results: 199

Results 1-20

  • 11 reviews of books on anime
  • 7 articles on anime in peer-reviewed scholarly journals
  • 1 general/non-scholarly article
  • 1 false positive

Discussion

One thing can be said right away, and with certainly – if you want to find some kind of published commentary on anime/manga and don’t particularly care about the specifics of what you will find, Google Scholar will be an adequate resource for your purposes.

However, overall, the shortcomings inherent to Google Scholar as a research tool are clear. At least in a search for broad, fairly general terms:

  • Many results will be false positives: I.e., results where ‘Manga’ is an author’s last name, or where ‘Anime’ refers to something other than Japanese animation.
  • Many results will be materials that are not scholarly publications – conference presentations, dissertations, paper prepared for class, and general-interest books
  • Some results will not be “publications” in the usual sense at all, such as Wikipedia articles
  • Some results will obviously have been included in error – as is the case with the “Wreckreation Publishing” results in my sample searches.
  • In searches with additional limiters such as time periods, some results will fall outside those ranges.

Of course, though, these shortcomings must be balanced against Google Scholar’s many strengths. Again, as mentioned earlier, the biggest one of these is that it is a resource that is entirely free to use – it does not require access to an academic or even public library, simply access to the Internet. Another strength – one that I do not talk about here at length, but one that I am certainly aware of (and one that I have seen mentioned in the relevant literature) is Google Scholar’s utility when searching for materials containing unique, highly unusual terms – such as the name of an anime director or manga author, or the title of a particular anime series.

Suggestions for Further Research

I will also be the first person to admit that the analysis in this post is extremely basic. I use only a few search terms, and only compare Google Scholar to one other electronic resource. A more intensive study would build on the models used by researchers like Karen Chapman (2002), Full-text database support for scholarly research in finance, Journal of Business and Finance Librarianship, 7(4), 35-44, Marie Speare (2010), Environmental microbiology: A database coverage study, Science & Technology Libraries, 29(1-2), 1-15, and particularly, William H. Wauters, (2007). Google Scholar coverage of a multidisciplinary field, Information Processing & Management, 43(4), 1121-1132, to specifically compare the way that “anime and manga studies” as a field is covered across a number of databases – including Google Scholar.

Another approach could specifically survey people who use Google Scholar to conduct research on anime/manga – academics and educators, graduate students, undergraduates, and the public – about their perceptions: this is similar to what Tanya Cothran (2011) finds in Google Scholar acceptance and use among graduate students: A quantitative studyLibrary and Information Science Research, 33(4), 293-301.

And a third could simply focus on one other particular aspect of Google Scholar that is sometimes not immediately obvious – as with all other Google products and services, it is always in flux, and at least hopefully, improving every day. The results I get for a search today will be different from those for you tomorrow, and utterly different from those for either of us a year from now. So, a “longitudinal” study of this kind is also a possibility – for an example, see Jacso, Peter (2008), Google Scholar revisited, Online Information Review, 32(1), 102-114.

So, in conclusion, really all that I can say as a scholar, a researcher, and an information professional is that I cannot, in good faith, completely warn you against using Google Scholar in your work. But, I would very strongly suggest that you do not rely on it entirely, and even to the extent that you do rely on it, you should be aware of its limitations, faults, and shortcomings – many of which are actually inherent in its very design and purpose.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s