Google, Public Libraries, and the Deep Web
A February 2012 Pew Internet survey found that 91% of online adults use search engines to find information on the web (Purcell, Brenner, & Rainie, 2012). Asked which search engine they use most often, 83% of search users say Google.
This paper outlines a (not insurmountable) challenge to public libraries, which have a mandate to facilitate public access to information. A preliminary study reveals that many public libraries in Canada currently block their catalogues from Google’s indexing robots, effectively rendering their resources invisible to the majority of internet searchers. This fact undermines the purpose and functionality of public libraries’ Online Public Library Catalogues (OPACs). This paper will address the problem of dynamic content for search engine robots, and suggest solutions, such as the creation of sitemaps, as well as illustrating the need for future studies.
Anderson, R. (2005). The (uncertain) future of libraries in a Google world: Sounding an alarm. Miller, W. & Pellen, R. M. (eds.). Libraries and Google.
Binghamton, New York: The Haworth Information Press.
Artlitsch, K., O’Brien, P., and Rossmann, B. (2013). Managing search engine optimization: An introduction for library administrators. Journal of Library Administration, 53(2-3), 177-188.
Auletta, K. (2009). Googled: The End of the World as We Know It. New York: Penguin Group.
Badke, W. B. (2013). Coming Back to Google Scholar. Online Searcher, 37(5), 65-67
Baker, L (2008). Google now controls 69% of online advertising market. Search Engine Journal. Retrieved from:
Bell, S. (2004). The infodiet: How libraries can offer an appetizing alternative to Google. Chronicle of Higher Education, 50(24), B15.
Bergman, M. K. (2001). White paper: The deep web: surfacing hidden value. Journal of Electronic Publishing, 7(1), 1-37.
Blankenship, E. F. (2009). Who holds the keys to the web for libraries? Miller, W. & Pellen, R. M. (eds.). Googlization of Libraries. New York: Routledge.
Boston, T. (2005). Exposing the deep web to increase access to library collections. Proceedings of the 11th Australian World Wide Web Conference, Gold Coast, Australia.
Bozdag, E. (2013). Bias in algorithmic filtering and personalization. Ethics and Information Technology, 15, 209-227.
Cahill, K. (2009). An opportunity, not a crisis: How Google is changing the individual and the information profession. Miller, W. & Pellen, R. M. (eds.). Googlization of Libraries. New York: Routledge.
Cahill, K. and Chalut, R. (2009). Optimal results: What libraries need to know about Google and search engine optimization. The Reference Librarian, 50(3), p. 234-247.
Cothran, T. (2011). Google Scholar acceptance and use among graduate students: A quantitative study. Library & Information Science Research (07408188), 33(4), 293-301.
Devine, J. and Egger-Sider, F. (2004). Beyond Google: The invisible Web in the academic library. The Journal of Academic Librarianship, 30(4), 265-269.
Devine, J. and Egger-Sider, F. (2005). Google, the invisible Web, and librarians: Slaying the research Goliath. Internet Reference Services Quarterly, 10(3-4), 89-101.
Doughetery, W. (2010). The Google Books project: Will it make libraries obsolete? The Journal of Academic Librarianship, 36(1), 86-89.
Ettinger, D. (2008). The triumph of expediency: The impact of Google Scholar on library instruction. Journal of Library Administration, 46(3-4), 65-72.
Giglierano, J. (2008). Attitudes of OhioLINK librarians toward Google Scholar. Journal of Library Administration, 47(1-2), 101-113.
Google. (n.d.). Google’s Mission. Retrieved from:
Google. (2012). Code of Conduct. Retrieved from:
Halifax Public Libraries (2014a). Robots.txt. Retrieved from:
Halifax Public Libraries (2014b). Pride and Prejudice. Retrieved from:
Halifax Public Libraries (2014c). Sitemap.ashx. Retrieved from:
iProspect (2008). Blended Search Results Study. Retrieved from:
Jansen, B.J. & Spink, A. (2006). How are we searching the World Wide Web? A comparison of nine search engine transaction logs. Information Processing and Management, 42, 248-263.
Kumar, V. (2012). Exposing library catalogues to search engines. Journal of Library & Information Technology, 32(6), p. 493-498.
Levy, S. (2011). In the Plex: How Google Thinks, Works, and Shapes Our Lives. New York: Simon & Schuster.
Ludwig, M. (Winter 2003). Breaking through the invisible web. NetConnect, 8-10.
Miller, W. (2005). Introduction: Libraries and their interrelationships with Google. Miller, W. & Pellen, R. M. (eds.). Libraries and Google. Binghamton, New York: The Haworth Information Press.
OCLC (2010). Perceptions of libraries, 2010: Context and community. Retrieved from:
O’Leary, M (2005). Google Scholar: What’s in it for you? Information Today, 22(7), 35-39.
Onaifo, D. and Rasmussen, D. (2012). Increasing libraries’ content findability of the web with search engine optimization. Library Hi Tech, 31(1), p. 87-108.
Purcell, K., Brenner J., & Rainie, L. (2012). Search engine use 2012. PewResearch Internet Project. Retrieved from:
Robots.txt (2007). About. Retrieved from:
Sherman, C. and Price, G. (2003). The invisible Web: Uncovering sources search engines can’t see. Library Trends, 52(2), 282-298.
Wright, A. (February 22, 2009). Exploring a ‘deep web’ that Google can’t grasp. The New York Times. Retrieved from :
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution 3.0 License.