Virginia Libraries Logo

Virginia Libraries

Main Contact:
Ginny Pannabecker

Virginia Libraries
Volume 45, Number 3

DLA Ejournal Home | VALib Home | Table of Contents for this issue | Search VAlib and other ejournals

Anatomy of the Internet Reference Resources Web Page: a UVA Library Experiment

by Scott Silet


This just in: The number of Web sites surpasses 53 gazillion worldwide--number of reference librarians experiencing information meltdown attributed to the number of Web sites at an all-time high. While this isn't an actual headline (yet), I suspect many of us feel as though we read it every week. Keeping up with the deluge of information available on the Internet is a full-time job. Indeed, organizations like Yahoo, The Scout Report, and the Internet Public Library have sizable staffs whose sole responsibility is to survey the farthest reaches of the virtual universe and keep the rest of us informed about what is new and interesting on the Web. As the number of resources available over the Internet has grown, many reference librarians around the state have responded by establishing their own virtual collections of reference resources. (For samples of such collections see Virginia Tech's, "General Reference Resources," William & Mary's, "Reference Resources," Fairfax County's, "Reference Bookshelf," and Rockbridge Regional Library's, "Index of WWW Resources." The University of Virginia also maintains a collection called, "Internet Reference Resources" (IRR) which has existed in one form or another since the Spring of 1995. The following article is a brief history of that reference page, a discussion of how it is currently maintained and administered, and a listing of issues and challenges that face it and similar collections in the future.


As is the case with many such pages at other libraries, the IRR essentially started out as a series of handy Web sites which University of Virginia reference librarians found and book marked on their office or reference desk computers. As the number of these resources grew, we found they became increasingly useful in our work answering reference questions. In response to this trend, many of us created lists of Internet-accessible resources on our library's reference pages. Unfortunately for our users, most were difficult to find and poorly organized--indicating, perhaps, that they were designed primarily for ourselves and not our library users. Lacking any standards of style or development, the quality of pages varied widely. Some used a straight alphabetic listing while others attempted to annotate each site or to group items by subject, but what these lists gained in organization they frequently lost in comprehensiveness. The pages grew independently of one another over the next two years, a fact reinforced by the lack of communication between librarians who maintained them.

As the Web grew, the time needed to maintain these pages increased as well. Realizing that the task of maintaining this collection was better handled by a centralized effort, I proposed in the spring of 1997 that we consolidate our efforts and create a single page that would serve all UVa libraries. An Advisory Committee made up of six reference librarians (many of whose pages were integrated to make IRR) was established to coordinate the creation, design, and maintenance of IRR which officially went public in the Fall of 1997.

Page Design

What's in a Name?: One of the first tricky issues the Advisory Group had to resolve was what to name the page--a seemingly simple task which ultimately proved to be somewhat problematic. Many variations on a theme for reference pages existed already: Virtual Reference Desk, Electronic Reference Shelf, On-line Reference Works, Reference Sources on the Internet, etc. (See for the variety of names used at other university libraries.) We all felt that no one name adequately described the unique nature of the collection, specifically how it differed from the Library's primary (fee-based) collections of resources accessible on the Web, including InfoTrac, Lexis-Nexis, and Encyclopedia Britannica. After much debate, we settled on Internet Reference Resources--our thinking being that that name would best represent a collection of resources which had come from "out there" on the Internet and not owned by (i.e. from within) the Library.

Categories: The Advisory Group wrestled with several items related to the organization of the page starting with establishing a coherent and logical subject hierarchy (see image #1). We settled on five main categories and concluded that the top-level page should display general resources sources--materials which would most likely be of interest to users in all libraries. We chose four other subject-specific categories (Arts & Humanities, Engineering, Science & Medicine, and Social Sciences) to reflect the major areas of study at the University. The five categories, six if you consider the "New" category which was added later, were color-coded to give users a sense of location within the complex IRR structure. We then divided each category into a series of subcategories (topics).

Topics: We created the list of topics for the General category after looking at our existing resources as well as by looking at those topics used by other college and university reference pages. Topics for the four subject-specific areas, on the other hand, relate directly to the fields of study at the University and are named primarily after academic departments. Several fields have a large number of resources (government/foreign affairs-64, business-58, travel-43, and books-32) so we chose to further subdivide these topics into logical and useful groupings (e.g., geographical, chronological, material type, etc.). Once topics have been created and subdivided, items are then organized according to their usefulness or popularity relative to the grouping. Here, we rely on our own experience using the page to determine the most beneficial order of entries which does, it should be noted, change from time to time particularly with time-sensitive materials. Entries may be cross-listed between categories and/or topics, if appropriate. If a source lends itself well to more than two topics within the same category, it is placed in a special "general/disciplinary" topic heading.

Annotations: Annotations in IRR describe the content and evaluate the usefulness of a resource. As a general rule, annotations should alert the user to the authority, comprehensiveness, currency, and ease of use of an item as well other things like the existence of search engine or index. They should also highlight any additional resources such as glossaries, source lists, or advanced searching forms which might prove helpful when using the page. Equally as important, annotations serve as words which are indexed by the IRR's internal search engine; therefore the selector/annotator needs to use words which users might employ to describe and find that item.

Layout: We were fortunate to have a number of talented individuals within the Library to assist with the graphical and mechanical design of the IRR. Our Publications Office helped with the design of the left-hand frame and category/topic headers. I should add that it was only after some discussion that the Advisory Group accepted frames as the best way to organize the IRR, and we haven't looked back since. This Office also provides us with a regular report of broken, orphaned, and redirected URLs using LinkBot. We have a graduate student assistant fix outdated links on a monthly basis. PERL-meister Patrick Yott, director of our Geo-Statistical Center, also assisted us by writing a series of complex scripts to update the graphical, navigational, and functional changes required by each of the dozens of HTML files that make up the IRR.

Collection Development

It is important to have guidelines governing the selection of library materials, and Web reference sources are no exception. Some institutions, like the University of Oregon, have established collection development policies specifically for Internet resources (See While UVa has not gone to these lengths, selectors contributing to IRR use many of the same guidelines for inclusion that govern our print reference materials. These include, but are not limited to:

Purpose/Audience- With few exceptions, sites included in IRR support the instructional and research programs of the University. If the resource is general or basic in nature, then it will be included if it is perceived by the selector to be of interest to our faculty, students, and staff. If there is some doubt as to the appropriateness of a resource to the collection then the determining scenario is "Might someone within the University community ask us a question which this source would help answer?"

Scope- The resources included must serve to answer a factual or quick reference question, just as if it were a title added to our print ready reference collection. The relatively narrow scope of the IRR is what differentiates it from other library or commercial reference pages. It generally differs from its colleagues in one of two ways: those sites only include general, not subject-specific, resources or they include sites that would not be considered ready reference in nature by our selection criteria. The site at created by Jian Liu and now maintained by Anne Graham, is an excellent example of the former and, The Librarian's Index to the Internet, is an equally good example of the latter. I survey both regularly to supplement our collection.

Access- In order to be included in IRR, it is necessary that reference resources either be freely-accessible or free with a simple registration. No detailed registrations or ones that require personal information are included because it is not the intention of the IRR to profile our users for Web capitalists any more than browser cookies already do. Resources appearing to be free for limited time are included at the discretion of the selector and noted in the site's annotation. Only in a few instances do we list subscription-based Web databases, another characteristic that distinguishes our page from those of many other libraries. There are many reasons for doing this. Even now, the majority of the Library's databases cannot directly be linked because they are part of a suite of databases that are (IP) authenticated at the collection level, and this can be quite confusing. Additionally, more than half of our users are not affiliated with Uva, and the Advisory Group considers it bad form to include resources to which not all have access. Lastly, we decided it that keeping up with what was available "outside" the Library provided us with a sufficient amount of work for now.

Authority/Accuracy- Whenever possible, we verify information within a reference Web site against print sources to verify the accuracy of the data. Authorship, if known or not apparent from the URL, is typically included in the annotation.

Currency- Ideally, sites for inclusion are regularly updated. Exceptions to this rule include resources such as historical works (biographies, bibliographies, etc.). Notes on currency, if known, are added to the site's annotation.

Arrangement/Ease of Use- The content of resources included in IRR should be organized to allow the user to easily extract needed information. Poorly designed sites, in spite of the utility of the information they contain, are regularly excluded. One of the keys to a good site is the availability of a good index or search engine. When linking to a site, we try to lead users directly to a search engine or index screen, rather than placing the onus of finding the resources on them. As a general rule, we do not include resources that are produced and viewed with Adobe Acrobat (.pdf) since these files are typically large with no logical or apparent means of internal navigation other than a search tool. Additionally it's not possible to link to a specific item such as a table or an appendix within a .pdf file, and this also limits utility.

Access & Use of IRR

One of the perpetual challenges we face with the IRR is promoting the wide variety of resources contained in it--whether they be a popular resource of local interest like the Charlottesville Movie Guide or a nationally-known scholarly resource like the Handbook of Latin American Studies. Since it came into being two years ago, the IRR has benefited greatly from its location on the top-level of the Library's homepage (serendipity continues to play a role in our Library's collections). Many patrons also learn about IRR when we use it to help answer their questions at the reference desk. Additionally, whenever appropriate, we use it in classes taught through the Library's instruction programs.

Another source of user awareness comes from Virgo, our on-line catalog. Each of the resources in IRR is cataloged, many of them courtesy of Jackie Shieh--a former member of our Cataloging Department (See image #2). We have no data as yet indicating how many people find and use these resources while searching our library catalog, but it might prove useful to look into this.

Use Statistics: Currently the top-level IRR page receives roughly 5,000 hits per month (up nearly 500% since July 1997), approximately 40% of which come from within the University. The top eleven most frequently accessed areas during April, 1999 were:

1). General-Phone (816)
2). Arts & Humanities (651) - the most popular topic within this category was English Language & Literature (229)
3). General-Encyclopedias (603)
4). General-Style Manuals/Grammar (496),
5). General-People/Biographies (371)
6). General-Books/Publishers (345)
7). General-Libraries/Archives (339)
8). Humanities-Popular Culture (movies, television, music) (334)
9). Social Sciences (334) - the most popular topic within this category was Government & Foreign Affairs (212)
10). General-Dictionaries (323)
11). General-Local (C'ville, UVa, Virginia) (284)
General topics get substantially more use, we presume, because their scope is useful to a broader range of users and because these topics are also what the user sees first after connecting to the page. We use a software program called NetTracker to compile a series of use statistics including number of views by month and day, average time viewing a page, plus the user's two and three-digit domain name extensions (i.e. .edu, .uk), browser, and platform. This data is useful in determining who comprises our user base, what resource areas they are using, and, more generally, where we should be focusing our efforts in the future.

For those users who do not want to browse the subject hierarchy for a resource, we offer a moderately useful search engine, Webinator, which indexes every word in the site and is updated weekly by our Library systems department. Unfortunately, this search engine is not entirely sufficient for our needs because it retrieves only the name of topics, not individual items, which contain the user's search term. This forces users to search that topic page to find the specific resource. We are currently investigating using a more sophisticated mark-up language (SGML) which would allow for much more effective user search and retrieval. SGML will also allow us to track which individual resources are being used.

Administration & Maintenance: While suggestions for resources come from many different areas of the library, updates to the IRR are done primarily by two reference librarians, one covering the Sciences, Medicine, and Engineering and one covering the rest. We are currently working on a web-based form generated by a complex PERL script which would allow users and librarians to suggest resources for inclusion on-line. Entries to IRR are added on an as-find/have-time basis (generally 2-4 times a month). We added a "New" category about a year ago to highlight recent entries. The IRR's custodians themselves find these categories particularly useful when scanning comprehensive reference sites like the "New This Week" section of (Librarian's Index to the Internet), and, at the same time, to provide our Cataloging Department with a current list of resources to be added to the library catalog.

Immediate/Long-term Challenges

User awareness is, arguably, the key to the success or failure of reference web pages which, in turn, justifies the amount of effort that goes into building and maintaining them. Awareness of these resources starts at home. It is important that UVa reference librarians regularly scan newly added resources to IRR, just as we do to keep up with new print and subscription-based electronic resources. We must also continue our efforts to publicize throughout the library those new resources which are of interest to us all.

Keeping up with the deluge is a problem that will most likely persist for IRR selectors indefinitely. One response is to continue to identify high-quality current awareness resources like the Scout Report and the Librarians Index to the Internet to assist in this discovery and selection process. Another is to investigate building consortial collections which would enable librarians from peer institutions to contribute to a shared resource or, at the very least, develop a mechanism by which we share our discoveries with each other. These could take the form of a collective web page or a current awareness list-serv to highlight new reference resources.

Another challenge for reference librarians is to use our unique knowledge of the Web to compile or create new Web reference sources. For example, we received a reference question a few months ago from someone who needed to know the individual companies that made up several major stock indexes. We could not find a source (print or electronic) that compiled this data, but in the process of searching we found that many individual indexes had this information available on their Web sites. We then brought these lists together in one resource on the IRR called Stock Index Directories ( --> Social Sciences --> Business --> Stocks - #20).

As the beginning of the 21st century approaches, we find that nearly everything related to the library and its collections are in a state of flux. Here are a few things that will characterize that state: users will be increasingly defined by who comes in through our homepages rather than our doors, the number of reference resources on the Web will be inversely proportional to the amount of time our users have to find them as well as our ability to collect, organize, and make it all accessible, and resource management in the dawn of the information age of must take place concurrently with the building of electronic collections and not be an afterthought. Resources like the Internet Reference Resources page are poised to make a valuable and lasting contribution to a library's reference collection, but we must meet these challenges head-on in order to ensure our continued success.

Scott Silet is Instruction and Information Services Librarian at the University of Virginia.

DLA Ejournal Home | VALib Home | Table of Contents for this issue | Search VAlib and other ejournals