Counting What Counts
By Joseph Janes
American Libraries Columnist
Assistant Professor, Information School, University of Washington.
intlib@ischool.washington.edu
Column for May 2003
My first library job taught me so much, and I continue to marvel at the ways in which that experience comes back to me, even now. It’s been more than 20 years since I first stood behind the desk of the Oneida (N.Y.) Library, learning everything from the mechanics of circulation (using one of those old photographic microfilm systems that took pictures of the book card and a slip with the due date on it) to reference and readers’ advisory and all the rest of what goes on in a small-town public library. It was great fun; there are days I’d happily go back.
One of the things I was taught there was how to keep statistics of reference transactions. In those days, some libraries in New York received state funding for reference materials based on the number of questions they answered. When I started on the desk, the director mentioned this to me and then in a semi-nonchalant way suggested that I record every question I got. “Where is the bathroom?” “What time do you close?” Needless to say, we had a pretty terrific reference collection.
So I’ve always had a somewhat jaundiced eye when it came to library statistics of any kind, especially reference statistics. In general, we’ve counted what was easy to count. Just about everybody counts circulation figures and holdings (and the first person who gives me the ideal definitions of “books” and “volumes” and “holdings” and whatnot wins a prize). Many places also count gate traffic, serial use (often by what needs to be reshelved), program attendance, and so on.
There’s nothing wrong with any of that, but I’ve never been sure exactly how useful those numbers are. Sure, we can count those things, but what do they really tell us? If our motivation for keeping statistics is to know what resources people are using, then surely we must include all of the ways in which users interact with library resources. And now there is a great deal more data about precisely that. The vast—and I think largely untapped—source of data here is the logs of how people are using library Web sites. In addition to knowing what resources are being checked out, we now also know what things are being looked at and even how they’re searched for. These logs can be studied and analyzed to determine what kinds of resources people are using, what databases they are searching, how they are using the catalog and locally maintained Web resources, and even how they are making their way through our Web sites.
In the library, no matter where
In addition to giving us a better picture of usage and interest, such analysis also reinforces an important idea: that people who are engaging with high-quality sources of information provided to them via their libraries are for all purposes in the library, regardless of where they might be physically. This is something worth stressing not only to ourselves, but also to our communities. Catalogs and databases don’t come cheap or free; neither does the time it takes for you to evaluate or create good, freely available Web stuff. That’s a crucial case to make in budget-constrained times.
Many tools are available for this sort of work; a search for “log analysis” in either Yahoo or Google Directory will pull up categories of potentially useful software. There’s been good discussion on how to do it on discussion lists such as Web4Lib; searches of their message archives would also be useful.
In a post–Patriot Act world, of course, poking around in logs could have sinister overtones; I know a lot of libraries are choosing to wipe out certain records after 24 hours to avoid having to turn things over if and when the knock comes at the door. It would be a shame to be on the brink of having this valuable source of data about our communities and how they are trying to find quality information, but then disposing of it for fear of where it might wind up. That’s a valid and important concern, however, and we’ll need to be clever in how and how quickly we extract useful information.
This analysis would be a step in the right direction. The next challenge, of course, is to measure and assess what really matters—how people are being helped by their libraries, what difference we make. This goes way beyond simple counting and will force us to think really hard about what libraries are for, and what they’re trying to achieve, and to figure out how to get our hands around that. But that’s another story.
|
