How Libraries and Librarians Help:
Context-Centered Methods for Evaluating Public Library Efforts at Bridging the Digital Divide and Building Community
According to findings from our 1998-00 IMLS-funded research, Helpseeking in an Electronic World, 73% of librarians said they were unsatisfied and frustrated with current evaluation tools. Librarians reported that, aside from anecdotal data, they have few indicators of how people benefit from library programs”a gap that impedes their strategic planning. This observation formed the basis for our second IMLS-funded study series, How Libraries and Librarians Help: Context-Centered Methods for Evaluating Public Library Efforts at Bridging the Digital Divide and Building Community. In addition to continuing our focus on how people solve problems of everyday life, our ultimate aim was to derive context-based evaluation tools that public librarians could use to capture their contributions to creating and sustaining vibrant communities through community information services
To that end, IBEC partnered with different libraries to develop and test qualitative instruments that librarians could use to evaluate the impact(s) of a wide range of community programs, including:
- community technology programming for children and teenagers, based on data collected on the Wired For Youth and CIAO programs of the Austin, Texas and Flint, Michigan public libraries, respectively
- programming on social connectedness, based on data from the King County Library System of Washington State.
- literacy, life skill, and cultural programs for immigrants, derived from research on the New Americans and Adult Learner Programs of the Queens Borough Public Library System of New York
- community information services, researched in partnership with the Community Information Program of the Peninsula Public Library System of California
Although the specific instruments and theoretical lens for each study differed, our general approach of understanding the context of service delivery and use from the perspectives of service users, organizers and other stakeholders was consistent throughout. In a climate of increasing demands for accountability and government scrutiny, we aimed at distilling a series of steps that could be implemented and yield useable results by community librarians under a broad array of conditions. Our findings show that public libraries and librarians do indeed contribute a rich array of outcomes to their constituents and that, for themselves and their communities, librarians can discover and document their outcomes using a contextual evaluation approach as outlined in our Outcome Toolkit.
A key output of the How Libraries and Librarians Help study, the Outcome Toolkit, a four-step guide to determining outcomes, broke new ground as the first context-framed approach to measuring library impact. The Toolkit focuses on developing indicators that reflect the range of social contexts and users rich patterns of use of public library services and how these services build community. In recognition that one-size-fits-all methods fail to capture the impact(s) that libraries make in their communities, the Toolkit presents an approach that can be used by librarians to identify outcomes that are unique to specific types of services.
Representatives of a range of institutions, from libraries in major urban centers and ones in rural and remote areas to state library agencies and universities, were trained on the Toolkit and tested its approach. From October 25 -27, 2002, the Information School at the University of Washington hosted the How Libraries and Librarians Help Outcome Evaluation Research Workshop in Seattle. Workshop participants from eleven organizations from around the country immersed themselves in outcomes training, returned to their communities, designed and implemented outcome evaluation studies, and, through ongoing involvement in an IBEC listserv, helped to refine the Toolkit even further.
Since its launch in November 2002, leaders in the field have recognized the Outcome Toolkit as an important component of outcomes assessment. According to Neel Parikh, Director of the Pierce County Library system in Washington State, the effects of the toolkit approach have permeated their entire library system. In July 2003, Dr. Joan Durrance presented the Toolkit at the IMLS-sponsored Outcomes Evaluation and Consulting Institute at the University of Illinois at Urbana-Champaign. Additionally, our contextual approach to outcomes gained national attention through varied publications about the project, and is the focus of our upcoming book, How Libraries and Librarians Help: Why Librarians Need to Focus on Outcomes and How to Go About It, to be published by ALA Editions in late 2003.