- Laura Probst, Head of Public Services
- Linda Klimczyk, Information Center Consultant
- Eric Novotny, Humanities Librarian
- Paula Contreras, Reference Librarian
The University Libraries Assessment Task Force has been in existence for only two years, and yet its influence is already being felt in the Libraries. With its goal of making assessment part andparcel of the way the Libraries operate, the task force is striving to create a sea change in the way librarians interact with their customers, or “users” as they are called in the language of librarians.
The co-chairs of the assessment task force– Laura Probst, head of Libraries public services, and Eric Novotny, humanities librarian–along with Linda Klimczyk, information center consultant, and Paula Contreras, reference librarian, spoke to approximately 30 participants at the Quality Advocates Network meeting on November 14, 2003.
The four librarians discussed the challenges of assessing the needs of patrons who are often more comfortable accessing the library through the Web from the comfort of their own dorm rooms than making the trip across campus to talk to a librarian or peruse the stacks. Users who access materials online, however, miss the opportunity to benefit from the personal touch of a librarian who can assist them and offer advise. To address this gap in service, UniversityLibraries is making it their goal to provide an online experience that duplicates as closely as possible the service a patron would receive in person.
Before databases could be developed that would facilitate this kind of service, patrons’ needs had to be more clearly understood. This is whereKlimczyk’s talents came in to play. She wrote a software program that tabulates and assesses the type of queries patrons make as they access the online catalog. Users can now contact staff in the Libraries with their questions through chat rooms and by email. Klimczyk’s software stores the questions that are posed, keeps track of the campus location of patrons, and refers them to the appropriate subject library, knowledge base, etc.
In the future Klimczyk hopes to build a database that will store the answers to patrons’ questions so librarians won’t have to begin anew each time they research a query. Ultimately, Klimczyk would like to have the capability to refer questions to other Penn State knowledge bases, as well as to those external to the university, and export and import Q&A and FAQ data to and from library systems nationwide.
Penn State Libraries, under the guidance of the assessment task force, has participated in several national assessment programs, allowing them to benchmark their performance with other libraries across the country.
“In comparing the results of a national Web-based survey of hundreds of thousands of users, our service quality was rated similar to the national data,” said Contreras.
When asked what service they valued most, national surveys found that library patrons gave the following items the highest ratings (on a 1-9 Likert scale):
Access to resources from home or office 8.35
A Website that supports independent use 8.35
Convenient business hours 8.28
Access to tools that support independent use 8.24
These data indicate that users clearly want their library to provide the tools and database support that will allow them to maintain their independence while searching for information, looking for references, conducting scholarly research, etc. National data show that faculty and graduate students seem to expect a hybrid environment of printing and electronic resources, while undergraduates seem more willing to live in a wholly online world.
In a Penn State Libraries Web usability study, a group of 30 patrons were asked to “test drive” some of the Libraries’ Web functions. Changes in procedures have been made based on patrons’ reactions and feedback. For example, an “ASK” button now appears on all the Web-based catalog (CAT) pages. A patron can submit his or her questions through:
Access to live real time chat rooms with librarians
An email address to send a librarian a question
A list of library reference desk locations and telephone numbers at all library locations
Links to technical help
A FAQs searchable knowledge base
Research guides by course
A link for making comments or suggestions
It is projected that by the end of the current fall semester, nearly 1000 students will have submitted questions through ASK functions. In the future, an ASK button will be added to the Penn State Portal, Angel, and all Libraries’ databases. This could multiply the number of questions submitted in one semester by tenfold.
Novotny described a “usability study” that he conducted to ascertain patrons’ satisfaction with the reference encounter. Two groups of nine students were asked to complete a set of five basic tasks that users should be able to accomplish using the online catalog. One group was made up of novices who had never used the catalog, and the other group was composed of students who had used it 10 - 30 times. Novotny assessed the approaches each group used. He asked participants to talk about the process as they worked their way through the Web links, the paths they took to find information, the difficulties they encountered, the rationale for their approaches, etc. CAMTASIA software was used to track the students’ Web links and record their comments.
What are the next steps for the Libraries’ assessment task force? Probst commented that in the future they will be looking at outcomes-based assessment. “In other words,” she said, “after students participate in a library skills/tools class, how much do they retain and for how long?”
The Quality Advocates Network meets several times each semester to share ideas and examples of improvement and change. To join the Quality Advocates Network mailing list or to learn more about the meetings scheduled, contact the staff at firstname.lastname@example.org.
The Quality Advocates Network is open to all Penn State faculty, staff, administrators, and students.