Skip to main content

TRU Library Assessment Framework

This guide tracks current and upcoming assessment activities at the TRU Library, for the purposes of values alignment and evidence-based practice

Your Librarian

Amy McLay Paterson's picture
Amy McLay Paterson
Contact:
Office: L208, Main Library
250-852-7832

Home

Purpose and Goals

This framework outlines and tracks the assessment activities of the Thompson Rivers University (TRU) Library. It is intended to be flexible and adaptable, in order to accommodate changing strategic priorities of our institution and our library. In order to remain accountable; collections, services, and spaces are periodically assessed both for their relevance to our users, and their alignment to our strategic priority and stated values.

Questions about any information contained here may be directed to Amy McLay Paterson, TRU Assessment Librarian.

Goals of Assessment Planning

  1. Strategic Thinking: We connect our own work with that of our colleagues, our institution, and the library profession
  2. Aligning Service to Values: Our work is shaped and guided by our core values. Through periodic assessment, we can congratulate ourselves on our successes, identify gaps where we are falling short, and make plans for improvement
  3. Evidence-based Decision Making: Change is constant, both in our library, and our institutional environment. We need the right kinds of data to guide us through changing and improving our services
  4. Transparency and Accountability: Sharing our plans, goals, and projects shows our commitment to both our community needs and following through with our promises and plans
  5. Communication and Co-operation: Comparing information makes it easier to work together and keeps us in touch with our colleagues' efforts to improve our Library

Guidelines for Starting an Assessment Project

Assessment projects may be planned in alignment with TRU Library Strategic Priorities, but they may also be done periodically within library functional areas. Here are some questions to consider and steps you can take before getting your project started.

Questions to Consider:

Are you wanting to assess within your own functional area of the library, or would this assessment cover multiple areas?

  • If only your own functional area will be affected, you will have more freedom to proceed. Consider discussing your project with the Department Chair to sort out timing and workload issues, and with the Assessment Librarian for advice on planning and to identify opportunities for coordination
  • If your project will affect multiple areas, bring it up at a Department meeting. Large assessment projects will need a higher level of coordination, and those that support current strategic initiatives will likely be prioritized

What commitment level or buy-in will you need from library staff? How will this affect processes and/or workloads?

  • It may be helpful to set up a working group for your project. Coordinate with other librarians to determine which LCSAs would be useful to your group
  • Discuss with relevant area librarians if you need to make temporary changes to usual processes
  • Use the Library Communication Moodle to provide updates while the project is in progress

What kind of timeline does this project need?

  • If your work includes multiple phases, consider whether these phases need to be simultaneous, subsequent, or if there will be time in between
  • Depending on what you are measuring, some times of year would likely work better than others, so make sure that your project is timed correctly

Will you be involving students or other library users? 

  • If you will be collecting personal information or using human subjects, you may need Ethics approval. Generally, if the purpose of your project is limited to Quality Assurance, then Ethics is not required. For more information, view the TRU Research Ethics Policy.

What are your objectives? And how do you expect this assessment to impact the library as a whole?

  • Is your work linked to any particular strategic priority?
  • Having particular objectives in mind will keep the scope of your project clear and it will help you craft specific tools to answer your questions

What kind of data will you be collecting?

  • Discuss with the Assessment Librarian whether any existing library data would be helpful to answer your questions. You may also discuss whether there are any potential modifications to our collected data that would be helpful
  • If you will be collecting a large amount of data, you may benefit from having a Data Management Plan. The Scholarly Communications Librarian can advise you as to whether such a plan is necessary and how to create one

What tools will you be using? 

  • The Assessment Librarian can provide insight on what tools are available, as well as associated advantages and disadvantages. She can also provide criticism and advice on any homegrown tools (ie. surveys) you would like to create

How will you be communicating your findings?

  • Think about who the stakeholders of your project are and when and how often certain groups may need updates
  • If you are not collecting personal information, you may want to make your findings public through an article, a conference presentation or simply to contribute to the knowledge economy
  • If your work identifies unmet needs or points to major changes, you may wish to think about an action plan before presenting your findings to external groups

Once you have conceptualized your assessment project, fill in a copy of the Assessment Project Template box below and post it in the Future Assessment Projects section. You may find that information is sparse in the beginning, but you will fill in more information as you go along.

Assessment Project Template Box

To add an Assessment Project to this framework, make a copy of this template box and fill in the information specific to your project. You may not have all information at the outset, but you can and should update the project as time goes on.

Status: Is this project current, future, or ongoing?

Background: Write a short paragraph giving background information and context for what you are doing

Responsibility: Who is the project lead?

Assessment Objectives:

  1. What are you trying to learn?
  2. What outcomes are you expecting?
  3. Are there any specific strategic priorities you are trying to support?

Assessment Tools:

  1. What evaluation tools will you be using?
  2. This can be a combination of regularly collected data, surveys, interviews, focus groups, environmental scans, etc.
  3. You may choose to link directly to the tools, though if you are linking to live surveys, please make sure they are not accessible to persons outside the test group

Outcomes:

  1. This category will likely be TBA until the project is complete
  2. Acknowledge the impact of your findings and any concrete changes that were made as a result

Available Reports: Here you can attach any relevant reports from your assessment. LibGuides includes a mechanism for password protection, should you wish to make a document private but the rest of your project public

Data Collected by TRU Library

Regularly Collected Data

The TRU Library regularly collects aggregate data pertaining to usage of its collection, spaces, and services. These numbers are reported to internal and external bodies for posterity, benchmarking, and to make evidence-based decisions concerning library resources. Personal information is not used for assessment purposes. Any collection of personal information by the TRU library is used for the management of library services and is not shared with external bodies. Circulation records for individual users are not retained after an item is returned. The TRU library complies with the University Policy on Protection of Personal Information.

 

The following are types of data regularly collected and periodically used for assessment by the TRU Library:

To access any of the following data, contact the library, or you may access yearly data in aggregate at the CPSLD website. All data listed here is stored on an off-site, Canadian server, compliant with all BC Privacy Legislation.

Gate Counts

Collection: Gate counter automatically records when a library user enters/leaves a Kamloops campus library. Numbers are recorded monthly.

Use: Reporting to external bodies; track usage patterns

Head Counts

Collection: Counts are taken 3-4 times daily by library staff manually counting numbers of people occupying various library spaces. Head counts are taken in the Main Library and the Williams Lake Library only.

Use: Identify in-demand and under-used spaces; identify busy times of day; provide snapshots of library usage at different moments in time

Reference and Information Services Questions

Collection: Library staff records basic details (location, question type) of questions asked at our information desks or emailed to library@tru.ca   Periodically, question details may be retained or recorded for the purposes of staff training or gathering additional input. No personal or identifying information is retained after the question has been answered.

Use: Reporting to external bodies, identify staffing and training needs

Research Consultations

Collection: Subject Librarians record details of in-depth research consultations. Details do not include personal information but generally include user type (student, faculty, community, etc.), academic department, referral method, and amount of time spent.

Use: Subject Librarians can self-evaluate outreach to liaison areas and report time investment in APARs and tenure/promotion documents; data may be reported to external bodies in aggregate

Library Instruction Classes

Collection: Subject Librarians record details of instruction sessions taught, including the names of the requesting faculty member, the length of the session, number of students attending, location of the session and the amount of time spent preparing.

Use: Numbers of sessions and numbers of attendees are reported externally in aggregate. Subject librarians may use data on their own sessions to self-evaluate and improve liaison outreach efforts and to report in APARs and tenure/promotion documentation.

Workshop Attendance

Collection: Attendance numbers at library-created and sponsored workshops is recorded. When names and email addresses are submitted as part of event registration, this information will be used only to communicate with registrants about the event signed up for and will not be released to any other party or used for any other purpose.

Use: Workshop numbers are reported externally in aggregate. Attendance is also used internally by librarians to assess future workshops and may be self-reported in APARs and/or tenure and promotion documents.

Circulation statistics

Collection: Monthly circulation numbers are captured by library, item type, and call number range. Circulation includes all loans and renewals but does not currently account for in-house item use.

Use: Reporting to external consortia and evaluation of collections for the purposes of further acquisitions and retention

Inter-Library Loan usage

Collection: Borrowing and lending numbers are tracked by month. Titles requested from other libraries are retained apart from any personal information about the requestor.

Use: Numbers of items borrowed and lent are externally reported, as well as used by the Open Education Librarian to evaluate future services. Requested titles are kept for purchase consideration by Subject Librarians and the Collection Services Librarian.

Services to DROL users

Collection: Physical items sent to Open Learning students and faculty are recorded, as well as ILL items sent directly to DROL members. Reference and consultation services provided to DROL members are also recorded in addition to the method by which the question or consultation was received (ex. phone, email).

Use: The Open Education Librarian reports DROL service usage to TRU Open Learning. She also evaluates and plans future services.

Website / Research Guides Views

Collection: Website visits and views are tracked using Google Analytics. Storage, access, and retention of this data is controlled by TRU Marketing and Communications in accordance with B.C. legislation and university policy. Research guide visits and views are tracked internally by the library. This information is stored on a secure, Canadian server; referring URLs are collected, but no personal or identifying information is collected or retained.

Use: Website and Research Guide Analytic information is used by the Electronic Resources Librarian to optimize future web services. Research Guide statistics are additionally used by all librarians for assessment of their functional and subject areas.

Electronic Resource Usage

Collection: E-Resource usage is provided by library vendors and/or occasionally consortia. Availability and access to usage statistics is normally one consideration before licensing an electronic resource. Where possible, COUNTER reports are harvested via SUSHI protocol and stored on a secure, Canadian server.

Use: E-resource usage statistics are used by the Collection Services Librarian, the Electronic Resources Librarian, and other Subject Area Librarians to evaluate the value of our electronic resources for collection and retention purposes. Usage statistics are one of many considerations when deciding to add a new resource or retain an existing one. 

Current Assessment Initiatives

LNAP Event Assessment

Status: Ongoing, twice yearly during and after LNAP events

Background: Long Night against Procrastination (LNAP) usually occurs twice a year in November and March as an all-night study event for students. The library partners with the TRU Writing Centre as well as other campus groups to host and staff the event. For more information, see https://libguides.tru.ca/lnap

Responsibility: User Engagement and Student Success Librarian/ Assessment and User Experience Librarian

Assessment Objectives:

  1. Ensure that future events are adequately staffed and resourced, based on attendance, feedback, and student engagement with various event features
  2. Tailor future events to stated student needs and goals, based on survey feedback

Assessment Tools:

  1.  Registration and attendance numbers
  2. Head counts taken throughout the night
  3. Workshop attendance numbers
  4. Entrance/exit survey results: A sampling of students are polled upon entering the event in the evening and at breakfast the next morning. Participation is voluntary, and responses are anonymous. Entrance and exit survey questions change at each event, and are chosen by the User Engagement and Student Success Librarian in consultation with the Assessment Librarian
  5. Participant survey: Standard full-length survey sent to all participants after the event. Responses are anonymous and used only for internal purposes

Outcomes:

‚ÄčNov. 2017: Students were asked in Entrance Survey how they heard about LNAP, with Posters being the overwhelming response. LNAP Marketing team chose to continue to focus on print posters for future LNAP events, rather than brainstorm new and emerging social media campaigns.

March 2018: The team were curious about whether students who lived on campus were more or less likely to stay until the Survivor's breakfast, with results indicating that very few LNAP "survivors" live on campus. Students were also asked in the Entrance survey about how they arrived at campus, with the survey being used as one tool to discover whether transportation is an ongoing issue for LNAP participants.

Nov. 2018: LNAP organizers were curious about student perceptions and attitudes toward the "all-night" aspect of the event. The entrance survey asked students how long they planned to stay, with the majority responding "all night" and whether students would attend the event if it ended at 1am, with the majority answer being "no."

Available Reports:

COPPUL SPAN Monograph Project

Status: Current

Background: In partnership with other COPPUL libraries, the TRU Library is reviewing its monograph collection to retain unique monograph resources and weed unused monographs owned by other nearby COPPUL libraries.

Responsibility: Collection Services Librarian

Assessment Objectives:

  1. Identify uniquely-held monographs and mark these for retention
  2. Weed unused or outdated monographs, especially those held by nearby COPPUL libraries

Assessment Tools:

  1. SPAN Collection Data

Available Reports:

Faculty Feedback on Library Instruction Classes

Status: Current, possibly ongoing at the end of Fall and Winter semesters

Background: Faculty who booked a Library Instruction session in the previous semester were sent a short survey asking about their opinions and their perception of the session's impact

Responsibility: Assessment and User Experience Librarian/ Research and Instruction Librarian

Assessment Objectives:

  1. Provide course instructors a safe, anonymous opportunity to provide feedback suggestions for improvement
  2. Identify specific impacts of library instruction
  3. Identify instruction areas that may require greater emphasis

Assessment Tools:

  1. Faculty feedback survey: https://tru-ca.libinsight.com/facultyfeedback 

Outcomes:

  1. The majority of provided feedback was very positive, with varied types of impact identified
  2. Citation and plagiarism was identified as the most common area where course instructors believe students could improve
  3. Further study in this area was identified as a priority

Available Reports: 

Future Assessment Initiatives

Website Usability Study

Status: Slated for 2018-19

Background: The TRU Library Website transitioned to new templates developed by TRU Marketing and Communications in December 2016, which required many changes to the look and feel. Additionally, Research Guides have gradually taken a more prominent role in the Library Web Presence. There has not yet been a comprehensive user study to discover how users are interacting with the TRU library web content and to make evidence-based decisions about our web presence going forward.

Responsibility: E-Resources and Assessment Librarian

Assessment Objectives: 

  1. Find out which web resources are most used by the TRU community and highlight these resources
  2. Increase the intuitiveness of the website for performing standard tasks, such as finding a book or locating library staff contact information
  3. Ensure the TRU website is accessible and usable for the entire TRU community

Assessment Tools:

  1. Website and Research Guide usage statistics
  2. TBD

Available Reports:

Past Assessment Initiatives

2016-2017

In-Depth Service Desk Study

Status: Completed April 2017

Background: Before consolidating the Reference and Circulation desks, in-depth information about every Public Service interaction at TRU's Main Library was recorded over a sample period of 4 weeks, 2 of which were identified as typically busy weeks, and 2 were identified as lighter. 

Responsibility: E-Resources and Assessment Librarian

Assessment Objectives:

  1. Determine busy times of day to better align staffing needs with the new Service model
  2. Identify training needs for staff

Assessment Tools:

  1. In-depth Public Service interaction survey, which tracked circulation, reference, directional, and other questions together. It also recorded additional details about the interaction, such as sources used to answer the question, approximate time spent, and how the staff member thought the transaction went.

Outcomes:

  1. Findings largely supported proposed hours for the new staffing model
  2. Numbers and complexity of reference questions helped shape discussion over future librarian role in the Service Desk

Available Reports: 

2017-2018

Service Model Assessment

Status: Completed

Background: In 2017, the TRU Library transitioned its Service Model to a Single Service desk, providing both Circulation and Information Services. Additionally, roles for LCSAs and Librarians in providing Information Services were updated.

Responsibility: Access and User Services Librarian

Assessment Objectives:

  1. Ensure that all roles are backed up and staff vacation and sick leave absences are covered
  2. Provide quality service for students by eliminating confusion due to multiple service points and ensuring that staff are cross-trained
  3. Attempt to break down work silos to create an effective triage partnership between librarians and staff

Assessment Tools:

  1. Reference and circulation statistics
  2. Service manual usage statistics
  3. Libstaffer and observational data for vacation and sick time coverage issues or successes
  4. ACD phone reports for DROL
  5. Librarian and LCSA feedback survey

Outcomes:

  1. Libstaffer and observational data revealed that vacation and sick leave coverage was working well
  2. All-staff retreat in April 2018 provided suggestions to expand on staffing model results, improve communication, and in some cases, vote on proposed solutions

Available Reports:

Citation Guide Sustainability

Status: Current

Background: TRU Library maintains citation guides in 6 different formats: APA, MLA, Chicago, CSE, ASA and ACS. However, many of the guides have not been updated because of demands on librarian workload, in addition to easy availability of external resources. The working group was formed to investigate and recommend a sustainable path to updating citation guides that accounts for both student need and staff workload demands.

Responsibility: E-Resources and Assessment Librarian

Assessment Objectives:

  1. Recommend to Area Librarians a sustainable solution for the electronic citation guides that accounts for both student needs and staff workload demands

Assessment Tools:

  1. Citation Guide usage statistics
  2. Environmental scan of other academic institutions
  3. Reference statistics: number of citation questions received at the information desk, in addition to the citation format, were tracked from February 2018
  4. Librarian Interviews: Librarian feedback was sought to ascertain workload considerations, as well as willingness to continue working on existing guides
  5. LCSA Survey: LCSA feedback was sought to get their perceptions on the approximate number of citation questions answered at the Info desk, what sources they use for reference, and which styles are asked about most frequently
  6. User survey: A small sample of 50 in-person users were asked which citation style they used most often, what sources they used to help with citation, and whether or not they had used the TRU guides before

Available Reports:

2018-2019

Service Desk Buzzer Use

Status: Completed Fall 2018

Background: When on-duty staff are required to leave the Main Library Service Desk, they use the Service Desk pager to notify the on-call LCSA to support the desk. Sample weeks have been identified in October and November 2018 to take an in-depth look at Pager Use on the Service Desk

Responsibility: E-Resources and Assessment Librarian

Assessment Objectives:

  1. Identify how often the pager is being used and the length of time the on-call LCSA is spending on the desk.
  2. Explore patterns in pager use, and identify any relevant implications on scheduling

Assessment Tools:

  1. Pager use tracking tool, to be used during the identified sample weeks

Outcomes:

  1. Pager use was not frequent enough to draw any firm conclusions; the recommendation was made to investigate barriers to implementing the pager system at the desk and to find alternate systems if necessary.

Available Reports: