I’m in the middle of a usability study of Fresno State’s Henry Madden Library website. We’re gearing up to run 42 Fresno State undergrads through ten tasks on the site, from locating a book on a particular subject to finding information about special collections. The goal is to assess the extent to which the site is useful and effective at delivering access to information. Can users efficiently use the site to accomplish their tasks? We know the site does some things well, and that it has problems, but we’re excited to actually explore the depth of the site’s pros and cons with a statistically significant sample.
What’s fascinating and fun about this is the multidisciplinary nature of the project and the constant struggle to clearly define what we are doing. The team consists of three librarians, Allison Cowgill, Amanda Dinscore and Patrick Newell, plus myself and one of my advanced anthro students, Kim Arnold. The librarians know the site best, and they know what students need to accomplish on it. All have prior experience with library- and information-related research. I’ve found my own best contributions relate to how to handle live human subjects with both respect and a firm guiding hand. Plus, I think it’s helpful to the project to have non-librarians around who can probe the rationale behind the various questions. I have found myself asking, repeatedly, “What are you trying to get at when you ask a user to do this?” – forcing an examination of assumptions and goals. I hope I always have someone asking me those kinds of questions when I’m formulating projects.
Right now, we’re in the testing phase and the instructions to the users and wording of some of the questions are still up in the air. We’ve tested the protocol on each other, and on the moderators – five anthropology majors who will administer to the test to the actual subjects. Each time we’ve administered the protocol, we’ve turned up a new issue. One open issue we have right now is how to handle the well-known student practice of going off site to find resources that are on site. For example, we know that some students use Google Scholar to search for resources – a practice that sometimes brings them back to the library website itself. Another example: in a test session, one of the anthro student moderators playing test subject opened a browser page and used Google to find the requested information, and then tracked back to the library website – “found it.” So, in the test, do we prohibit users from going off site?
Right now, I’m leaning toward prohibition. We know students use Google and other tools to make the library’s website work for them. But this study is a test of the library’s website. If we’re going to gauge how well it works for students, we need to keep them on the site – even if it means short-circuiting their tendency to exit the site and come back from another direction. If a user struggles to find something, or fails to find it, then we’ll know that’s an area where we need re-examine the site’s architecture. If we let them go off site to solve the problem, we won’t be able to record the process through which they try to solve the problem using the library site.
This is an odd place for me to come down, since most of the work I do is highly strategic. I place a high value on trying to see the world from another’s point of view. In a different context, I’d really want to let the user go to Google and show me how they would really solve the problem. That’s real user practice and it’s significant. But in this context, I keep coming back to the goals of the study: this is a test of the library website, not a study of student search practices.
I can’t wait will our next workgroup meeting where we can hash out this and other issues. I suspect some of my colleagues will have different approaches to the issues. Like I said, these collaborative, multidisciplinary projects are FUN.
Comments and opinions are most welcome.
TheAnthroGeek
anthroguy
Felicia Salcido
13 comments
Comments feed for this article
February 26, 2010 at 5:36 pm
Eileen Walsh
This project does sound like fun; it has a fairly clear goal of discovery, with a variety of perspectives to incorporate. I too often want to ask people “what are you trying to get at by asking that?” but in most settings there is too much potential to come across as quarrelsome! You folks HAVE TO find out what people mean. Our assumptions are so unstated. I look forward to hearing what you all find. Sounds like a great team!
May 14, 2010 at 2:45 pm
Ashley B
I think this study is a great evaluation of how comfortable and savvy students are to the resources provided for them. I think it is an indicator that as soon as students turn to google to answer thier questions, that maybe the libraries resources are not as straight forward as googles. However, in the name of your research it might not be a bad idea to short out access to google. At that point, having no other option, students would turn to navigating the site provided by the library. And I have no doubts that if their was something they really needed to find, they would find it.
May 17, 2010 at 9:55 pm
PedroR
I think the usability study of the Fresno State Library is a great thing to do especially since the library is the heart of an academic institution. We all know that there has been a technological revolution in the 21st century, and new technology will keep coming. It is up to universities and libraries to take advantage of existing technology and not reinvent the wheel. Well I personally like our libraries website; I do see the difficulty in navigating it and searching for what I need. I also look at outside sources such as Google, yahoo, and others first before I even log into the library website. The reason why I think many students move in and out of the site is because many of the other sites offer more information, resources, or just plain easier to access/navigate through. My recommendations would be to centralize everything into one search that can index all the databases, catalogs, articles, etc, or like many of my own professors do is actually upload all their required reading onto their course reserves.
May 17, 2010 at 10:50 pm
OwedarL
This sounds like a very interesting and maybe even entertaining project. Many a time I have used a site and have thought to myself “wow, this could’ve been done better .” I don’t think I’ve ever thought of actual ways that the site could’ve been better. Usability should be the most important aspect of a site, and I’m glad that you’re working on a project like this. I wish that I could’ve been a subject. Not letting subjects venture out onto Google is a good idea for the study, I think. This way, subjects would only have to go through the studied site. But then again, maybe the fact that google is used attests to the level of usability within the library’s site.
May 19, 2010 at 1:08 pm
Nicholas Miconi
This test sounds very interesting. I agree that the test should be about the user’s ability to navigate through the website and not find ways around it. A suggestion, maybe restrict them from going off site for the test then after have them show you how they would find the problem. I believe many students do not know how useful the library resources are. The problem is trying to figure out how to use the thing and I agree with Pedro about the ease of access and the centralize everything into one search.
May 20, 2010 at 11:26 am
Maria Capetillo
Wow sounds like a very interesting project and at the same time fun because it is prohibiting the use of different sites. I think that makes it interesting because it is only centered on the library’s website. This study will be very useful for us students, as a student sometimes I feel like the library website could be more useful. Like many other students I sometimes have to research information outside of the website. I have to say the website could be overwhelming at times and some of the search engines provided are very specific to the topics you search. It will be interesting to see the results of your usability study.
July 18, 2010 at 10:11 pm
Keri Ponce
This is a very interesting study! The library is an essential part of a student’s success whether it be to find information about services offered like tutoring or the hours of operation, and of course research. I think evaluating the library’s website is very important since it’s the main way of accessing articles, books and information by today’s students. Studying the websites ease of access can benefit the students tremendously by making it their first choice for articles and such instead of Google, or Google scholar. Many professors have the fear of their students obtaining and using information from unreliable websites and I think the results from this study could help reduce those fears.
July 20, 2010 at 8:38 pm
Michelle Alcantara
This project sure seemed like it was a lot of fun! I wish I could’ve been one of the test subjects honestly. It was good that there was a clear goal in mind, but with that goal comes many unanswered questions and complications. So it was good that those were anticipated as well. By being prepared for all sorts of possibilities, the library website can be improved as needed.
In regards to restricting test subjects to go off the library website, I understand where you are coming from. Just as you said, the study was conducted to see just how well the library website works for students, not some other website that can help them through it. However, I feel that restricting them from google, or any search engine, can make them feel stressed by the limited options available. Instead of having multiple choices, only one website is available for them to use. This in turn can make them think that the 10 tasks they need to do are much more complicated than they really are. This can either skew the results because the tasks could have been very simple, or it could go in the studies favor and help improve the website, making it less complicated to search for items.
Also in regards to not using google: everyone uses it. It is probably the main search engine for the internet itself and I am pretty sure that ALL college students use it. It might even be a habit for someone to go straight to it when searching for anything on the internet. Knowing this, letting them use google can be of benefit by, as you said, seeing just how students really go about solving their problems. When it comes to searching for something online, what are their goals? What kind of key words do they type in? When searching for a book, do they type in the title? Author? Description of the book? This could have really helped in improving the options available on the library website when searching for key words for certain books and other research materials. By letting students go about their natural way of searching for things online, it could have helped in seeing what is essential in the library website. Yes, the point was to study the library website itself. Ok, I got that. So restrict students to use google during this part of the study and have a second study with the same test subjects where they naturally look for things online their own way. If this happened, I would have suggested that they do their own way first before solely using the library website.
All in all, I am guessing this project was long done since the library website is up and running today. I have used the website myself when searching for certain materials, and I must congratulate you: the website is wonderful. I never had any trouble searching for anything on the website. It was clear, concise, and provided many options when key words were typed in. Finding research materials for research papers have never been easier! Thank you for your hard work on making this website work so efficiently for us!
July 22, 2010 at 11:16 am
Whitney P
February 16, 2010 The website of the Henry Madden Library is intense because there are so many layers of information and tools. Although it is made to help students find information and save time, it fails to do that in certain ways. It is good that there is a study going on to improve it because students should not be avoiding the library in person or on the internet. I definitely agree with prohibition in testing of outside search engines. Using those outside sources makes the trial a waste of time.
It is a fact that students avoid the library more than they probably should because they access the usual internet search engines and prefer to get most of their resources for essays online. One problem not addressed concerning the library is the hours it is open. Not having physical access when needed (for example at midnight on a week night) forces the student to rely on the internet and become so used to using it that the thought of actually going to the library is not attractive. I know this blog is about the website usability, but one must also address how user friendly the actual library is too. If the physical library is user friendly, then the website should be too. The two go hand in hand. So, in short, it would be great if students could combine using the library and the internet together through the Henry Madden Library website.
July 25, 2010 at 11:13 pm
Kenneth Morshead
I believe that this project is great in the sense that the end result will actually be implemented into practice. I recently was required to access online journals at the Fresno State Library’s website for the first time and it was not the most user-friendly to say the least. The first search I conducted resulted in problems because the program allowed me to conduct a search without first logging in and then after being prompted to log in it did not return me to my search. However, I don’t intend to bore anyone with the difficulties that I experienced but it is very nice to experience a problem and shortly there after find out that it is being worked on by others looking to analytically improve things by studying human behavior.
I also tend to agree with the idea of keeping the study “pure”. It has been my experience on studies that are based on efficiency or “ease of use” that the more variables that are included increases the chance for errors in the accuracy of the resulting data. Also, as was mentioned, it becomes simpler to decide to keep this study more narrow due to the fact that this is a study on the libraries websites and not how to search the web.
July 25, 2010 at 11:46 pm
HilaryH
As a student who has used the Henry Madden Library website I can attest to using Google to supplement my Library search; recently I had to search the libraries’ scientific journal online database for a research project and I had to spend considerable time searching for exactly what I wanted. Later after having found the article I wanted by reading the abstract, yet for some reason the access to the whole article was denied and I was forced to use Google to locate the journal at some other location.
All together I rarely use the Library website and when I do I struggle and fumble around. I find the journal databases to be very difficult when typing in topics and keywords; they tend to pull up articles which contain the key words yet the article as a whole can many times be seemingly unrelated to the desired subject. Another problem I’ve had a library website user is the ability to reserve group study rooms; last semester I needed to reserve a study room online yet when I attempted to I wasn’t able to maneuver with ease around the website. I was so discouraged by the whole ordeal of not being able to figure it out that I had to give up trying to go about it the correct way and instead I was forced to find an empty room and pray that it was reserved.
July 26, 2010 at 12:04 pm
AnntrellG
I can see where you struggle with allowing the test group to use Google or not. I vote that you let students use the site as they normally would, including Google, the reason being that it is an indicator of how well the Henry Madden web site works. If students are leaving the site it is a good indicator that the site needs to be revised for those situations that students are most frequently leaving for. Removing Google as an option created a false situation for users. In my own experience the Henry Madden web site is very busy and can be confusing, which may be the reason students are giving up and going to the more user friendly Google option. The Fresno State University web site uses Google as its internal search engine, why not do the same for the Henry Madden Library web site? Another idea that comes to mind is that the site may need to include a user tutorial. If the issue is that students are using Google because of perceived ease than a tutorial may be a more efficient way to illustrate to users that they do not have to leave the Henry Madden Library web site to find the information that they need.
July 28, 2010 at 4:32 pm
Ashley H
I think this is a great study to conduct. Personally, I find it difficult to sometimes get what I need from the library website, especially when they change how to use things. Most people are comfortable doing things the way they learned how and once something changes it sometimes makes a task almost impossible to complete. It is easier for someone who works in the library to get through the database because they are familiar with how to work the system. For students on the other hand, we seem to encounter more problems with the library database than we would by simply searching a subject through google. I hope this study was successful and allows for an easier, more student friendly library database. Although students at the collegiate level should know how to use library search databases for research and writing papers, but the more user (student) friendly the system is, the easier it would be on the life of a college student, as if we do not have enough to worry about already.