Knowledge Management System Evaluation

Why didn't the staff use the knowledge management system?

Life Science Institute (LSI) is a research institute in the University of Michigan. They had a knowledge management system (KMS) running in their intranet. However, the usage rate remained low. To figure out the reasons behind it, they reached out to us.

In this project, I worked with other 3 graduate UX researchers to find out users' attitudes toward the KMS and tried to give recommendations based on our findings.

Met with the client and scope the project

Before started working on the research, we met with the manager of the Intranet and asked her about the systems and users. In the meeting, we found out the main issues the institute facing are that:

Issues needed to be addressed

Unclear Information Scope

Users were not familiar with the information available on the KMS

Unnavigable Structure

Users had problems looking for information they need

Low Engagement

The visit traffic remained low. Almost no people were using it

Conducted interviews

We conducted 6 interviews with users in the institute. The users are categorized by their roles in the institute as well as their relationship with the system.

Roles of the interviewees

Scientists

The target users of the KMS, they search for and upload information to the system

IT engineers

They are the tech guys developing and maintaining the system

Administrative team

They are in charge of the system, managing the structure and contents; Also upload some information

During the interview, interviewees were asked to demonstrate or talk about the following topics/moves:

• Their working environment and working process.
• When do they use the KMS, how they use it, and their experience using it.
• How they retrieve information other than using the KMS.

Two researchers were attended per interview. One was the main interviewer and focus on having conversation with the interviewees, the other one took notes and concentrate on the environment and interviewees' body language. In this way, we could have a comprehensive and detailed understanding of users' attitude toward the KMS.

Created an affinity diagram to synthesize findings

For better efficiency, We worked together for an afternoon and created a physical affinity wall. With the physical diagram, it's easier to navigate through the notes, and the nature of human vision/moves created body memory engaged in all the findings.

In this session, we classified the notes and quotes into 5 categories, including:

From the affinity diagram, we found out that users with different role had very divergent thoughts about the KMS. Following are the findings about at what part did users have consensus on and vice versa.

What's good?

Quick links and administrative content are useful.

Scientists

Administrative team

What's not so good?

Information is outdated, and the content is too administrative-focused.

The bad structure and visual design are why people don't use it.

Scientists

Administrative team

What did they think KMS is for?

To store information.

A platform to interact on.

Scientists

Administrative team

IT engineers

What did they use as alternatives?

I can simply send an e-mail or make a phone call to ask questions.

Scientists

Final Finding

From the interviews, we found out that the target audiences - the researchers already had their own ways to get help or find information. Since they can do it the easy way, there was almost no reason for them to use the KMS. Also, while the content on the Intranet is administrative-focused, they do not really need to access it in any circumstance.

Ceased the Operation

Users did not need it

During the interview, we found out thatthe usage s mainly by the users assigned to upload information. However, users didn't look for information on it; Instead, they only use the quick links to find other pages.

Maintaining required budget and human resource

During the interview, we found out that the IT department did not have enough staff to maintain the system. Also, researchers did not have enough time to update everything on the Intranet.
Also, while the system is hosted by an outside provider, and it costs. Based on the inactivity and lack of engagement with the Intranet, there was not sufficient reason to continue the operation of the site.

If they wanted to keep it

• More discussion between the administrative team, IT, and researchers on Expectation to the intranet, Divide the maintaining workload.
• Survey the researchers for features or information they would like to have on the Intranet.
• The interface and the structure should be simplified for the non-administrators to understand.