Ratings or Helpful/ Not Helpful? > How are these working for you?

[Jayne Loh]

Happy new year everyone!

This is a topic that we come back to very frequently - if we should use either one of the feature to rate an article with a 5 star rating scale or Helpful/Not Helpful scale.

I was combing through these forums and found that many kept both which we originally did but we didn't get good data points from both and found it confusing to explain difference between top rated vs helpful/not helpful.

Our current setup: Helpful/Not Helpful - not helpful triggers KFT creation. Note we had comments turned off as well since we had issues moderating comments shown publicly on a HR Knowledge Base. 

Most common setup seen here: Having both, Not Helpful + Rating 3 stars or less triggers KFT.

Would love to hear from your experiences and best practice.

1. If you use both, why and how to you measure these metrics for reporting
2. If you use either one, why and is it working well for you

Thanks in advance and hope to learn from your experiences!

[JamieSuz]

We would like to make comments required for the Not Helpful selection. I find that people will select that feedback option for an article but then do not provide any further feedback.  We have tried to reach out via the feedback task to ask these questions but still do not get much in the way of feedback. I think requiring comments will at least give us some sort of feedback that we can use.

[Leri Andrews]

We are now totally out-of-the-box so they're both on and comments are off (so people can't post confidential/rude stuff for everyone to see) and generate a task if 2 or below/No is chosen. A notification is generated to all ownership group members and the task is assigned to the ownership group manager. I wish it was auto-assigned to the author/reviser instead.

I tend to think of them as measuring different things - an article might be of horrible quality (1 star) but was useful.

A few things I've noticed over the years:

1. Turning comments off doesn't work properly if your knowledge is in the HR scope

2. They both sit at the bottom of the article so if its long and unhelpful people never get to the bottom to tell us that. 

3. You only get to see the star rating when you're in the article - it would be useful if it was in the search results/topic page filters as it was before we replatformed.  Will have to investigate if that's config we can do

4. We also use 'flag article' because otherwise HR colleagues have to rate something poorly to get the chance to comment on improvements they want to an article they actually think is good.

5. The ownership group manager finds it really tricky to assign the task appropriately because they can only see the KB number and not the author of the article

6. The user can cancel the popup modal but the rating still happens

7. The user can leave the comment box empty and submit 

8. The users often comments that they can't see the attachment (because it's in a widget they haven't noticed)

9. You have to look at the related list at the bottom of an article with a low useful rating to understand if it's 20% of 5 people or 20% of 500 people who find the article useless. I suspect the related list also only shows feedback associated with that version so even more detective work/modification is required

6. Useful rating is shown if you have deflection set up on a raise a case form and also if your agent attaches the article to a case. Not sure how you work out where the feedback comes from

7. People don't realise you can click on the stars 

8. Mobile view is different

On balance I think they're both useful and not difficult to report on, but I wish they plugged into a much slicker 'case deflection' module.

[Rene Dugas]

Hey guys,

Over time we've tried multiple combinations of feedback options and I can't say we've found the perfect setup yet.

It also depend with the size of the organisation and who you are publishing articles for.

We have 3 main audiences: Regular employees (50 000), IT employees (4000) and Servicedesk employee (400).

We've noticed that if you can't closely communicate your expectations related to the feedback that these audiences can give you, it won't be that relevant.

We get relevant feedback from our Servicedesk employees because we explain to them what feedback we expect from them and what it should mean. It's included in their training. 

But as soon as we get to the 2 others groups, feedback is not that relevant.

We have called back employees who gave 2 stars on KBs and others that marked not helpful.

And when we get to talk to them, more that often, they've given negative feedback but not for the reasons we expected. Often, they mark not useful simply because that KB was not the one they were looking for.

Also, using the star rating average score as not proven usefull for us as we do not get enough rating per article.

Our position now is that feedback without comment, is costly feedback if you want to act on it.

And feedback that is not clearly explained to a large audience can't be that relevant because the audience does not have a clear idea of what the feedback options means and what we expect from them.

Hope this helps !

Rene

[Jayne Loh]

Great angle to consider - i.e. audience of what the feedback is about. This has given me food for thought in thinking about the targeted audience + change management to gather feedback that is valuable. 100% agree that feedback without comment is an almost impossible one to put to action. 

[jeanderson]

@Jayne I appreciate your initial post. We are just now switching to ServiceNow and these are discussions we are having currently for which feedback mechanisms to include.  What are you referring to when you say "triggers KFT creation"?  Thanks!