How is relevancy calculated in article search?

[kalyansonu5]

Hi All,

I wanted to understand, how is relevancy calculated in articles search (related search) from a HR case form. Sometimes the relevancy score shows very high, for ex: 1000+ points and sometimes it shows in 100's. I want to know how it is calculated. I understand that the match of Article short description, meta, and body plays a role here. But, want to understand the entire concept behind this.

Thanks in advance.

[David Kay]

I worked in search technology for many years (although it's been a moment.)  No relevancy score I ever worked with would have been meaningful (or explainable) to users in absolute terms.  They were only meaningful in relative terms, for the specific search. We would get RFPs or SoWs that asked us to display the relevancy search.  We could do it, of course, but we strongly discouraged customers from making it visible. 
Unless someone with detailed knowledge of the search algorithm tells us otherwise, I'd recommend the same to you.

[Dr Evil]

I would agree, as neither I nor any company I have worked for have been able to pull any meaningful information from the relevancy score. The two companies I have worked for, and friends I have met in this industry, do not use it. If I was taking a guess it would be looking and weighting the following: keyword matching, views, rating, helpful, how often it is updated, click rank, metadata, and tags. 

Here is some additional information pulled from SN documentation. 

Procedure

1. Access Search Event Table: Navigate to the relevant URL.
2. Identify Unsuccessful Searches: Look for queries with Has results as false.
   • Consider creating knowledge articles or synonyms for better results.
3. Calculate Click Rank Average: Lower values indicate more relevant results.
4. Determine Click-Through Rate: Higher rates suggest better relevance.
5. Average Refinements: Fewer refinements indicate easier access to relevant results.
6. Access Search Source Event Table: Repeat steps for source-specific queries.

Goal: Improve search relevance by analyzing user interactions and adjusting content accordingly.

[Qatarpainting]

Relevancy in article search is calculated based on how well an article matches the user's search query. The calculation typically involves several factors and algorithms that determine the relevance of the content to the search terms. Here’s an overview of how relevancy is usually calculated:

1. Keyword Matching

• Exact Match: Articles containing the exact keywords entered by the user are considered more relevant. The more times the keyword appears in the article, especially in the title, headings, or first few sentences, the higher the relevancy score.
• Partial Match: Articles that include variations of the keyword (e.g., synonyms, related terms) are also considered relevant, though they may score lower than exact matches.

2. Keyword Placement

• Title and Headings: Keywords found in the article’s title or headings are weighted more heavily, as these are typically strong indicators of the content’s main focus.
• First Paragraph: Keywords appearing in the first paragraph or introduction of the article also contribute significantly to relevancy, as this section is usually a summary of the entire article.

3. Content Quality and Structure

• Article Length: Longer, more comprehensive articles that cover a topic in depth may be considered more relevant, as they are likely to provide thorough information on the subject.
• Content Structure: Well-organized content with clear headings, subheadings, and bullet points can increase relevancy because it’s easier for search algorithms to understand and categorize the information.

4. User Behavior

• Click-Through Rate (CTR): Articles that are frequently clicked on when shown in search results are deemed more relevant. A higher CTR indicates that users find the article useful based on the search query.
• Dwell Time: The amount of time users spend on an article after clicking on it is another indicator of relevancy. Longer dwell times suggest that the content is engaging and meets the user's needs.
• Bounce Rate: A lower bounce rate, where users do not quickly return to the search results after clicking on an article, also contributes to a higher relevancy score.

5. Freshness of Content

• Publication Date: More recent articles may be considered more relevant, especially for topics that require up-to-date information, such as news or technology.
• Updated Content: Articles that are regularly updated with new information or corrections may also rank higher in relevancy.

6. Semantic Search and Natural Language Processing (NLP)

• Understanding Context: Modern search engines use NLP to understand the context of the search query and match it with articles that address the intent behind the query, not just the exact keywords.
• Synonyms and Related Concepts: Relevancy is enhanced by recognizing synonyms, related terms, and phrases that mean the same thing as the search query, providing a more comprehensive set of results.

7. External Factors

• User Personalization: Search engines may factor in a user's previous search history or preferences to tailor the relevancy of search results specifically to that user.
• Popularity and External Links: Articles that are frequently linked to by other websites or have high social media engagement may be considered more relevant.

8. Metadata and Tags

• Meta Descriptions: The meta description and keywords associated with an article are analyzed for relevancy. Articles with well-optimized metadata that aligns with the search query can score higher.

[Gianluca Roncat]

This is a screenshot from a ServiceNow webinar (match first, characteristic follows), hope this helps.