Zing generates search results in four phases

  • Release version: Xanadu
  • Updated December 12, 2024
  • 1 minute to read

    • Generate search results by dividing documents among shards, searching shard documents, scoring shard documents, and merging scores into an index.

    Zing generates search results in four phases:
    1. Divide searchable documents among index shards.
    2. Search and filter shard documents.
    3. Score shard documents.
    4. Merge shard document scores into an index.

    Phase 1: Divide searchable documents among index shards

    When a search is executed, the system equally divides all searchable documents among 10 index shards. Each shard has a unique list of documents.

    To maximize search efficiency, the system creates 10 query threads to simultaneously search each index shard.

    Figure 1. Divide documents among index shards
    The system divides all searchable documents among 10 index shards from Shard 0 to Shard 9.

    Phase 2: Search and filter shard documents

    Each query thread searches through its list of shard documents to identify documents that match the search criteria. For example, if you search for "wifi network," the system returns all documents containing both "wifi" and "network" in no particular order.
    Note:
    A multiple word search is equivalent to multiple single word searches connected by an AND operator. For example, [contains][wifi][AND][contains][network].

    The query thread assigns each matching document a numeric value (a document ID) to uniquely identify it. All other documents are ignored.

    Phase 3: Score shard documents

    The query thread scores each matching shard document.

    Figure 2. Query threads search, filter, and score shard documents
    The system creates 10 Query threads to search, filter, and score shard documents.

    Phase 4: Merge and sort shard document scores

    The system merges the document scores into a single index and sorts the documents from highest to lowest document score. The documents with the highest document score are most relevant to the search query.

    Figure 3. Merge and sort shard document scores
    Merge and sort shard document scores.