Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
69 views
in Technique[技术] by (71.8m points)

"match" query along with "should" clause giving more than required match results in Elasticsearch

I have written the following lucene query in elasticsearch for getting documents with Id field as mentioned:

GET requirements_v3/_search
  {
   "from": 0, 
   "size": 10, 
   "query": {
   "bool": {
  "filter": {
    "bool": {
      "should": [
    {"match": {
      "Id": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b"
    }},
    {
      "match": {
      "Id": "048b7907-2b5a-438a-ace9-f1e1fd67ca69"
      }
    },
    {
      "match": {
      "Id": "3b385896-1207-4f6d-8ae9-f3ced84cf1fa"
      }
    },
    {
      "match": {
      "Id": "0aa1db52-c0fb-4bf6-9223-00edccc32703"
      }
    },
    {
      "match": {
      "Id": "8c399993-f273-4ee0-a1ab-3a85c6848113"
      }
    },
    {
      "match": {
      "Id": "4461eb37-487e-4899-a7be-914640fab0e0"
      }
    },
    {
      "match": {
      "Id": "07052261-b904-4bfc-a6fd-3acd28114c6a"
      }
    },
    {
      "match": {
      "Id": "95816ff0-9eae-4196-99fc-86c6f43395fd"
      }
    },
    {
      "match": {
      "Id": "ea8a59a6-2b2f-467a-9beb-e281b1581a0a"
      }
    },
    {
      "match": {
      "Id": "33f87d98-024f-4893-aa1c-8d438a98cd1f"
      }
    }
  ]
 }
 }
 }     
}

The response for the above query is:

 {
  "took": 14,
  "timed_out": false,
  "_shards": {
  "total": 5,
  "successful": 5,
  "skipped": 0,
"failed": 0
},
"hits": {
"total": 18,
"max_score": 0,
"hits": [
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "9d8060da-c3e2-4f6d-b4e2-17e65b266c76",
    "_score": 0,
    "_source": {
      "Id": "9d8060da-c3e2-4f6d-b4e2-17e65b266c76",
      "Name": "Create Extended/Limited Warranty Configuration"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "4461eb37-487e-4899-a7be-914640fab0e0",
    "_score": 0,
    "_source": {
      "Id": "4461eb37-487e-4899-a7be-914640fab0e0",
      "Name": "Create Extended/Limited Warranty Configuration"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "33f87d98-024f-4893-aa1c-8d438a98cd1f",
    "_score": 0,
    "_source": {
      "Id": "33f87d98-024f-4893-aa1c-8d438a98cd1f",
      "Name": "Create Configurator"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "d75d9a7c-e145-487e-922f-102c16d0026f",
    "_score": 0,
    "_source": {
      "Id": "d75d9a7c-e145-487e-922f-102c16d0026f",
      "Name": "Create Configurator"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "007eadb7-adda-487e-b7fe-6f6b5648de2e",
    "_score": 0,
    "_source": {
      "Id": "007eadb7-adda-487e-b7fe-6f6b5648de2e",
      "Name": "Detail Page - Build"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "95816ff0-9eae-4196-99fc-86c6f43395fd",
    "_score": 0,
    "_source": {
      "Id": "95816ff0-9eae-4196-99fc-86c6f43395fd",
      "Name": "Create Extended/Limited Warranty Configuration"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "07052261-b904-4bfc-a6fd-3acd28114c6a",
    "_score": 0,
    "_source": {
      "Id": "07052261-b904-4bfc-a6fd-3acd28114c6a",
      "Name": "HUC"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "d60daf3a-4681-4bfc-a3a9-b04b5b005f73",
    "_score": 0,
    "_source": {
      "Id": "d60daf3a-4681-4bfc-a3a9-b04b5b005f73",
      "Name": "DAMS UpsertUnenrollPrice"        }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "c1b367f2-a57a-487e-994c-84470e0f9db4",
    "_score": 0,
    "_source": {
      "Id": "c1b367f2-a57a-487e-994c-84470e0f9db4",
      "Name": "Item Setup"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b",
    "_score": 0,
    "_source": {
      "Id": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b",
      "Name": "Installments"        
   }
  }
 ]
}
}

This mentions totalHits as '18'. Why is it returning more items than 10? I believe match query should be used for 'exact' matches, so why more documents are returned here?

P.S.: I know I can use the Ids query for this, but I want to know why is this not returning the correct response

Update: Setting the size to 20 returns the following response:

 {
  "took": 195,
  "timed_out": false,
  "_shards": {
  "total": 5,
 "successful": 5,
 "skipped": 0,
"failed": 0
},
"hits": {
 "total": 18,
 "max_score": 0,
 "hits": [
   {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "9d8060da-c3e2-4f6d-b4e2-17e65b266c76",
    "_score": 0,
    "_source": {
      "Id": "9d8060da-c3e2-4f6d-b4e2-17e65b266c76",
      "Name": "Create Extended/Limited Warranty Configuration"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "4461eb37-487e-4899-a7be-914640fab0e0",
    "_score": 0,
    "_source": {
      "Id": "4461eb37-487e-4899-a7be-914640fab0e0",
      "Name": "Create Extended/Limited Warranty Configuration"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "33f87d98-024f-4893-aa1c-8d438a98cd1f",
    "_score": 0,
    "_source": {
      "Id": "33f87d98-024f-4893-aa1c-8d438a98cd1f",
      "Name": "Create Configurator"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "d75d9a7c-e145-487e-922f-102c16d0026f",
    "_score": 0,
    "_source": {
      "Id": "d75d9a7c-e145-487e-922f-102c16d0026f",
      "Name": "Create Configurator"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "007eadb7-adda-487e-b7fe-6f6b5648de2e",
    "_score": 0,
    "_source": {
      "Id": "007eadb7-adda-487e-b7fe-6f6b5648de2e",
      "Name": "Detail Page - Build"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "95816ff0-9eae-4196-99fc-86c6f43395fd",
    "_score": 0,
    "_source": {
      "Id": "95816ff0-9eae-4196-99fc-86c6f43395fd",
      "Name": "Create Extended/Limited Warranty Configuration"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "07052261-b904-4bfc-a6fd-3acd28114c6a",
    "_score": 0,
    "_source": {
      "Id": "07052261-b904-4bfc-a6fd-3acd28114c6a",
      "Name": "HUC"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "d60daf3a-4681-4bfc-a3a9-b04b5b005f73",
    "_score": 0,
    "_source": {
      "Id": "d60daf3a-4681-4bfc-a3a9-b04b5b005f73",
      "Name": "DAMS UpsertUnenrollPrice"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "c1b367f2-a57a-487e-994c-84470e0f9db4",
    "_score": 0,
    "_source": {
      "Id": "c1b367f2-a57a-487e-994c-84470e0f9db4",
      "Name": "Item Setup"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b",
    "_score": 0,
    "_source": {
      "Id": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b",
      "Name": "Installments"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "b9437079-47c4-487e-abf0-1ff076f69e0f",
    "_score": 0,
    "_source": {
      "Id": "b9437079-47c4-487e-abf0-1ff076f69e0f",
      "Name": "Detail Page - Strings "
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "0aa1db52-c0fb-4bf6-9223-00edccc32703",
    "_score": 0,
    "_source": {
      "Id": "0aa1db52-c0fb-4bf6-9223-00edccc32703",
      "Name": "Create Extended/Limited Warranty Configuration"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "ea8a59a6-2b2f-467a-9beb-e281b1581a0a",
    "_score": 0,
    "_source": {
      "Id": "ea8a59a6-2b2f-467a-9beb-e281b1581a0a",
      "Name": "Create Configurator"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "fd259359-4f6d-4530-ac29-fcebe00d66a6",
    "_score": 0,
    "_source": {
      "Id": "fd259359-4f6d-4530-ac29-fcebe00d66a6",
      "Name": "Invite Platform"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "1b2ba0bb-3e7f-46fb-b904-07460b84848b",
    "_score": 0,
    "_source": {
      "Id": "1b2ba0bb-3e7f-46fb-b904-07460b84848b",
      "Name": "Training"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "8c399993-f273-4ee0-a1ab-3a85c6848113",
    "_score": 0,
    "_source": {
      "Id": "8c399993-f273-4ee0-a1ab-3a85c6848113",
      "Name": "Configure ASIN for Reporting"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "3b385896-1207-4f6d-8ae9-f3ced84cf1fa",
    "_score": 0,
    "_source": {
      "Id": "3b385896-1207-4f6d-8ae9-f3ced84cf1fa",
      "Name": "Create Extended/Limited Warranty Configuration"
    }
  },
  {
    "_index": "requirements_v3",
    "_type": "_doc",
    "_id": "048b7907-2b5a-438a-ace9-f1e1fd67ca69",
    "_score": 0,
    "_source": {
      "Id": "048b7907-2b5a-438a-ace9-f1e1fd67ca69",
      "Name": "Invite Platform"
     }
    }
  ]
 }
}
See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

Lets understand this by the following mapping e.g:

{
  "_doc": {
    "properties": {
      "Id": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Name": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      }
    }
  }
}

The above mapping is created dynamically by elasticsearch. Lets us now focus on Id field. Its type is text. By default the analyzer for text datatype is standard analyzer. When this analyzer is applied on the input for this field it get tokenized into terms. So for example if you input value for Id is 33f87d98-024f-4893-aa1c-8d438a98cd1f following tokens get generated:

33f87d98
024f
4893
aa1c
8d438a98cd1f

As you can see the input value is splitted by - being used as delimiter. This is because standard analyzer is applied on it.

There is another sub-field under Id which is keyword and its type is keyword. For type keyword the input is indexed as it is without applying any modification.

Now lets understand why more documents get matched and result count is more than expected. In your query you used match query on Id field as below:

{
  "match": {
    "Id": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b"
  }
}

By default match query uses the same analyzer that is applied on the field in mapping. So on the Id value in the query again the same analyzer is applied and the input is splitted into tokens in a similar way as above. The default operator that is applied between tokens of match query input string is OR and hence your query actually becomes:

b8bf49a4 OR 960b OR 4fa8 OR 8c5f OR a3fce4b4d07b

There if any of the above tokens match to any of the indexed terms stored in Id field, the document is considered a match.

Solution for the above based on above mapping:

Use the keyword field instead. So the query becomes:

{
  "match": {
    "Id.keyword": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b"
  }
}

More on how match works see here.

Also as mention by @Curious_MInd in his answer its better to use terms than using multiple match in should.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...