ElasticSearch advanced query and indexing principle

Match query [match_all]

• match_all: returns all documents in the index
• match: The search term will be segmented first, and then matched with the target query field. If any word in the segment matches the target field, it can be queried
• match_phrase: Do not divide the search word into words, and require the search word and field content to be matched in an orderly and coherent manner. All words and sequences must be exactly the same, except for punctuation marks
• match_phrase_prefix : Similar to match_phrase, the difference is that prefix matching is allowed

Explain the difference between them with an example

• First store a piece of data i like eating and cooking The default tokenizer should divide the content into “i” “like” “eating ” “and” “kuing”

Summary:

1. match will segment the search term before matching, match_phrase and match_phrase_prefix will not segment the search term
2. match and match_phrase are exact matches, and match_phrase requires an orderly and coherent match between the search term and the field content
3. match_phrase_prefix is not an exact match, it allows the last word to use a prefix match on the basis of match_phrase

Keyword query [term]

term keyword: use keyword query

• keyword type: When using term to query fields of type keyword, all content needs to match
• Integer type, double type, date type: no word segmentation, must match all
• text type: default es standard tokenizer, Chinese word segmentation, English word segmentation

So except for the text type, other types are not participle

The standard tokenizer is used by default in es, Chinese word segmentation, English word segmentation

# query statement
GET /products/_search
{
  "query": {
    "term": {
      "title": {
        "value": "Pigman"
      }
    }
  }
}
#result
"hits": {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.2039728,
    "hits" : [
      {
        "_index" : "products",
        "_type" : "_doc",
        "_id" : "rj5iCYABiB8dDekOlCwE",
        "_score" : 1.2039728,
        "_source" : {
          "id" : 2,
          "title" : "Pigman",
          "price" : 0.5,
          "created_at" : "2022-04-08",
          "description" : "5 cents a pack"
        }
      }
    ]
  }

Range query [range]

range keyword: used to query documents within a specified range

# range query range
GET /products/_search
{
  "query": {
    "range": {
      "Field name": {
        "gte": 2, #lower bound
        "lte": 4 #upper boundary
      }
    }
  }
}

Prefix query [prefix]

prefix keyword: query according to the prefix of the document

# prefix query prefix
GET /products/_search
{
  "query": {
    "prefix": {
      "FIELD": {
        "value": ""
      }
    }
  }
}

Wildcard query [wildcard]

Wildcard queries can be used:

• ? matches a single character
• * matches multiple characters

#wildcard query
GET /products/_search
{
  "query": {
    "wildcard": {
      "FIELD": {
        "value": "VALUE"
      }
    }
  }
}

Query [ids] by id array

Query documents through an id array

#Query through a set of ids
GET /products/_search
{
  "query": {
    "ids": {
      "values": [1,2]
    }
  }
}

Fuzzy query [fuzzy]

Fuzzy search for documents containing specified keywords

Note: fuzzy fuzzy query maximum fuzzy error must be between 0-2

• The length of the search keyword is 2, ambiguity is not allowed
• The length of the search keyword is 3-5, allowing one fuzzy
• The search keyword length is greater than 5, allowing a maximum of 2 blurs

GET /products/_search
{
  "query": {
    "fuzzy": {
      "FIELD": "xxxx"
    }
  }
}

Boolean query[bool]

Elasticsearch can use the bool keyword to combine multiple conditions to achieve complex queries, similar to the operations of AND, OR and NOT in SQL

The Boolean logic types supported by Elasticsearch include the following:

Types include the following:

• must: The document must meet all the query conditions. When multiple conditions are included, it is similar to AND in SQL, and & amp; & amp;

• should: The document must meet any one or more of the query conditions (the number of conditions that need to be satisfied can be specified by minimum_should_match), similar when multiple conditions are included OR in SQL, || in operators

• must_not: The document must not meet all the query conditions, similar to NOT in SQL, and does not participate in the calculation of the score, the returned Branches are all 0

• filter:: first filter out the documents that meet the conditions, and do not calculate the score. Under normal circumstances, we should first use the filter operation to filter out part of the data, and then use the query to accurately match the data to improve query efficiency

must query

When using a must query, documents must match all of the query conditions included therein.

{<!-- -->
  "query": {<!-- -->
    "bool": {<!-- -->
      "must": [
        "term": {<!-- -->
          "age": 20
        }
      ]
    }
  }
}

This query is equivalent to the corresponding SQL statement below

SELECT * FROM xxx WHERE age = 20;

When using must, you can specify multiple query conditions at the same time. In DSL, it is expressed in the form of an array, and the effect is similar to the AND operation in SQL. For example the following example:

{<!-- -->
  "query": {<!-- -->
    "bool": {<!-- -->
      "must": [
        {<!-- --> "term": {<!-- --> "age": 20 } },
        {<!-- --> "term": {<!-- --> "gender": "male" } }
      ]
    }
  }
}

should query

The should query is similar to the OR statement in SQL. When it includes two or more conditions, the query result must satisfy at least one of them. When there is only one query condition, that is, the result must satisfy that condition.

{<!-- -->
  "query": {<!-- -->
    "bool": {<!-- -->
      "should": [
        {<!-- --> "term": {<!-- --> "age": 20 } },
        {<!-- --> "term": {<!-- --> "gender": "male" } },
        {<!-- --> "range": {<!-- --> "height": {<!-- --> "gte": 170 } } },
      ]
    }
  }
}

This query is equivalent to the corresponding SQL statement below:

SELECT * FROM xxx WHERE age = 20 OR gender = "male" or height >= 170;

The difference between the should query and the OR operation in SQL is that the should query can use the minimum_should_match parameter to specify at least Several conditions need to be met. For example, in the following example, the query result needs to meet two or more query conditions:

{<!-- -->
  "query": {<!-- -->
    "bool": {<!-- -->
      "should": [
        {<!-- --> "term": {<!-- --> "age": 20 } },
        {<!-- --> "term": {<!-- --> "gender": "male" } },
        {<!-- --> "term": {<!-- --> "height": 170 } },
      ],
      "minimum_should_match": 2
    }
  }
}

If there is no must or filter in the same bool statement, the default value of minimum_should_match is 1, that is At least one of the conditions must be met; but if other must or filter exist, the default value of minimum_should_match is 0. That is to say, the should query will be invalid by default

For example, in the following query, the age value of all returned documents must be 20, but it may include documents whose status value is not “active”. If you need both to take effect at the same time, you can add a parameter “minimum_should_match”: 1 to the bool query as in the above example.

{<!-- -->
  "query": {<!-- -->
    "bool": {<!-- -->
      "must": {<!-- -->
        "term": {<!-- -->
          "age": 20
        },
      },
      "should": {<!-- -->
        "term": {<!-- -->
          "status": "active"
        }
      },
       "minimum_should_match": 1
    }
  }
}

must_not query

The must_not query is similar to the NOT operation in SQL statements, it will only return documents that do not meet the specified conditions. For example:

{<!-- -->
  "query": {<!-- -->
    "bool": {<!-- -->
      "must_not": [
        {<!-- --> "term": {<!-- --> "age": 20 } },
        {<!-- --> "term": {<!-- --> "gender": "male" } }
      ]
    }
  }
}

This query is equivalent to the following SQL query statement (because MySQL does not support the following statement using NOT, so it is rewritten to use !=):

SELECT * FROM xxx WHERE age != 20 AND gender != "male";

In addition, must_not is the same as filter, and the filter is used to execute without calculating the score of the document, so the corresponding score of the returned result is 0.

filter query

When using filter query, its effect is equivalent to must query, but it is different from must query in that it first filters out the documents that meet the conditions, and does not calculate score

For example, in the following query, all documents whose status is "active" will be returned, and their scores are all 0.0.

{<!-- -->
  "query": {<!-- -->
    "bool": {<!-- -->
      "filter": {<!-- -->
        "term": {<!-- -->
          "status": "active"
        }
      }
    }
  }
}

Boolean combination query