Table of Contents
1. What is index monitoring?
2. The health status of the index
3. Allocation of index shards
4. Recovery of index fragments
5. Index statistical indicators
1. What is index monitoring
When you want to know the status and statistical indicators of index operation, you need to use the Elasticsearch monitoring endpoint. This article will introduce the various monitoring endpoints of the index, including monitoring the health status of the index, monitoring the segment data, allocation and recovery of the index shards, and monitoring the statistical indicators of the index.
2. Index health status
Create an index order001, the code is as follows
PUT order001
{
"settings": {
"number_of_shards": 3,
"number_of_replicas": 1
},
"mappings": {
"properties": {
"id":{
"type": "keyword"
}
}
}
}
Use the cat endpoint of the index to view the health status of index order001. The code is as follows
GET _cat/indices/order001?v & amp;format=json
The return result is as follows
[
{
"health" : "yellow",
"status" : "open",
"index" : "order001",
"uuid" : "n58cYBLMQiehern0D36c-A",
"pri" : "3",
"rep" : "1",
"docs.count" : "0",
"docs.deleted" : "0",
"store.size" : "624b",
"pri.store.size" : "624b"
}
]
From the above returned results, you can view the index’s health status, running status, number of primary shards and replica shards for each primary shard, total number of existing documents, total number of deleted documents, space occupied by the index, primary shards The amount of space occupied by the slice. Since the index runs on a single node, replica shards cannot be allocated, so the space occupied by the primary shard is the same as the total space occupied by the index. The health status of the index is divided into three types. If there is a primary shard that has not been allocated, the health status is red; if there is a replica shard that has not been allocated, the health status is yellow; if both the primary shard and the replica shard have been allocated distribution, the health status is green. You can start multiple es nodes locally for verification.
3. Allocation of index shards
Use _cat/shards to view the allocation results of each shard of the index. You can also use the _shard_stores endpoint to view the location of allocated shards in the index. The location of unallocated shards is not displayed.
GET order001/_shard_stores
The information of shard storage obtained by using this endpoint is more detailed, including the allocation id and detailed information of the node where the shard is located, as follows.
{
"indices" : {
"order001" : {
"shards" : {
"1" : {
"stores" : [
{
"sby8CsbPRPKoXSyNT9r6Ow" : {
"name" : "365f20b90286",
"ephemeral_id" : "2ij1ohfBRri2o_ezx_bfcg",
"transport_address" : "172.17.0.2:9300",
"attributes" : {
"ml.machine_memory" : "1019383808",
"xpack.installed" : "true",
"transform.node" : "true",
"ml.max_open_jobs" : "20"
}
},
"allocation_id" : "9jdizMerQWaK8u_UHSuL-w",
"allocation" : "primary"
}
]
},
"2" : {
"stores" : [
{
"sby8CsbPRPKoXSyNT9r6Ow" : {
"name" : "365f20b90286",
"ephemeral_id" : "2ij1ohfBRri2o_ezx_bfcg",
"transport_address" : "172.17.0.2:9300",
"attributes" : {
"ml.machine_memory" : "1019383808",
"xpack.installed" : "true",
"transform.node" : "true",
"ml.max_open_jobs" : "20"
}
},
"allocation_id" : "DjFY4N9MQeKo_1MlRHOakg",
"allocation" : "primary"
}
]
},
"0" : {
"stores" : [
{
"sby8CsbPRPKoXSyNT9r6Ow" : {
"name" : "365f20b90286",
"ephemeral_id" : "2ij1ohfBRri2o_ezx_bfcg",
"transport_address" : "172.17.0.2:9300",
"attributes" : {
"ml.machine_memory" : "1019383808",
"xpack.installed" : "true",
"transform.node" : "true",
"ml.max_open_jobs" : "20"
}
},
"allocation_id" : "RypjWGAETP6AiCgkXayzcA",
"allocation" : "primary"
}
]
}
}
}
}
}
4. Recovery of index fragments
Use the _cat/recovery endpoint to view the recovery status of the fragment, and use the _recovery endpoint to view more detailed information about the fragment recovery.
GET order001/_recovery
In the following return result of this endpoint, in addition to the fragment number (id), recovery type (type), start time (start time in millis), end time (stop time in millis), data source (source) and In addition to the regular fields of the target node (target), it also contains statistical information during the fragment recovery process, such as how many files were recovered (files.total), how much space they occupied (total in bytes), and the number of recovered transaction logs. Number (translog), etc.
{
"order001" : {
"shards" : [
{
"id" : 1,
"type" : "EMPTY_STORE",
"stage" : "DONE",
"primary" : true,
"start_time_in_millis" : 1698130829083,
"stop_time_in_millis" : 1698130829216,
"total_time_in_millis" : 133,
"source" : { },
"target" : {
"id" : "sby8CsbPRPKoXSyNT9r6Ow",
"host" : "172.17.0.2",
"transport_address" : "172.17.0.2:9300",
"ip" : "172.17.0.2",
"name" : "365f20b90286"
},
"index" : {
"size" : {
"total_in_bytes" : 0,
"reused_in_bytes" : 0,
"recovered_in_bytes" : 0,
"percent" : "0.0%"
},
"files" : {
"total" : 0,
"reused" : 0,
"recovered" : 0,
"percent" : "0.0%"
},
"total_time_in_millis" : 61,
"source_throttle_time_in_millis" : 0,
"target_throttle_time_in_millis" : 0
},
"translog" : {
"recovered" : 0,
"total" : 0,
"percent" : "100.0%",
"total_on_start" : 0,
"total_time_in_millis" : 49
},
"verify_index" : {
"check_index_time_in_millis" : 0,
"total_time_in_millis" : 0
}
},
{
"id" : 2,
"type" : "EMPTY_STORE",
"stage" : "DONE",
"primary" : true,
"start_time_in_millis" : 1698130829103,
"stop_time_in_millis" : 1698130829216,
"total_time_in_millis" : 113,
"source" : { },
"target" : {
"id" : "sby8CsbPRPKoXSyNT9r6Ow",
"host" : "172.17.0.2",
"transport_address" : "172.17.0.2:9300",
"ip" : "172.17.0.2",
"name" : "365f20b90286"
},
"index" : {
"size" : {
"total_in_bytes" : 0,
"reused_in_bytes" : 0,
"recovered_in_bytes" : 0,
"percent" : "0.0%"
},
"files" : {
"total" : 0,
"reused" : 0,
"recovered" : 0,
"percent" : "0.0%"
},
"total_time_in_millis" : 62,
"source_throttle_time_in_millis" : 0,
"target_throttle_time_in_millis" : 0
},
"translog" : {
"recovered" : 0,
"total" : 0,
"percent" : "100.0%",
"total_on_start" : 0,
"total_time_in_millis" : 42
},
"verify_index" : {
"check_index_time_in_millis" : 0,
"total_time_in_millis" : 0
}
},
{
"id" : 0,
"type" : "EMPTY_STORE",
"stage" : "DONE",
"primary" : true,
"start_time_in_millis" : 1698130829110,
"stop_time_in_millis" : 1698130829217,
"total_time_in_millis" : 107,
"source" : { },
"target" : {
"id" : "sby8CsbPRPKoXSyNT9r6Ow",
"host" : "172.17.0.2",
"transport_address" : "172.17.0.2:9300",
"ip" : "172.17.0.2",
"name" : "365f20b90286"
},
"index" : {
"size" : {
"total_in_bytes" : 0,
"reused_in_bytes" : 0,
"recovered_in_bytes" : 0,
"percent" : "0.0%"
},
"files" : {
"total" : 0,
"reused" : 0,
"recovered" : 0,
"percent" : "0.0%"
},
"total_time_in_millis" : 59,
"source_throttle_time_in_millis" : 0,
"target_throttle_time_in_millis" : 0
},
"translog" : {
"recovered" : 0,
"total" : 0,
"percent" : "100.0%",
"total_on_start" : 0,
"total_time_in_millis" : 39
},
"verify_index" : {
"check_index_time_in_millis" : 0,
"total_time_in_millis" : 0
}
}
]
}
}
5. Statistical indicators of index
Elasticsearch provides a statistical indicator viewing endpoint through which you can view the statistical data of each index. The calling method is as follows
GET order001/_stats
The above request queries the statistical indicators of index order001, and the returned results are as follows
{
"_shards" : {
"total" : 6,
"successful" : 3,
"failed" : 0
},
"_all" : {
"primaries" : {
"docs" : {
"count" : 0,
"deleted" : 0
},
"store" : {
"size_in_bytes" : 624,
"reserved_in_bytes" : 0
},
"indexing" : {
"index_total" : 0,
"index_time_in_millis" : 0,
"index_current" : 0,
"index_failed" : 0,
"delete_total" : 0,
"delete_time_in_millis" : 0,
"delete_current" : 0,
"noop_update_total" : 0,
"is_throttled" : false,
"throttle_time_in_millis" : 0
},
"get" : {
"total" : 0,
"time_in_millis" : 0,
"exists_total" : 0,
"exists_time_in_millis" : 0,
"missing_total" : 0,
"missing_time_in_millis" : 0,
"current" : 0
},
"search" : {
"open_contexts" : 0,
"query_total" : 0,
"query_time_in_millis" : 0,
"query_current" : 0,
"fetch_total" : 0,
"fetch_time_in_millis" : 0,
"fetch_current" : 0,
"scroll_total" : 0,
"scroll_time_in_millis" : 0,
"scroll_current" : 0,
"suggest_total" : 0,
"suggest_time_in_millis" : 0,
"suggest_current" : 0
},
"merges" : {
"current" : 0,
"current_docs" : 0,
"current_size_in_bytes" : 0,
"total" : 0,
"total_time_in_millis" : 0,
"total_docs" : 0,
"total_size_in_bytes" : 0,
"total_stopped_time_in_millis" : 0,
"total_throttled_time_in_millis" : 0,
"total_auto_throttle_in_bytes" : 62914560
},
"refresh" : {
"total" : 6,
"total_time_in_millis" : 0,
"external_total" : 6,
"external_total_time_in_millis" : 0,
"listeners" : 0
},
"flush" : {
"total" : 3,
"periodic" : 0,
"total_time_in_millis" : 0
},
"warmer" : {
"current" : 0,
"total" : 3,
"total_time_in_millis" : 0
},
"query_cache" : {
"memory_size_in_bytes" : 0,
"total_count" : 0,
"hit_count" : 0,
"miss_count" : 0,
"cache_size" : 0,
"cache_count" : 0,
"evictions" : 0
},
"fielddata" : {
"memory_size_in_bytes" : 0,
"evictions" : 0
},
"completion" : {
"size_in_bytes" : 0
},
"segments" : {
"count" : 0,
"memory_in_bytes" : 0,
"terms_memory_in_bytes" : 0,
"stored_fields_memory_in_bytes" : 0,
"term_vectors_memory_in_bytes" : 0,
"norms_memory_in_bytes" : 0,
"points_memory_in_bytes" : 0,
"doc_values_memory_in_bytes" : 0,
"index_writer_memory_in_bytes" : 0,
"version_map_memory_in_bytes" : 0,
"fixed_bit_set_memory_in_bytes" : 0,
"max_unsafe_auto_id_timestamp" : -1,
"file_sizes" : { }
},
"translog" : {
"operations" : 0,
"size_in_bytes" : 165,
"uncommitted_operations" : 0,
"uncommitted_size_in_bytes" : 165,
"earliest_last_modified_age" : 0
},
"request_cache" : {
"memory_size_in_bytes" : 0,
"evictions" : 0,
"hit_count" : 0,
"miss_count" : 0
},
"recovery" : {
"current_as_source" : 0,
"current_as_target" : 0,
"throttle_time_in_millis" : 0
}
},
"total" : {
"docs" : {
"count" : 0,
"deleted" : 0
},
"store" : {
"size_in_bytes" : 624,
"reserved_in_bytes" : 0
},
"indexing" : {
"index_total" : 0,
"index_time_in_millis" : 0,
"index_current" : 0,
"index_failed" : 0,
"delete_total" : 0,
"delete_time_in_millis" : 0,
"delete_current" : 0,
"noop_update_total" : 0,
"is_throttled" : false,
"throttle_time_in_millis" : 0
},
"get" : {
"total" : 0,
"time_in_millis" : 0,
"exists_total" : 0,
"exists_time_in_millis" : 0,
"missing_total" : 0,
"missing_time_in_millis" : 0,
"current" : 0
},
"search" : {
"open_contexts" : 0,
"query_total" : 0,
"query_time_in_millis" : 0,
"query_current" : 0,
"fetch_total" : 0,
"fetch_time_in_millis" : 0,
"fetch_current" : 0,
"scroll_total" : 0,
"scroll_time_in_millis" : 0,
"scroll_current" : 0,
"suggest_total" : 0,
"suggest_time_in_millis" : 0,
"suggest_current" : 0
},
"merges" : {
"current" : 0,
"current_docs" : 0,
"current_size_in_bytes" : 0,
"total" : 0,
"total_time_in_millis" : 0,
"total_docs" : 0,
"total_size_in_bytes" : 0,
"total_stopped_time_in_millis" : 0,
"total_throttled_time_in_millis" : 0,
"total_auto_throttle_in_bytes" : 62914560
},
"refresh" : {
"total" : 6,
"total_time_in_millis" : 0,
"external_total" : 6,
"external_total_time_in_millis" : 0,
"listeners" : 0
},
"flush" : {
"total" : 3,
"periodic" : 0,
"total_time_in_millis" : 0
},
"warmer" : {
"current" : 0,
"total" : 3,
"total_time_in_millis" : 0
},
"query_cache" : {
"memory_size_in_bytes" : 0,
"total_count" : 0,
"hit_count" : 0,
"miss_count" : 0,
"cache_size" : 0,
"cache_count" : 0,
"evictions" : 0
},
"fielddata" : {
"memory_size_in_bytes" : 0,
"evictions" : 0
},
"completion" : {
"size_in_bytes" : 0
},
"segments" : {
"count" : 0,
"memory_in_bytes" : 0,
"terms_memory_in_bytes" : 0,
"stored_fields_memory_in_bytes" : 0,
"term_vectors_memory_in_bytes" : 0,
"norms_memory_in_bytes" : 0,
"points_memory_in_bytes" : 0,
"doc_values_memory_in_bytes" : 0,
"index_writer_memory_in_bytes" : 0,
"version_map_memory_in_bytes" : 0,
"fixed_bit_set_memory_in_bytes" : 0,
"max_unsafe_auto_id_timestamp" : -1,
"file_sizes" : { }
},
"translog" : {
"operations" : 0,
"size_in_bytes" : 165,
"uncommitted_operations" : 0,
"uncommitted_size_in_bytes" : 165,
"earliest_last_modified_age" : 0
},
"request_cache" : {
"memory_size_in_bytes" : 0,
"evictions" : 0,
"hit_count" : 0,
"miss_count" : 0
},
"recovery" : {
"current_as_source" : 0,
"current_as_target" : 0,
"throttle_time_in_millis" : 0
}
}
},
"indices" : {
"order001" : {
"uuid" : "n58cYBLMQiehern0D36c-A",
"primaries" : {
"docs" : {
"count" : 0,
"deleted" : 0
},
"store" : {
"size_in_bytes" : 624,
"reserved_in_bytes" : 0
},
"indexing" : {
"index_total" : 0,
"index_time_in_millis" : 0,
"index_current" : 0,
"index_failed" : 0,
"delete_total" : 0,
"delete_time_in_millis" : 0,
"delete_current" : 0,
"noop_update_total" : 0,
"is_throttled" : false,
"throttle_time_in_millis" : 0
},
"get" : {
"total" : 0,
"time_in_millis" : 0,
"exists_total" : 0,
"exists_time_in_millis" : 0,
"missing_total" : 0,
"missing_time_in_millis" : 0,
"current" : 0
},
"search" : {
"open_contexts" : 0,
"query_total" : 0,
"query_time_in_millis" : 0,
"query_current" : 0,
"fetch_total" : 0,
"fetch_time_in_millis" : 0,
"fetch_current" : 0,
"scroll_total" : 0,
"scroll_time_in_millis" : 0,
"scroll_current" : 0,
"suggest_total" : 0,
"suggest_time_in_millis" : 0,
"suggest_current" : 0
},
"merges" : {
"current" : 0,
"current_docs" : 0,
"current_size_in_bytes" : 0,
"total" : 0,
"total_time_in_millis" : 0,
"total_docs" : 0,
"total_size_in_bytes" : 0,
"total_stopped_time_in_millis" : 0,
"total_throttled_time_in_millis" : 0,
"total_auto_throttle_in_bytes" : 62914560
},
"refresh" : {
"total" : 6,
"total_time_in_millis" : 0,
"external_total" : 6,
"external_total_time_in_millis" : 0,
"listeners" : 0
},
"flush" : {
"total" : 3,
"periodic" : 0,
"total_time_in_millis" : 0
},
"warmer" : {
"current" : 0,
"total" : 3,
"total_time_in_millis" : 0
},
"query_cache" : {
"memory_size_in_bytes" : 0,
"total_count" : 0,
"hit_count" : 0,
"miss_count" : 0,
"cache_size" : 0,
"cache_count" : 0,
"evictions" : 0
},
"fielddata" : {
"memory_size_in_bytes" : 0,
"evictions" : 0
},
"completion" : {
"size_in_bytes" : 0
},
"segments" : {
"count" : 0,
"memory_in_bytes" : 0,
"terms_memory_in_bytes" : 0,
"stored_fields_memory_in_bytes" : 0,
"term_vectors_memory_in_bytes" : 0,
"norms_memory_in_bytes" : 0,
"points_memory_in_bytes" : 0,
"doc_values_memory_in_bytes" : 0,
"index_writer_memory_in_bytes" : 0,
"version_map_memory_in_bytes" : 0,
"fixed_bit_set_memory_in_bytes" : 0,
"max_unsafe_auto_id_timestamp" : -1,
"file_sizes" : { }
},
"translog" : {
"operations" : 0,
"size_in_bytes" : 165,
"uncommitted_operations" : 0,
"uncommitted_size_in_bytes" : 165,
"earliest_last_modified_age" : 0
},
"request_cache" : {
"memory_size_in_bytes" : 0,
"evictions" : 0,
"hit_count" : 0,
"miss_count" : 0
},
"recovery" : {
"current_as_source" : 0,
"current_as_target" : 0,
"throttle_time_in_millis" : 0
}
},
"total" : {
"docs" : {
"count" : 0,
"deleted" : 0
},
"store" : {
"size_in_bytes" : 624,
"reserved_in_bytes" : 0
},
"indexing" : {
"index_total" : 0,
"index_time_in_millis" : 0,
"index_current" : 0,
"index_failed" : 0,
"delete_total" : 0,
"delete_time_in_millis" : 0,
"delete_current" : 0,
"noop_update_total" : 0,
"is_throttled" : false,
"throttle_time_in_millis" : 0
},
"get" : {
"total" : 0,
"time_in_millis" : 0,
"exists_total" : 0,
"exists_time_in_millis" : 0,
"missing_total" : 0,
"missing_time_in_millis" : 0,
"current" : 0
},
"search" : {
"open_contexts" : 0,
"query_total" : 0,
"query_time_in_millis" : 0,
"query_current" : 0,
"fetch_total" : 0,
"fetch_time_in_millis" : 0,
"fetch_current" : 0,
"scroll_total" : 0,
"scroll_time_in_millis" : 0,
"scroll_current" : 0,
"suggest_total" : 0,
"suggest_time_in_millis" : 0,
"suggest_current" : 0
},
"merges" : {
"current" : 0,
"current_docs" : 0,
"current_size_in_bytes" : 0,
"total" : 0,
"total_time_in_millis" : 0,
"total_docs" : 0,
"total_size_in_bytes" : 0,
"total_stopped_time_in_millis" : 0,
"total_throttled_time_in_millis" : 0,
"total_auto_throttle_in_bytes" : 62914560
},
"refresh" : {
"total" : 6,
"total_time_in_millis" : 0,
"external_total" : 6,
"external_total_time_in_millis" : 0,
"listeners" : 0
},
"flush" : {
"total" : 3,
"periodic" : 0,
"total_time_in_millis" : 0
},
"warmer" : {
"current" : 0,
"total" : 3,
"total_time_in_millis" : 0
},
"query_cache" : {
"memory_size_in_bytes" : 0,
"total_count" : 0,
"hit_count" : 0,
"miss_count" : 0,
"cache_size" : 0,
"cache_count" : 0,
"evictions" : 0
},
"fielddata" : {
"memory_size_in_bytes" : 0,
"evictions" : 0
},
"completion" : {
"size_in_bytes" : 0
},
"segments" : {
"count" : 0,
"memory_in_bytes" : 0,
"terms_memory_in_bytes" : 0,
"stored_fields_memory_in_bytes" : 0,
"term_vectors_memory_in_bytes" : 0,
"norms_memory_in_bytes" : 0,
"points_memory_in_bytes" : 0,
"doc_values_memory_in_bytes" : 0,
"index_writer_memory_in_bytes" : 0,
"version_map_memory_in_bytes" : 0,
"fixed_bit_set_memory_in_bytes" : 0,
"max_unsafe_auto_id_timestamp" : -1,
"file_sizes" : { }
},
"translog" : {
"operations" : 0,
"size_in_bytes" : 165,
"uncommitted_operations" : 0,
"uncommitted_size_in_bytes" : 165,
"earliest_last_modified_age" : 0
},
"request_cache" : {
"memory_size_in_bytes" : 0,
"evictions" : 0,
"hit_count" : 0,
"miss_count" : 0
},
"recovery" : {
"current_as_source" : 0,
"current_as_target" : 0,
"throttle_time_in_millis" : 0
}
}
}
}
}
Among them, the content in _all is the statistical data of the two indexes, and indices contains separate statistical data for each index. The statistics of each index also include the statistics of primary shards (primaries) and the statistics of all shards (total). The indicators of the statistical data of the index of the stats endpoint are as shown in the figure
If you feel that too many indicators are returned, you can pass parameters to specify the statistical indicators that need to be viewed. The following request only views the statistics of index segment merging and refresh times.
GET /order001/_stats/merge,refresh