How to use viewer filtering and search functions to locate data?

Foreword

We’ve discussed how Observation Cloud enables more comprehensive data correlation analysis by connecting built-in views to viewers. (See “Built-in view linkage viewer to achieve data correlation analysis”) The viewer mentioned here is actually a comprehensive and powerful data viewing and analysis tool. It provides a variety of search and filtering methods, and supports combinations to obtain final data results.

In this article, we will start from actual scenarios and tell youhow to use the search and filter functions of the viewer to help you quickly and accurately retrieve data and locate faults.

The viewer of the observation cloud can be used in functional modules such as infrastructure, events, logs, application performance monitoring, user access monitoring, CI visualization, availability monitoring, and security inspection. Here, we use [Log Viewer] as the scenario:

Search

When we are faced with the massive and messy data collected by the system, searching directly in the search bar at the top of the page is one of the fastest and fastest ways to locate the target data. In Observation Cloud, [Search] generally consists of terms (can be words or phrases) and operators. When we enter a word or phrase, superimpose the wildcard operator (? means matching any character, * matches 0 or more characters), and press Enter to realize the search action.

Specific examples are as follows:

# word
guance // precise search
guanc[e // There are examples of how to write special characters (no need to add \escaping)

# Word wildcard search (for performance reasons, the observation cloud does not currently support the prefix * writing method. If there is a wildcard search, the following writing methods are supported)
Guance*
gua?ce*
gua*ce

# Phrase (the content enclosed in double quotation marks is collectively called a phrase. In this way, the content in double quotation marks will be used as a whole to initiate a matching search)
"guance test" // Query the matching results of "guance test" content in the full-text index field
"guance 127.0.0.1" // Example of writing when special characters exist

The actual effect is as follows:

In addition to the search methods we mentioned above, Observation Cloud also supports you to use JSON search in the Log Viewer. By default, this search method accurately retrieves the content of message in the log viewer. It also requires that message be in JSON format, and log content in other formats is required. This search method is not supported. For example, we can enter search content in the format of @key:value in the search bar. If it is multi-level JSON, it can be connected with “.”, that is, @key1.key2:value.

It should be noted that this search method currently only supports the log viewer, and only supports China Region 1 (Hangzhou) and China Region 3 (Zhangjiakou) and China Region 4 (Guangzhou) are the three sites, and the workspace needs to be created after June 23, 2022.

Specific examples are as follows:

message information is as follows:
{
    __namespace:tracing,
    cluster_name_k8s:k8s-demo,
    meta:{
        service:ruoyi-mysql-k8s,
        name:mysql.query,
        resource:select dict_code, dict_sort, dict_label, dict_value, dict_type, css_class, list_class, is_default, status, create_by, create_time, remark
                from sys_dict_data
 }
}

# Query cluster_name_k8s = k8s-demo
@cluster_name_k8s:k8s-demo // Exact matching
@cluster_name_k8s:k?s* // Fuzzy matching

# Query service under meta = ruoyi-mysql-k8s
@meta.service:ruoyi-mysql-k8s // Accurate matching
@meta.service:ruoyi?mysql* // Fuzzy matching

Filter

In addition to the search mentioned above, entering filter conditions in the search bar is also a quick way to query. We can filter the format of tag/attribute and splice it in the order of field operator value.

Field

Fields are the prerequisite for filtering conditions to be established. When we add fields according to actual needs, you can choose that the observation cloud will list the fields in the current time range by default. If not found, you can add fields customarily.

See Field Management – Observation Cloud Documentation

Operator

Operators are used to connect fields to values. We can filter on [String] or [Value] type fields.

String field operators: = ≠ wildcard not wildcard exist not exist;
Numeric field operators: = ≠ > >= < < = [xx TO xx] exist not exist.

Descriptions of the different operators are as follows:

Operator	Description
=	Equal, example: key:value, = ≠ can be used in combination with other operators below.
≠	Not equal, example: -key:value, = ≠ can be combined with other operators below.
wildcard	Fuzzy matching, example: key:value*, reverse filtering is achieved by superposition ≠.
exist	exists, filter and return all data results that exist for the current key, example: key:*. Reverse filtering is achieved by superposition ≠.
>	is greater than, example: key:>value. Reverse filtering is achieved by superposition ≠.
>=	Greater than or equal to, example: key:>=value. Reverse filtering is achieved by superposition ≠.
<	Less than, example: key:
<=	Less than or equal to, example: key:<=value. Reverse filtering is achieved by superposition ≠.
[xx - xx]	Interval, example: key: [1 - 100]. Reverse filtering is achieved by superposition ≠.

value

Value is the desired target value within the filter. We can combine it with the [AND] or [OR] operator to query.

Specific examples are as follows:

Exact value
key:(value1 AND value2 OR value3)

# Contains fuzzy matching
key:(value1 OR test* OR value3)

# Contains * exists
key:(value1 OR * OR value3) // Equivalent to key:*
key:(value1 AND *) // Equivalent to key:value1
key:(value1 AND * OR value3) // Equivalent to key:(value1 OR value3)

The form AND/OR/NOT we use here is known in technical terms as a Boolean operator. We can further combine such operatorswith the search and filter criteria mentioned above to associate searches and filters.

Descriptions of the different operators are as follows:

Logical relationship	Description	Remarks
a AND b	Get the intersection of previous and previous query results	By default, `AND` is used to connect search and filter conditions. AND can use spaces, that is, `a` AND `b` = `a` `b`.
a OR b	Get the union of the previous and previous query results	The returned result must contain any keyword of a or b. Example: `a` OR `b:value`
NOT c	Exclude the current query results	NOT is mostly used for search writing. Use `≠` instead for filtering exclusion logic.

The actual effect is as follows:

Quick filter

In addition to entering filter conditions in the search bar, we can also directly use the Observation Cloud> Quick Filter function. Here, we can perform positive and negative selection, field value search, add display column, query value TOP 5 and other operations based on a series of fields to locate the target data we need.

For more related operation instructions, see the power of the viewer - Observation Cloud Documentation

Conclusion

The complexity and massiveness of data are huge challenges facing business data management. When faced with the large amount of data collected and saved by our system, if we can use accurate and effective search or filtering methods to filter and query the data, our data processing efficiency will be doubled with half the effort. In the future, Observation Cloud will also explore more viewer-related query and analysis functions, so stay tuned!