agent-one integrates multiple agents into one agnet tool integration package
Background
Currently there are multiple agents: JMX, skywalking, online stress testing, security agent, etc.
Multiple Agents exist, and the injection method and loading order of each Agent are not fixed, and the user injection process is complicated.
There is no unified status monitoring interface, which is not conducive to management.
Functions that oneagent needs to support:
Support loading the jar in the specified directory, and execute its specified agentStartMethod, the default is premain
Support specifying the order of loaded agents. Set by agentEnableList
Support users to set the type of classLoader, the default is the system class loader
Support adding jar package to bootstrap
Support setting jvm parameters, agent args running parameters
Design
Overall package structure design
Overall process
Configuration design
System configuration loading priority
When the project starts, the system configuration agent_one_sys-{agent_env}.properties will be loaded. The environment can be set by -Dagent_env
Loading priority order:
1 sys_config_path in jvm parameters/system environment variables
2 {path where the agent jar is located}/agent-one/agent-conf/
3 {inside the agent jar package}/META-INF/agentone/
System configuration content
The configuration items of the system configuration can support environment variables/jvm -D parameter settings
# Startup list, multiple agents separated by commas
agent_enable_list=jmx
# The agent's default configuration directory will load the xx_agent_conf.properties file
agent_default_conf_dir=#{agent_one_class_path}/agent-one/agent-conf/
# Global setting whether to start the agent in thread mode
agent_default_enable_thread=true
#Class loader type system: system class loader url: custom urlClassLoader default is system
agent_default_class_type=system
# Agent configuration path, you can configure the configuration of an agent in the system configuration format: {agent name}.{configuration attribute}
# Custom plugin configuration address
jmx.conf_dir=#{agent_one_class_path}/agent-one/agent-conf/
jmx.agent_path=#{agent_one_class_path}/agent-one/agent-modules/jmx_prometheus_javaagent-0.16.2-SNAPSHOT.jar
# The class name of the agent
jmx.agent_class_name=io.prometheus.jmx.shaded.io.prometheus.jmx.JavaAgent
# agent parameter
jmx.agent_param=19090:#{jmx_agent_class_path}/config.yaml
# Agent startup function
jmx.agent_start_method=premain
jmx.jvm_param=
#internal internal agent external agent file file url download file
jmx.agent_type=agent
# file/download address
jmx.filePath=
agent configuration
After loading the system configuration, load the xx_agent_conf.properties file according to the directory address configured by agent_default_conf_dir/xx.conf_dir. If the file does not exist, it will not be loaded.
Environment variables/jvm -D parameter settings are not supported
agent configuration content
name=jmx
agent_path=#{agent_one_class_path}/agent-one/agent-modules/jmx_prometheus_javaagent-0.16.2-SNAPSHOT.jar
# The class name of the agent
agent_class_name=io.prometheus.jmx.shaded.io.prometheus.jmx.JavaAgent
# agent parameters
agent_param=19090:#{jmx_agent_class_path}/config.yaml
# Agent startup function
agent_start_method=premain
jvm_param=
#internal internal agent external agent file file url download file
agent_type=agent
# file/download address
file_path=
Configuration environment variable support
- format#{lable}
- System environment variables and jvm -D parameters all support
- “#{agent_one_class_path} represents the jar package path of the entire agent one”
- “#{xx_agent_class_path} represents the jar package path of xx agent, for example #{jmx_agent_class_path}”
Configuration priority
System configuration supports jvm and environment variable settings, agent configuration does not support
For example: you can set agent_enable_list through jvm parameters or system environment variables to control the list of starting agents
jvm > system environment variables > agent configuration > system configuration
classLoder loading method
Class loader type system: system class loader url: custom urlClassLoader default is system
system: use the system class loader
url: Each agent creates a new URLClassLoader whose parent class is Thread.currentThread().getContextClassLoader().
Third-party plug-in access method
Method 1: url download method
- Configure your own script to download the agent and put it on the file server (oss)
- Configure the download address to the agent configuration file
- The configuration file of the new agent can be set through the xx.conf_dir environment variable. Or directly built into the agent package
- agent_enable_list enable agent
Security iast access agent configuration example:
name=iast
agent_path=/app/deploy/iast/agent.jar
agent_class_name=cn.xmirror.iast.agent
agent_jvm_param=xmiast.ip=xxx;xmiast.port=xxxx;xmiast.projectname=#{SYSTEM_CODE} + #{SERVICE_NAME};xmiast.writeconfig=false;xmiast.nodename=#{IMAGE_ADDRESS}
agent_start_method=premain
agent_class_type=system
agent_type=url
agent_file_path=http://itao-ujp-core-shenzhen-xili1-oss.sit.sf-express.com:8080/v1/AUTH_ITAO-UJP-CORE/ujp/iast.sh
Method 2: Mount file method
- Configure your own script to download the agent and mount it to k8s
- Configure the file address to the configuration file of the agent
- The configuration file of the new agent can be set through the xx.conf_dir environment variable. Or directly built into the agent package
- agent_enable_list enable agent
Security iast access agent configuration example:
name=iast
agent_path=/app/deploy/iast/agent.jar
agent_class_name=cn.xmirror.iast.agent
agent_jvm_param=xmiast.ip=xxxx;xmiast.port=xxxx;xmiast.projectname=#{SYSTEM_CODE} + #{SERVICE_NAME};xmiast.writeconfig=false;xmiast.nodename=#{IMAGE_ADDRESS}
agent_start_method=premain
agent_class_type=system
agent_type=file
agent_file_path=/app/deploy/iast.sh
System configuration class
package com.sf.cnp.agent.config;
import java.util.List;
import java.util.Map;
public class SysConfig {<!-- -->
/**
* Start agent list
*/
List<String> agentEnableList;
/**
* The default path where the agent is located
*/
String agentConfDir;
/**
* key: agentName value: AgentConfig
*/
Map<String,AgentConfig> agentConfigMap;
/**
* Start in thread mode
*/
String enableThread;
/**
* Class loader type system: system class loader url: custom urlClassLoader
*/
String classLoaderType;
}
A configuration class for an agent
package com.sf.cnp.agent.config;
public class AgentConfig {<!-- -->
String name;
/**
* configuration directory
*/
String confDir;
/**
* agent jar path
*/
String agentPath;
/**
* agent class name
*/
String agentClassName;
/**
* parameter
*/
String agentParam;
/**
* jvm parameter
*/
String jvmParam;
/**
* start method
*/
String agentStartMethod;
/**
* Destruction method
*/
String agentStopMethod;
/**
* internal internal agent external agent file file url download file
*/
String type;
/**
* file or url address
*/
String filePath;
/**
* Start in thread mode
*/
String enableThread;
/**
* Class loader type system: system class loader url: custom urlClassLoader
*/
String classLoaderType;
/**
* Whether to add to the bootstrap path
*/
String addBootStrapPath;
public AgentConfig(){<!-- -->
//Defaults
this.type = "agent";
this.agentStartMethod = "premain";
this.addBootStrapPath = "false";
}
}