Skywalking process analysis_2 (configuration loading and custom class loader initialization)

Read configuration

SnifferConfigInitializer.initializeCoreConfig(agentArgs)This method is to read the configuration file. The file agent.config is read in this method

public static void initializeCoreConfig(String agentOptions) {<!-- -->
    //Start loading configuration information priority (the smaller the number, the greater the priority) 1: The age of the startup command
    nt parameter 2: system environment variable 3: configuration of agent.config
    AGENT_SETTINGS = new Properties();
    //Read configuration file
    try (final InputStreamReader configFileStream = loadConfig()) {<!-- -->
        AGENT_SETTINGS.load(configFileStream);
        for (String key : AGENT_SETTINGS.stringPropertyNames()) {<!-- -->
            String value = (String) AGENT_SETTINGS.get(key);
            //Configuration item placeholder replacement
            AGENT_SETTINGS.put(key, PropertyPlaceholderHelper.INSTANCE.replacePlaceholders(value, AGENT_SETTINGS));
        }

    } catch (Exception e) {<!-- -->
        LOGGER.error(e, "Failed to read the config file, skywalking is going to run in default config.");
    }

    try {<!-- -->
        //Replace if environment variables are configured
        overrideConfigBySystemProp();
    } catch (Exception e) {<!-- -->
        LOGGER.error(e, "Failed to read the system properties.");
    }

    agentOptions = StringUtil.trim(agentOptions, ',');
    if (!StringUtil.isEmpty(agentOptions)) {<!-- -->
        try {<!-- -->
            agentOptions = agentOptions.trim();
            LOGGER.info("Agent options is {}.", agentOptions);
            //Replace with the agent's configuration file
            overrideConfigByAgentOptions(agentOptions);
        } catch (Exception e) {<!-- -->
            LOGGER.error(e, "Failed to parse the agent options, val is {}.", agentOptions);
        }
    }
    //Map configuration information into the Config class
    initializeConfig(Config.class);
    // reconfigure logger after config initialization
    //Configure logs based on configuration information
    configureLogger();
    LOGGER = LogManager.getLogger(SnifferConfigInitializer.class);

    setAgentVersion();

    // Check if the name and address are set
    if (StringUtil.isEmpty(Config.Agent.SERVICE_NAME)) {<!-- -->
        throw new ExceptionInInitializerError("`agent.service_name` is missing.");
    } else {<!-- -->
        if (StringUtil.isNotEmpty(Config.Agent.NAMESPACE) || StringUtil.isNotEmpty(Config.Agent.CLUSTER)) {<!-- -->
            Config.Agent.SERVICE_NAME = StringUtil.join(
                SERVICE_NAME_PART_CONNECTOR,
                Config.Agent.SERVICE_NAME,
                Config.Agent.NAMESPACE,
                Config.Agent.CLUSTER
            );
        }
    }
    if (StringUtil.isEmpty(Config.Collector.BACKEND_SERVICE)) {<!-- -->
        throw new ExceptionInInitializerError("`collector.backend_service` is missing.");
    }
    if (Config.Plugin.PEER_MAX_LENGTH <= 3) {<!-- -->
        LOGGER.warn(
            "PEER_MAX_LENGTH configuration:{} error, the default value of 200 will be used.",
            Config.Plugin.PEER_MAX_LENGTH
        );
        Config.Plugin.PEER_MAX_LENGTH = 200;
    }
    //Initialization completion identifier
    IS_INIT_COMPLETED = true;
}

Summary

• Read the configuration into AGENT_SETTINGS according to priority
• Load AGENT_SETTINGS into the Config class
• IS_INIT_COMPLETEDThe initialization completion identifier is set to true

Load the plug-in and initialize the custom AgentClassLoader class loader

pluginFinder = new PluginFinder(new PluginBootstrap().loadPlugins());
First, analyze the new PluginBootstrap().loadPlugins() method. It is very important to initialize the custom AgentClassLoader class loader.

new PluginBootstrap().loadPlugins()

public List<AbstractClassEnhancePluginDefine> loadPlugins() throws AgentPackageNotFoundException {<!-- -->
    //Customize AgentClassLoader class loader
    AgentClassLoader.initDefaultLoader();
    //Get all skywalking-plugin.def files
    PluginResourcesResolver resolver = new PluginResourcesResolver();
    List<URL> resources = resolver.getResources();

    if (resources == null || resources.size() == 0) {<!-- -->
        LOGGER.info("no plugin files (skywalking-plugin.def) found, continue to start application.");
        return new ArrayList<AbstractClassEnhancePluginDefine>();
    }
    
    for (URL pluginUrl : resources) {<!-- -->
        try {<!-- -->
            //Read the specified plug-in configuration in skywalking-plugin.def
            PluginCfg.INSTANCE.load(pluginUrl.openStream());
        } catch (Throwable t) {<!-- -->
            LOGGER.error(t, "plugin file [{}] init failure.", pluginUrl);
        }
    }

    List<PluginDefine> pluginClassList = PluginCfg.INSTANCE.getPluginClassList();

    List<AbstractClassEnhancePluginDefine> plugins = new ArrayList<AbstractClassEnhancePluginDefine>();
    for (PluginDefine pluginDefine : pluginClassList) {<!-- -->
        try {<!-- -->
            LOGGER.debug("loading plugin class {}.", pluginDefine.getDefineClass());
            //The class loader loads the plug-in of the plugin
            AbstractClassEnhancePluginDefine plugin = (AbstractClassEnhancePluginDefine) Class.forName(pluginDefine.getDefineClass(), true, AgentClassLoader
                .getDefault()).newInstance();
            plugins.add(plugin);
        } catch (Throwable t) {<!-- -->
            LOGGER.error(t, "load plugin [{}] failure.", pluginDefine.getDefineClass());
        }
    }
    //Load the plug-in based on xml definition
    plugins.addAll(DynamicPluginLoader.INSTANCE.load(AgentClassLoader.getDefault()));

    return plugins;

}

Customized AgentClassLoader class loader

AgentClassLoader.initDefaultLoader()

public class AgentClassLoader extends ClassLoader {<!-- -->
  static {<!-- -->
      /*
       * Try to solve the classloader dead lock. See https://github.com/apache/skywalking/pull/2016
       *
       * Register parallel capability concurrent class loader
       */
      registerAsParallelCapable();
  }
  /**
   * Initialize the AgentClassLoader class loader
   * */
  public static void initDefaultLoader() throws AgentPackageNotFoundException {<!-- -->
      if (DEFAULT_LOADER == null) {<!-- -->
          synchronized (AgentClassLoader.class) {<!-- -->
              if (DEFAULT_LOADER == null) {<!-- -->
                  DEFAULT_LOADER = new AgentClassLoader(PluginBootstrap.class.getClassLoader());
              }
          }
      }
  }

  public AgentClassLoader(ClassLoader parent) throws AgentPackageNotFoundException {<!-- -->
      super(parent);
      //The directory where the jar package is located
      File agentDictionary = AgentPackagePath.getPath();
      classpath = new LinkedList<>();
      //Load plugins in the plugins, activations directory
      //public static List<String> MOUNT = Arrays.asList("plugins", "activations");
      Config.Plugin.MOUNT.forEach(mountFolder -> classpath.add(new File(agentDictionary, mountFolder)));
  }
//Omit
}

Summary

• registerAsParallelCapable() is to solve the ClassLoader deadlock problem and enable the parallel loading mode of the class loader.

  • Before jdk1.7, the class loader loaded classes serially. After loading the previous one, it loaded the next one, which was inefficient.
  • After JDK1.7, the class loader parallel capability is provided, which is to reduce the lock granularity. Previously, ClassLoader used itself as the lock when loading a class. Now it is optimized to lock on a specific class. Instead of locking the entire class loader

• AgentClassLoader overrides the findClass, findResource, findResources methods and reads the plug-in collection classpath from the plugins, activations directory code> to load

@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {<!-- -->
    List<Jar> allJars = getAllJars();
    String path = name.replace('.', '/').concat(".class");
    for (Jar jar : allJars) {<!-- -->
        JarEntry entry = jar.jarFile.getJarEntry(path);
        if (entry == null) {<!-- -->
            continue;
        }
        try {<!-- -->
            URL classFileUrl = new URL("jar:file:" + jar.sourceFile.getAbsolutePath() + "!/" + path);
            byte[] data;
            try (final BufferedInputStream is = new BufferedInputStream(
                classFileUrl.openStream()); final ByteArrayOutputStream baos = new ByteArrayOutputStream()) {<!-- -->
                int ch;
                while ((ch = is.read()) != -1) {<!-- -->
                    baos.write(ch);
                }
                data = baos.toByteArray();
            }
            //Read the configuration with PluginConfig annotation
            return processLoadedClass(defineClass(name, data, 0, data.length));
        } catch (IOException e) {<!-- -->
            LOGGER.error(e, "find class fail.");
        }
    }
    throw new ClassNotFoundException("Can't find " + name);
}

@Override
protected URL findResource(String name) {<!-- -->
    List<Jar> allJars = getAllJars();
    for (Jar jar : allJars) {<!-- -->
        JarEntry entry = jar.jarFile.getJarEntry(name);
        if (entry != null) {<!-- -->
            try {<!-- -->
                return new URL("jar:file:" + jar.sourceFile.getAbsolutePath() + "!/" + name);
            } catch (MalformedURLException ignored) {<!-- -->
            }
        }
    }
    return null;
}

@Override
protected Enumeration<URL> findResources(String name) throws IOException {<!-- -->
    List<URL> allResources = new LinkedList<>();
    List<Jar> allJars = getAllJars();
    for (Jar jar : allJars) {<!-- -->
        JarEntry entry = jar.jarFile.getJarEntry(name);
        if (entry != null) {<!-- -->
            allResources.add(new URL("jar:file:" + jar.sourceFile.getAbsolutePath() + "!/" + name));
        }
    }

    final Iterator<URL> iterator = allResources.iterator();
    return new Enumeration<URL>() {<!-- -->
        @Override
        public boolean hasMoreElements() {<!-- -->
            return iterator.hasNext();
        }

        @Override
        public URL nextElement() {<!-- -->
            return iterator.next();
        }
    };
}

private Class<?> processLoadedClass(Class<?> loadedClass) {<!-- -->
    final PluginConfig pluginConfig = loadedClass.getAnnotation(PluginConfig.class);
    if (pluginConfig != null) {<!-- -->
        // Set up the plugin config when loaded by class loader at the first time.
        // Agent class loader just loaded limited classes in the plugin jar(s), so the cost of this
        // isAssignableFrom would be also very limited.
        SnifferConfigInitializer.initializeConfig(pluginConfig.root());
    }

    return loadedClass;
}

private List<Jar> getAllJars() {<!-- -->
    if (allJars == null) {<!-- -->
        jarScanLock.lock();
        try {<!-- -->
            if (allJars == null) {<!-- -->
                allJars = doGetJars();
            }
        } finally {<!-- -->
            jarScanLock.unlock();
        }
    }

    return allJars;
}

private LinkedList<Jar> doGetJars() {<!-- -->
    LinkedList<Jar> jars = new LinkedList<>();
    //classpath is the plug-in collection just loaded
    for (File path : classpath) {<!-- -->
        if (path.exists() & amp; & amp; path.isDirectory()) {<!-- -->
            String[] jarFileNames = path.list((dir, name) -> name.endsWith(".jar"));
            for (String fileName : jarFileNames) {<!-- -->
                try {<!-- -->
                    File file = new File(path, fileName);
                    Jar jar = new Jar(new JarFile(file), file);
                    jars.add(jar);
                    LOGGER.info("{} loaded.", file.toString());
                } catch (IOException e) {<!-- -->
                    LOGGER.error(e, "{} jar file can't be resolved", fileName);
                }
            }
        }
    }
    return jars;
}

@RequiredArgsConstructor
private static class Jar {<!-- -->
    private final JarFile jarFile;
    private final File sourceFile;
}

Get all skywalking-plugin.def files

List resources = resolver.getResources()

public List<URL> getResources() {<!-- -->
    List<URL> cfgUrlPaths = new ArrayList<URL>();
    Enumeration<URL> urls;
    try {<!-- -->
        urls = AgentClassLoader.getDefault().getResources("skywalking-plugin.def");

        while (urls.hasMoreElements()) {<!-- -->
            URL pluginUrl = urls.nextElement();
            cfgUrlPaths.add(pluginUrl);
            LOGGER.info("find skywalking plugin define in {}", pluginUrl);
        }

        return cfgUrlPaths;
    } catch (IOException e) {<!-- -->
        LOGGER.error("read resources failure.", e);
    }
    return null;
}

Use the AgentClassLoader class loader to obtain all skywalking-plugin.def plug-ins in the plugins and activations directories.

Read skywalking-plugin.def and convert it into PluginDefine

PluginCfg.INSTANCE.load(pluginUrl.openStream())

void load(InputStream input) throws IOException {<!-- -->
    try {<!-- -->
        BufferedReader reader = new BufferedReader(new InputStreamReader(input));
        String pluginDefine;
        while ((pluginDefine = reader.readLine()) != null) {<!-- -->
            try {<!-- -->
                if (pluginDefine.trim().length() == 0 || pluginDefine.startsWith("#")) {<!-- -->
                    continue;
                }
                //Convert each plug-in into PluginDefine
                PluginDefine plugin = PluginDefine.build(pluginDefine);
                pluginClassList.add(plugin);
            } catch (IllegalPluginDefineException e) {<!-- -->
                LOGGER.error(e, "Failed to format plugin({}) define.", pluginDefine);
            }
        }
        //Exclude plug-ins that do not need to be enabled in the configuration file
        pluginClassList = pluginSelector.select(pluginClassList);
    } finally {<!-- -->
        input.close();
    }
}

Remember the original method pluginFinder = new PluginFinder(new PluginBootstrap().loadPlugins());? The initialization of the AgentClassLoader accumulator and the loading and generation of plug-ins are all executed in ew PluginBootstrap().loadPlugins()

Let’s analyze pluginFinder = new PluginFinder

pluginFinder = new PluginFinder

/**
 * The reason why the generic type of Map is <String, List> is that for the same class, there may be multiple plug-ins that need to add bytecode to this class.
 *
 * key -> target class
 * value -> All plug-ins that can take effect on this target class
 * */
private final Map<String, LinkedList<AbstractClassEnhancePluginDefine>> nameMatchDefine = new HashMap<String, LinkedList<AbstractClassEnhancePluginDefine>>();
private final List<AbstractClassEnhancePluginDefine> signatureMatchDefine = new ArrayList<AbstractClassEnhancePluginDefine>();
private final List<AbstractClassEnhancePluginDefine> bootstrapClassMatchDefine = new ArrayList<AbstractClassEnhancePluginDefine>();
private static boolean IS_PLUGIN_INIT_COMPLETED = false;

/**
 * Categorize plugins
 * Naming plug-in, indirect matching plug-in, jdk class library plug-in
 * */
public PluginFinder(List<AbstractClassEnhancePluginDefine> plugins) {<!-- -->
    for (AbstractClassEnhancePluginDefine plugin : plugins) {<!-- -->
        ClassMatch match = plugin.enhanceClass();

        if (match == null) {<!-- -->
            continue;
        }

        if (match instanceof NameMatch) {<!-- -->
            NameMatch nameMatch = (NameMatch) match;
            LinkedList<AbstractClassEnhancePluginDefine> pluginDefines = nameMatchDefine.get(nameMatch.getClassName());
            if (pluginDefines == null) {<!-- -->
                pluginDefines = new LinkedList<AbstractClassEnhancePluginDefine>();
                nameMatchDefine.put(nameMatch.getClassName(), pluginDefines);
            }
            pluginDefines.add(plugin);
        } else {<!-- -->
            signatureMatchDefine.add(plugin);
        }

        if (plugin.isBootstrapInstrumentation()) {<!-- -->
            bootstrapClassMatchDefine.add(plugin);
        }
    }
}

Summary

• PluginBootstrapinstantiates all plugins
  • PluginResourcesResolver loads the file of skywalking-plugin.def
  • PluginCfg encapsulates PluginDefine
  • DynamicPluginLoader loads plug-ins based on xml configuration
• PluginFinderClassification plug-in
  • NameMatch, naming plug-in
  • IndirectMatch, indirect matching plug-in
  • JDK class library plug-in

The above completes the analysis of the process of reading configuration files, customizing class loaders, and loading plug-ins.