Directory
1. Preparation before installation
2. Installation and configuration
2.1 Install ZK and Kafka
2.2 Create a Kafka topic
2.3 Install pinot (local installation)
2.4 Create Pinot table
2.5 Configure and run Pinpoint Collector and Web with URI statistics
2.6 The configuration value of the agent’s URI statistics
1. Preparation before installation
1.1 Official installation instructions:
https://pinpoint-apm.gitbook.io/pinpoint/documents/uri_statistics
The URI statistics menu is a new function in Pinpoint v2.5.0. The Pinpoint proxy aggregates URI templates and sends them to the Pinpoint collector via GRPC. Pinpoint Collector saves data into Pinot via Apache Kafka, and Pinpoint Web accesses Pinot to display data
Architecture diagram:
1.2 The general installation steps of pinpoint2.5.1 can be referred to (the deployment package is replaced with 2.5.1), which is described in detail:
https://blog.csdn.net/jianghaijun552683/article/details/129814933
2. Installation and Configuration
2.1 Install ZK and KAFKA
ZK version: apache-zookeeper-3.7.1
KAFKA version: kafka_2.13-3.4.0
There are many installation steps for ZK and KAFKA on the Internet. For the installation steps of a single instance, please refer to the following (it is recommended to install a cluster in a production environment):
https://blog.csdn.net/Loiterer_Y/article/details/104496841
2.2 Create Kafka topic
Create a topic (in kafka server) with name uri-stat .
bin/kafka-topics.sh --create --topic uri-stat --bootstrap-server localhost:9092
Verify Kafka:
bin/kafka-console-consumer.sh --topic uri-stat --from-beginning --bootstrap-server localhost:9092 bin/kafka-console-producer.sh --topic uri-stat --bootstrap-server localhost:9092
2.3 install pinot (local installation)
Official documentation:
https://docs.pinot.apache.org/basics/getting-started/running-pinot-locally
prerequisites
Install JDK11 or later (JDK16 is not supported).
1) Download pinot
https://downloads.apache.org/pinot/apache-pinot-0.12.1/apache-pinot-0.12.1-bin.tar.gz
2) Unzip the downloaded compressed package
tar -xzvf apache-pinot-0.12.1-bin.tar.gz
3) Manually set up the cluster
When deploying the Pinot service, it is best to start it in order to avoid other problems.
The sequence is as follows:
a) pinot-controller
b) pinot-broker
c) pinot-server
d) pinot-minion
Note: If the machine has multiple JDKs, you can add the following configuration to the start-XXX.sh file to start:
export JAVA_HOME=/opt/pinot/jdk-11.0.0.1 export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
Start command reference:
controllerIP, brokerIP, serverIP, ZKIP need to be replaced with your own server IP, and the port can also be replaced with other
start the controller
nohup ./start-controller.sh -clusterName uriPinotCluster -controllerHost controllerIP -controllerPort 9000 -dataDir /opt/pinot/apache-pinot-0.12.1-bin/data/PinotController -zkAddress ZKIP:2181 >/opt/pinot /apache-pinot-0.12.1-bin/logs/controller.log 2> & amp;1 & amp;
start broker
nohup ./start-broker.sh -clusterName uriPinotCluster -brokerHost brokerIP -brokerPort 8099 -zkAddress ZKIP:2181 >/opt/pinot/apache-pinot-0.12.1-bin/logs/broker.log 2> & amp ;1 &
start server
nohup ./start-server.sh -clusterName uriPinotCluster -serverHost serverIP -serverPort 7000 -serverAdminPort 7500 -dataDir /opt/pinot/apache-pinot-0.12.1-bin/data/PinotServerData -segmentDir /opt/pinot/ apache-pinot-0.12.1-bin/data/PinotServerSegment -zkAddress ZKIP:2181 >/opt/pinot/apache-pinot-0.12.1-bin/logs/server.log 2> & amp;1 & amp;</ pre>
<p>After the startup is complete, you can log in to the controller page: http://controllerip:9000 to view, and "DefaultTenant" will be created by default</p>
<p><img alt="" height="637" src="//i2.wp.com/img-blog.csdnimg.cn/b29c32fab7174ab381316e3f86606b8e.png" width="1200"></p>
<p></p>
<h2 id="2.4 Create Pinot table" style="margin-left:0px;">2.4 Create Pinot table</h2>
<p>The Pinot table json file for Pinpoint URI statistics is provided in the github repository:</p>
<p>https://github.com/pinpoint-apm/pinpoint/tree/master/uristat/uristat-common/src/main/pinot</p>
<p>Refer to the Pinot documentation to create the tables needed in your Pinot cluster.</p>
<p>https://docs.pinot.apache.org/basics/components/table#streaming-table-creation</p>
<p>It is necessary to create a mixed table of uriStat, that is, create two tables of REALTIME and OFFLINE.</p>
<p>Create hybrid table command reference:</p>
<p>1) Create an offline table</p>
<pre>cd /opt/pinot/apache-pinot-0.12.1-bin
bin/pinot-admin.sh AddTable \
-schemaFile /opt/pinot/creat_table/pinot-uriStat-schema.json \
-tableConfigFile /opt/pinot/creat_table/pinot-uriStat-offline-table.json\
-exec
2) Create a realtime table
cd /opt/pinot/creat_table
curl -i -X POST -H 'Content-Type: application/json' \
-d @pinot-uriStat-realtime-table.json localhost:9000/tables
2.5 Configuring and Running Pinpoint Collector and Web with URI Statistics
Note: The JAR packages of Pinpoint web and collector with URI indicators need to be compiled by themselves
Compilation steps:
1) Download the 2.5.1 version of the package to the local
? git clone --branch v2.5.1 https://github.com/pinpoint-apm/pinpoint.git
2) Modify the order of building project modules (there are too many modules):
Modify the file: pinpoint/pom.xml, place the pinot and metric-module modules at the front for construction, and stop after the required construction is completed, otherwise it will take a long time to complete all constructions.
<modules>
<module>pinot</module>
<module>metric-module</module>
<module>annotations</module>
<module>agent</module>
<module>agent-sdk</module>
?…
3) Execute project build
cd pinpoint ./mvnw install -DskipTests=true
If the JDK version is abnormal when building the project, you can execute:
export JAVA_11_HOME=/home/pinpoint-2.5.1/jdk-11.0.0.1 export JAVA_8_HOME=/hlapp/jdk1.8.0_311
4) Collector directory equipped with system indicators
pinpoint/metric-module/collector-starter/target/deploy/pinpoint-collector-starter-boot-2.5.1.jar
5) Web directory with system indicators
pinpoint/metric-module/web-starter/target/deploy/pinpoint-web-starter-boot-2.5.1.jar
Close the original running collector and web program, and use these two newly built jar packages to start
6) Collector and web configuration
There are many related configuration instructions in the official instructions, so I won’t explain them one by one here. The reference configuration items are as follows:
web.properties
spring.profiles.active=release
#hbase zk address
pinpoint.zookeeper.address=localhost
cluster.zookeeper.sessiontimeout=10000#Configure connection pinot, need to replace IP, user authentication is not enabled by default
pinpoint.pinot.jdbc.url=jdbc:pinot://pinot-IP:9000
pinpoint.pinot.jdbc.username=userId
pinpoint.pinot.jdbc.password=password#Configure the web page display, if you want to join systemmetric, you can see the official document
config.show.systemMetric=false
config.show.urlStat=true
#Configure tenantid
pinpoint.pinot.tenantId=DefaultTenant
collector.properties
spring.profiles.active=release
pinpoint.zookeeper.address=localhost#Configure connection pinot, need to replace IP, user authentication is not enabled by default
pinpoint.pinot.jdbc.url=jdbc:pinot://pinot-IP:9000 (need to replace IP)
pinpoint.pinot.jdbc.username=userId
pinpoint.pinot.jdbc.password=passwordpinpoint.metric.kafka.bootstrap.servers=KAFKA-IP:9092 (need to replace IP)
pinpoint.collector.type=ALL
collector.stat.uri=true
kafka.uri.topic=uri-stat
pinpoint.pinot.tenantId=DefaultTenant
These two configuration files need to be referenced separately when starting the web service and collector service
2.6 Configuration value of URI statistics of agent
The following are the default proxy configuration values for URI statistics.
#################################################### #########
# URI Stat
#################################################### #########
profiler.uri.stat.enable=true
profiler.uri.stat.spring.webmvc.enable=true
profiler.uri.stat.spring.webmvc.useuserinput=false
profiler.uri.stat.spring.webflux.enable=true
profiler.uri.stat.spring.webflux.useuserinput=false
profiler.uri.stat.vertx.enable=true
profiler.uri.stat.vertx.useuserinput=false
If you need to change these configurations, you need to rebuild the agent project, or customize the configuration file when starting the agent. If there is no special requirement, you can keep the default settings.
After the installation is complete, you can view the url Statistics statistics on the pinpoint web page
