Table of Contents Create table statement type of data Classification of tables Classification The difference between internal tables and external tables Example Basic operations of internal tables Knowledge points Example Basic operations of external tables Example View/modify table Knowledge points Example Default separator Example quick mapping table Knowledge Example Data import and export File data […]
Tag: hive
Operation of partition table and bucket table in Hive
Table of Contents Partition Table First level partition table Knowledge points Example multi-level partitioning Knowledge points Example Partition table operations Example hadoop_hive documentation Bucket table Important parameters Basic bucket table Knowledge points Example Bucket table sorting Knowledge points Example Bucketing principle The difference between partition table and bucket table Partition table Partitioned table features/benefits: It […]
Hive data warehouse environment construction and data warehouse modeling
Summary of the problem: After I completed the data warehouse modeling, one day I opened my dbeaver again and all the data was gone. I found many solutions but could not solve my problem. In a fit of anger, I deleted hive and all tables. Start over 1. Hive data warehouse environment construction Hive installation […]
Hive3 on Spark3 configuration
1. Software environment 1.1 Big data component environment Big Data Component Version Hive 3.1.2 Spark spark-3.0.0-bin-hadoop3.2 1.2 Operating system environment OS Version MacOS Monterey 12.1 Linux – CentOS 7.6 2. Construction of big data components 2.1 Hive environment construction 1) Hive on Spark description Hive engines include: default mr, spark, Tez. Hive on Spark: Hive […]
[Archive] vscode configure latex environment
Originally, I found a tutorial on another computer and configured it again. When I configured it again this time, the author of the tutorial updated it and changed it to a Linux-only script, so I saved it. True fragrance warning, vscode + latex writing in early 2023 – Zhihu (zhihu.com) Environment: win10/win11 + vscode + […]
Java operates hive database and creates tables
<dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>3.3.4</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-auth</artifactId> <version>3.3.4</version> </dependency> <dependency> <groupId>org.apache.hive</groupId> <artifactId>hive-jdbc</artifactId> <version>3.1.3</version> <!– Please check if there is a newer version –> <exclusions> <exclusion> <groupId>org.eclipse.jetty</groupId> <artifactId>jetty-runner</artifactId> </exclusion> </exclusions> </dependency> The tools for kerberos login can be used for authentication such as hive hdfs. import org.apache.hadoop.fs.Path; import org.apache.hadoop.security.HadoopKerberosName; import org.apache.hadoop.security.UserGroupInformation; import org.slf4j.Logger; import org.slf4j.LoggerFactory; […]
Is it possible to directly rm the dbf data file and restart the database in Oracle archive mode?
Author | JiekeXu Source | Public account JiekeXu DBA road (ID: JiekeXu_IT) If you need to reprint, please contact us for authorization | (Personal WeChat ID: JiekeXu_DBA) Hello everyone, I am JiekeXu. I am very happy to meet you again. Today I will let you know if it is possible to directly rm the dbf […]
Hive configuration file Hive-site.xml parameter description purpose
Parameter description and usage in the Hive configuration file hive-site.xml Article directory Parameter description and usage in Hive configuration file hive-site.xml Parameter Description Parameter example specific purpose: Parameter description Parameter name Default value Usage hive.metastore.uris – The URI of Hive metadata storage. hive.metastore.client.socket.timeout 600 Hive metadata client socket timeout. hive.metastore.warehouse.dir /user/hive/warehouse Hive data warehouse directory. […]
[1236]Causes and solutions for hive data skew
Article directory Performance of hive data skew Reasons for hive data skew Hive data skew solution Hive tilt group by aggregation tilt Map and Reduce optimization of Hive tilt When Hive tilt’s HQL contains count(distinct) Hive tilt join optimization in HQL Summary of the above description The underlying processing logic of ODPS MR can be […]
Hadoop and Hive installation
1. Tools Linux system: Centos, version 7.0 and above JDK: jdk1.8 Hadoop: 3.1.3 Hive: 3.1.2 Virtual machine: VMware mysql:5.7.11 Tool download address: https://pan.baidu.com/s/10J_1w1DW9GQC7NOYw5fwvg?pwd=0kdr Extraction code: 0kdr Tip: The following is the text of this article. The following cases are for reference. 2. JDK installation Download the jdk-8u181-linux-x64.tar.gz package and upload this package to the /opt […]