http://hive.apache.org/downloads.html
IP | 服务 | 角色 |
192.168.34.30 | NameNode、SecondaryNameNode、ResourceManager、hive | node1 |
192.168.34.31 | DataNode、NodeManager | node2 |
192.168.34.32 | DataNode、NodeManager |
wget http://mirror.bit.edu.cn/apache/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz
mkdir -p /opt/hive
tar -zxvf apache-hive-3.1.2-bin.tar.gz -C /opt/hive
#配置hive环境变量
vi /etc/profile
#添加如下环境变量
JAVA_HOME=/usr/java/jdk1.8.0_181-amd64
HIVE_HOME=/opt/hive/apache-hive-3.1.2-bin
export HADOOP_HOME=/opt/hadoop/hadoop-3.1.4
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin
source /etc/profile
cd /opt/hadoop/hadoop-3.1.4/bin #进入Hadoop主目录
hadoop fs -mkdir -p /user/hive/warehouse #创建目录
hadoop fs -chmod -R 777 /user/hive/warehouse #新建的目录赋予读写权限
hadoop fs -mkdir -p /tmp/hive/ #新建/tmp/hive/目录
hadoop fs -chmod -R 777 /tmp/hive #目录赋予读写权限
cd /opt/hive/apache-hive-3.1.2-bin #切换目录
mkdir tmp #创建临时目录
chmod -R 777 tmp/ #授权临时目录
#用以下命令检查目录是否创建成功
hadoop fs -ls /user/hive
hadoop fs -ls /tmp/hive
cp hive-default.xml.template hive-site.xml
vi hive-site.xml
-修改hive-site.xml中的临时目录
-将hive-site.xml文件中的${system:java.io.tmpdir}替换为hive的临时目录
<property>
<name>hive.downloaded.resources.dir</name>
<!--value>${system:java.io.tmpdir}/${hive.session.id}_resources</value-->
<value>/mydata/hive/apache-hive-3.1.2-bin/tmp/${hive.session.id}_resources</value>
<description>Temporary local directory for added resources in the remote file system.</description>
</property>-将配置文件中${system:user.name}都替换为root
<property>
<name>hive.server2.logging.operation.log.location</name>
<value>/opt/hive/apache-hive-3.1.2-bin/tmp/root/operation_logs</value>
<description>Top level directory where operation logs are stored if logging functionality is enabled</description>
</property>
---以上给出的只是配置文件中部分示例,请把全部替换掉。
vi hive-site.xml
-javax.jdo.option.ConnectionDriverName,将该name对应的value修改为MySQL驱动类路径:
<property
<name>javax.jdo.option.ConnectionDriverName</name
<value>com.mysql.jdbc.Driver</value>
</property>
-javax.jdo.option.ConnectionURL,将该name对应的value修改为MySQL的地址:
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://192.168.34.24:3306/hive?createDatabaseIfNotExist=true</value>
我的数据部署在192.168.34.24
-javax.jdo.option.ConnectionUserName,将对应的value修改为MySQL数据库登录名:
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
-javax.jdo.option.ConnectionPassword,将对应的value修改为MySQL数据库的登录密码:
<name>javax.jdo.option.ConnectionPassword</name>
<value>root</value>
我的数据库密码是root
-将MySQL驱动包上载到Hive的lib目录下
https://repo1.maven.org/maven2/mysql/mysql-connector-java/5.1.42/mysql-connector-java-5.1.42.jar
cp mysql-connector-java-5.1.42.jar /opt/hive/apache-hive-3.1.2-bin/lib #复制到hive lib 仓库里
vi /opt/hive/apache-hive-3.1.2-bin/conf/hive-env.sh
#添加内容自己hadoop和hive的路径
export HADOOP_HOME=/opt/hadoop/hadoop-3.1.4
export HIVE_CONF_DIR=/opt/hive/apache-hive-3.1.2-bin/conf
export HIVE_AUX_JARS_PATH=/opt/hive/apache-hive-3.1.2-bin/lib
# 启动和测试
---对MySQL数据库初始化
/opt/hive/apache-hive-3.1.2-bin/bin
schematool -initSchema -dbType mysql #对数据库进行初始化
-执行成功后,在mysql的hive数据库里已生成metadata数据表
---启动Hive
/opt/hive/apache-hive-3.1.2-bin/bin
./hive #执行hive启动
---测试Hive
-成功启动Hive后,会进入hive的命令行模式,下面进行一系列简单测试
hive> show functions;
OK
...
hive> desc function sum;
OK
...
---执行新建库、表以及导入数据的测试
create table req
(
request_id bigint,
message_type_id int,
request_type int,
request_time string,
request_url string,
request_source int,
request_body string,
response_time string,
send_time string,
attachment_id int,
template_id bigint,
linked_template_id string,
linked_job_id bigint,
application_id int,
created_by string,
creation_date string,
last_updated_by string,
last_updated_date string
) row format delimited fields terminated by '\t\t\t';
---hive启动报错
-hive启动报错:
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgumen
原因:
hadoop和hive的两个guava.jar版本不一致
两个位置分别位于下面两个目录:
- /mydata/hive/apache-hive-3.1.2-bin/lib/
-/mydata/hadoop/hadoop-3.1.4/share/hadoop/common/lib/
解决办法:
删除低版本的那个,将高版本的复制到低版本目录下
-通过hdfs管理界面查看/tmp目录出现如下错误:
Permission denied: user=dr.who, access=READ_EXECUTE, inode="/tmp":hadoop:supergroup:drwx------
hdfs dfs -chmod -R 755 /user