1,編譯tez 見http://www.lxweimin.com/p/b2569796dd27

2,將 編譯后的tez-0.9.2.tar.gz 上傳到hdfs上. tez-site.xml中會使用到.見tez.lib.uris屬性.
3,在 $HADOOP_HOME/etc/hadoop 下新建 tez-site.xml.內(nèi)容如下

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>

 <property>
    <name>tez.lib.uris</name>
    <value>${fs.defaultFS}/apps/tez/tez-0.9.2.tar.gz</value>
  </property>
<!-- 使用hadoop自身的lib包,設置為true的話可以使用minimal的tez包,false的話需要使用tez-0.9.2.tar.gz的包-->
<property>
     <name>tez.use.cluster.hadoop-libs</name>
     <value>false</value>
</property>
<property>
     <name>tez.am.resource.memory.mb</name>
     <value>1024</value>
</property>
<property>
     <name>tez.am.resource.cpu.vcores</name>
     <value>1</value>
</property>
<property>
     <name>tez.container.max.java.heap.fraction</name>
     <value>0.4</value>
</property>
<property>
     <name>tez.task.resource.memory.mb</name>
     <value>1024</value>
</property>
<property>
     <name>tez.task.resource.cpu.vcores</name>
     <value>1</value>
</property>
</configuration>

4,設置 hive. 修改hive-site.xml文件

  <property>
    <name>hive.execution.engine</name>
    <value>tez</value>
  </property>
<!-- 權限認證的方式,本地測試可以設置為NONE  詳情見 (http://lxw1234.com/archives/2016/01/600.htm)
-->
  <property>
    <name>hive.server2.authentication</name>
    <value>NONE</value>
  </property>

5,設置客戶端的tez. 將 tez-0.9.2.tar.gz 解壓到本地 /usr/tez下.

//設置軟連接方便日后升級.
$ ln -s tez-0.9.2 default
//配置本地環(huán)境變量
export TEZ_CONF_DIR=$HADOOP_CONF_DIR
export TEZ_HOME=/usr/tez/default
export TEZ_JARS=$TEZ_HOME/*:$TEZ_HOME/lib/*
export HADOOP_CLASSPATH=$TEZ_CONF_DIR:$TEZ_JARS:$HADOOP_CLASSPATH

6,重啟服務
會有一下一些錯誤.
錯誤1 :cause: org.apache.hadoop.service.ServiceStateException: java.lang.NoClassDefFoundError: com/google/common/net/UrlEscapers

2020-06-20T08:57:06,834 INFO [main] org.apache.hadoop.service.AbstractService - Service org.apache.tez.dag.app.DAGAppMaster failed in state STARTED; cause: org.apache.hadoop.service.ServiceStateException: java.lang.NoClassDefFoundError: com/google/common/net/UrlEscapers
org.apache.hadoop.service.ServiceStateException: java.lang.NoClassDefFoundError: com/google/common/net/UrlEscapers
    at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59) ~[hadoop-common-2.7.7.jar:?]
    at org.apache.tez.dag.app.DAGAppMaster.startServices(DAGAppMaster.java:1945) ~[tez-dag-0.9.2.jar:0.9.2]
    at org.apache.tez.dag.app.DAGAppMaster.serviceStart(DAGAppMaster.java:2012) ~[tez-dag-0.9.2.jar:0.9.2]
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) [hadoop-common-2.7.7.jar:?]
    at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2663) [tez-dag-0.9.2.jar:0.9.2]
    at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_91]
    at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_91]
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762) [hadoop-common-2.7.7.jar:?]
    at org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659) [tez-dag-0.9.2.jar:0.9.2]
    at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464) [tez-dag-0.9.2.jar:0.9.2]

解決: 我是將$HIVE_HOME/lib中的guava-14.0.1.jar 升級到guava-19.0.jar. 這個問題解決.但是日志中還是有些錯誤,不影響結果的產(chǎn)出.

image.png

錯誤2:
使用hive cli 的方式可以正常提交sql.并且在yarn上可以看到作業(yè)的運行情況.
但是使用 ** beeline -u jdbc:hive2://localhost:10000 ** .在yarn上看不到作業(yè)運行,并且報錯. 錯誤日志只能在hive ui上查看.