apache hive - Hadoop Hive - Hive Hadoop- hive tutorial - hadoop hive - hadoop hive - hiveql

The shell is the primary way to interact with Hive by issuing commands in HiveQL which is a dialect of SQL. To list TABLES (in order to see if HIVE works):

          hive> SHOW TABLES; OK Time taken: 10.425 seconds

For fresh install, the command takes a few seconds to run since it is lazily creating the metastore database on your machine.

In order to run Hive shell in non-interactive mode use –f switch and provide name of file.

          % hive -f script.q

For short scripts you can use -e option (the final semicolon is not required):

          % hive -e 'SELECT * FROM dummy'

In both interactive and non-interactive modes, Hive will print information to standard error. You can suppress these messages using the -S option.

Hive uses Hadoop so you must have hadoop in your path or run the following:
          export HADOOP_HOME= <hadoop-install-dir>

learn hive - hive tutorial - apache hive - Apache Hive - hive sql query flow mapreduce hadoop - hive examples

In addition, you must create /tmp and /user/hive/warehouse (aka hive.metastore.warehouse.dir) and set them chmod g+w in HDFS before you can create a table in Hive.

          $ <hadoop-directory>/bin/hadoop fs -mkdir /tmp
          $ <hadoop-directory>/bin/hadoop fs -mkdir /user/hive/warehouse
          $ <hadoop-directory>/bin/hadoop fs -chmod g+w /tmp
          $ <hadoop-directory>/hadoop fs -chmod g+w /user/hive/warehouse

You can also set HIVE_HOME

          $ export HIVE_HOME=<hive-install-dir>
          $HIVE_HOME /hcatalog/hcatalog/sbin/hcat_server.sh$
          $HIVE_HOME/hcatalog/sbin/hcat_server.sh

To run the WebHCat server from the shell in Hive release 0.11.0 and later:
          $HIVE_HOME/hcatalog/sbin/webhcat_server.sh

Hive by default gets its configuration from <install-dir>/conf/hive-default.xml.The location of the Hive configuration directory can be changed by setting environment variable:

They can be changed by (re)defining them in
          <install-dir>/conf/hive-site.xml.
          Log4j configuration is stored in
          <install-dir>/conf/hive-log4j.properties

Hive queries are executed using map-reduce queries, therefore the behavior of such queries can be controlled by the Hadoop configuration variables.

The CLI command 'SET' can be used to set any Hadoop (or Hive) configuration variable

          hive> SET mapred.job.tracker=myhost.mycompany.com:50030;
          hive> SET -v;

The second command shows all the current settings.

Without the -v option only the variables that differ from the base Hadoop configuration are displayed.

Hive compiler generates map-reduce jobs for most queries.

These jobs are then submitted to the Map-Reduce cluster indicated by the variable

          mapred.job.tracker

This points to a map-reduce cluster with multiple nodes, Hadoop also offers an option to run map-reduce jobs locally on the user's workstation.

To enable local mode of execution, the user can enable the following option:

          hive> SET mapred.job.tracker=local

Starting with release 0.7 Hive supports local mode execution.

To enable this, the user can enable the followig option

          hive> SET mapred.job.tracker = local

ERROR LOGS

Hive uses log4j for logging. By default logs are not emitted to the consol by the CLI.

The default logging level is WARN for Hive releases prior to 0.13.0.

Starting with Hive 0.13.0 , the default logging level is INFO.

The logs are stored in the folder

          /tmp/<user.name>/hive.log