apache hive - What is hive - hive tutorial - hadoop hive - hadoop hive - hiveql




apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

What is HIVE

  • Apache Hive is a component of Hortonworks Data Platform(HDP). apache hive provides a SQL-like interface to data stored in HDP.
  • In a simplified defintion, apache hive is nothing but a system for managing and querying structured data built on top of Hadoop
    • Map-Reduce for execution
    • HDFS for storage
    • Metadata on raw files
  • learn hive - hive tutorial - apache hive -  The SQL Interface to Hadoop -  hive examples

    learn hive - hive tutorial - apache hive - The SQL Interface to Hadoop - hive examples

    Hive: The SQL Interface to Hadoop
  • We used Pig, which is a scripting language with a focus on dataflows.
  • apache hive provides a database query interface to Apache Hadoop.
 learn hive tutorial - hive architecture - hive example

apache hive - learn hive - hive tutorial - hive architecture - hive example

  • Key Building Principles:
    • SQL as a familiar data warehousing tool
    • Extensibility – Types, Functions, Formats, Scripts
    • Scalability and Performance
  • Why to use apache hive?

    • People frequently ask why do Pig and Hive exist when they look to do much of the same thing.
    • Apache hive because of its SQL like query language is often used as the interface to an Apache Hadoop based data warehouse.
    • Apache hive is considered friendlier and more familiar to users who are used to using SQL for querying data.
    • Pig fits in through its data flow strengths where it takes on the tasks of bringing data into Apache Hadoop and working with it to get it into the form for querying.

    Apache hive is not

    • Not a relational database
    • Not a design for OnLine Transaction Processing (OLTP)
    • Not a language for real-time queries and row-level updates
    apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

    When to use Apache hive?

    • It provides us data warehousing facilities on top of an existing Hadoop cluster.
    • Along with that it provides an SQL like interface which makes your work easier, in case you are coming from an SQL background.
    • learn hive - hive tutorial - apache hive -  bigdata architecture -  hive examples

      learn hive - hive tutorial - apache hive - bigdata architecture - hive examples

    • You can create tables in Apache hive and store data there.
    • Along with that you can even map your existing HBase tables to Apache hive and operate on them.

    Features

    • Apache Hive supports analysis of large datasets stored in Hadoop's HDFS and compatible file systems such as Amazon S3 filesystem.
    • learn hive - hive tutorial - apache hive - Apache Hive Components -  hive examples

      learn hive - hive tutorial - apache hive - Apache Hive Components - hive examples

    • It provides an SQL-like query language called HiveQL with schema on read and transparently converts queries to MapReduce, Apache Tez and Spark jobs.
    • All three execution engines can run in Hadoop YARN. To accelerate queries, it provides indexes, including bitmap indexes.
    • Other features of Apache hive include:Indexing to provide acceleration, index type including compaction and bitmap index as of 0.10, more index types are planned.
    • Different storage types such as plain text, RCFile, HBase, ORC, and others.
    • Metadata storage in a relational database management system, significantly reducing the time to perform semantic checks during query execution.
    • Operating on compressed data stored into the Hadoop ecosystem using algorithms including DEFLATE, BWT, snappy, etc.
    • Built-in user-defined functions (UDFs) to manipulate dates, strings, and other data-mining tools. Apache hive supports extending the UDF set to handle use-cases not supported by built-in functions.
    • SQL-like queries (HiveQL), which are implicitly converted into MapReduce or Tez, or Spark jobs.
    • By default, Apache hive stores metadata in an embedded Apache Derby database, and other client/server databases like MySQL can optionally be used.
    • The first four file formats supported in Apache hive were
      • plain text,
      • sequence file,
      • optimized row columnar (ORC) format and
      • RCFile.
    • Apache Parquet can be read through plugin in versions later than 0.10 and natively starting at 0.13.
    • Additional Apache hive plugins support querying of the Bitcoin Blockchain.
    apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

    Apache hive Data Models

    learn hive - hive tutorial - apache hive - Apache Hive data models -  hive examples
    < p align="center">learn hive - hive tutorial - apache hive - Apache Hive data models - hive examples

    Table Data Models

  • Analogous to tables in relational DBs
  • Each table has corresponding directory in HDFS
  • Example
    • Page views table name: pvs
    • HDFS directory

    • ------------------/wh/pvs
  • Partitions Data Models

  • Analogous to dense indexes on partition columns
  • Nested sub-directories in HDFS for each combination of partition column values
  • Example
    • Partition columns: ds, ctry
    • HDFS subdirectory for ds = 20090801, ctry = US

    • ------------------/wh/pvs/ds=20090801/ctry=US
    • HDFS subdirectory for ds = 20090801, ctry = CA

    • ------------------/wh/pvs/ds=20090801/ctry=CA
  • Buckets Data Models

  • Split data based on hash of a column - mainly for parallelism
  • One HDFS file per bucket within partition sub-directory
  • Example
    • Bucket column: user into 32 buckets
    • HDFS file for user hash 0
      ------------------/wh/pvs/ds=20090801/ctry=US/part-00000
    • HDFS file for user hash bucket 20
      ------------------/wh/pvs/ds=20090801/ctry=US/part-00020
  • External Tables Data Models

  • Point to existing data directories in HDFS
  • Can create tables and partitions – partition columns just become annotations to external directories
  • Example: create external table with partitions
  • CREATE EXTERNAL TABLE pvs(userid int, pageid int, ds string, ctry string) PARTITIONED ON (ds string, ctry string) STORED AS textfile LOCATION ‘/path/to/existing/table’
  • Example: add a partition to external table
  • ALTER TABLE pvs ADD PARTITION (ds=‘20090801’, ctry=‘US’) LOCATION ‘/path/to/existing/partition’

    Wikitechy Apache Hive tutorials provides you the base of all the following topics . Enjoy learning on big data , hadoop , data analytics , big data analytics , mapreduce , hadoop tutorial , what is hadoop , big data hadoop , apache hadoop , apache hive , hadoop wiki , hadoop jobs , hadoop training , hive tutorial , hadoop big data , hadoop architecture , hadoop certification , hadoop ecosystem , hadoop fs , apache pig , hadoop cluster , cloudera hadoop , hadoop download , hadoop mapreduce , hadoop workflow , hive data types , hadoop hive , pig hadoop , hadoop administration , hadoop installation , hive hadoop , learn hadoop , hadoop for dummies , hadoop commands , hive definition , hiveql , learnhive , hive sql , hive database , hive date functions , hive query , apache hive tutorial , hive apache , hive wiki , what is a hive , hive big data , programming hive , what is hive in hadoop , hive documentation , how does hive work

    Related Searches to what is hive used for

    Adblocker detected! Please consider reading this notice.

    We've detected that you are using AdBlock Plus or some other adblocking software which is preventing the page from fully loading.

    We don't have any banner, Flash, animation, obnoxious sound, or popup ad. We do not implement these annoying types of ads!

    We need money to operate the site, and almost all of it comes from our online advertising.

    Please add wikitechy.com to your ad blocking whitelist or disable your adblocking software.

    ×