hive interview questions - Wikitechy

What is the difference between ‘select from table’ and ‘select column from table’ in hive ?

Editor — Tue, 13 Jul 2021 22:18:52 +0000

Difference between ‘select * from table’ and ‘select column from table’ in hive

Table in Hive is stored as a directory in the HDFS.
Using select from table the Hive query processor simply goes directory that have one or more files in table schema.
You may do this if you have very small data like less than a Gigabyte.
In real clusters if you hit ‘select * from table’, it may have data in Terabytes and displaying that will run for long time.
Hive achieved sequence of map reduce programs that reads data from table stored on Hadoop Distributed File System.
Any data processing you do in Hive is achieved through sequence of map reduce programs that reads data from table stored on HDFS.
Hive map reduce based on query processing engine.
Tables have wide number of columns that representing different values.To perform select column the map reduce program will scan all rows and extract a column.

What is the difference between hive view and a hive external table When should it be used ?

Editor — Tue, 13 Jul 2021 22:18:46 +0000

Difference between hive view and a hive external table When should it be used

A view is a query which is defined as a table.

For example:

CREATE VIEW x AS SELECT * FROM y;

When the user queries table x the query planner combines the queries and executes a single one.

This is something like

SELECT * FROM x;

is translated to

SELECT * FROM y;
Then:

DROP TABLE abc;
If the table is defined as external:

CREATE EXTERNAL table abc (….);

An external table is something totally different. An external TABLE is a table that when DROPPED will NOT remove the physical data.
This is a alternative that affects how data is loaded, controlled, and managed.

Use EXTERNAL tables:

The data is also used outside of Hive.
For example, the data files are browse and processed by an existing program that doesn’t lock the files.
Data needs to stay within the underlying location even after a DROP TABLE.
Pointing multiple patterns at a single data it sets repeats via possible patterns.User can use custom location like ASV.
Hive does not contain own data and control settings, dirs, etc.In Hive existing table (i.e) not modify.

Use INTERNAL tables:

Temporary data needs Hive to manage the table and data.

What is the difference between Hive and HBase ?

Editor — Tue, 13 Jul 2021 22:18:28 +0000

Difference between Hive and HBase

Hive	HBase
Hive is query engine	HBase is a data storage particularly for unstructured data.
Apache Hive is mainly used for batch processing i.e. OLAP	HBase is extensively used for transactional processing wherein the response time of the query is not highly interactive i.e. OLTP.
Operations in Hive are used to transformed into mapreduce jobs.	Operations in HBase are run in real-time on the database
For big data applications that require complex and fine grained processing, Hadoop MapReduce is the best choice.	HBase should be used when Data model schema is sparse.
It used for data warehousing requirements the programmers do not write complex mapreduce code.	HBase is an ideal big data solution if the application requires random read or random write operations or both.
Hive does not currently support update statements.	HBase queries are written in a custom language that needs to be learned.
Hive does not provide interactive querying it only runs batch processes on Hadoop.	Apache HBase is a NoSQL key/value store which runs on top of HDFS.
Hive has some limitations of high latency	HBase does not have analytical capabilities
Hive is to analytical queries.	HBase is to real-time querying
Hive used for analytical querying of data collected over a period of time.Hive should not be used for real-time querying.	HBase is perfect for real-time example Facebook use for messaging and real-time analytics. They may even be using it to count Facebook likes.

What is the difference between Apache Hive and Apache Spark ?

Editor — Tue, 13 Jul 2021 22:18:11 +0000

Difference between Apache Hive and Apache Spark

Apache-Hive	Apache-Spark
Apache hive is a distributed query engine built on top of the hadoop eco-system. Hive is introduced by facebook and later on they open sourced it.	Apache spark may be a cluster computing framework that runs on Hadoop and handles differing kinds of data.
The language used by hive to write queries is HQL i.e., Hive+SQL.	Spark has rich resources for handling the data and most importantly, it is 10-20x faster than Hadoop’s MapReduce.
These hive queries can internally get converted into MapReduce job and this is fully handled by the Hadoop framework reducing the work load on the developers.	It attains this speed of computation by its in-memory primitives. The data is cached and is present in the memory (RAM) and performs all the computations in-memory.
Tools to alter easy access to data via SQL, so enabling data warehousing tasks such as extract/transform/load (ETL), reporting, and data analysis.	It is fast and grouping the computing system. It provides high-level in Java, Scala, Python,R, and an optimized engine that supports execution graphs.
Access files to stored either directly in Apache HDFS or such as Apache HBase	Spark structured query language is a Spark module for structured data processing, in which in-memory processing is its core.

What is the location where “Hive” stores table data ?

Editor — Tue, 13 Jul 2021 21:51:17 +0000

“Hive” stores table data

The default location of the hive table is: /user/hive/warehouse/databasename
Go to in browser, it will take to a page with a Browse the filesystem link.
In the $HIVE_HOME/conf directory there is the hive-default.xml and/or hive-site.xml which has the hive.metastore.warehouse.dir property.
That value is where the user want to navigate to after clicking the Browse the filesystem link.
Once the user navigate to that location, they will see the names of their tables.
Clicking on a table name (which is simply a folder) can then expose the partitions of the table.
If you wish access to data, you can use a Hive query and output the result to a file.These will have the exact same structure (divider between columns, ect) because the files on the HDFS.

What is the best studio software/tool to run HIVE SQL/HQL queries by a data analyst ?

Editor — Tue, 13 Jul 2021 21:51:04 +0000

Best studio software/tool to run HIVE SQL/HQL queries by a data analyst

Hue has a Web SQL Editor with autocomplete, display of table/data samples, light graphing, query download

1. RazorSQL

SQL Query Tool and Database Client

RazorSQL is an SQL query tool, database editor, database browser, and database administration tool for Windows, macOS / Mac OS X, Linux, and Solaris.

2.SQL Editor

Edit SQL scripts.
Run SQL queries.
Auto column and auto table lookup.
Multi-tabular query display.
Supports over 20 programming languages including SQL, PL/SQL, T-SQL, PHP, HTML, XML, Java, and more.

3. Database editor

Edit database tables in a spreadsheet format with support for finding and replacing data, automatic generation of SQL, and support for editing multi-line data.

4. Sql query builder

Easily build select, insert, delete, and update SQL queries and statements. Build multi-table joins.

5. Database GUI Tools

Includes tools for creating tables, views, indexes, procedures, functions, triggers, databases, users and tools for altering and dropping tables and views.
Also includes tools for visually editing procedures, functions, and triggers.

What is DDL statement in hive ?

Editor — Tue, 13 Jul 2021 21:19:35 +0000

DDL statement in hive

DDL (Data Definition Language):

The outline of the database schema typically change by SQL statements they are creating, deleting, or modifying schema objects such as databases, tables, and views.
Most statements starts with the keywords CREATE, DROP, or ALTER.

Some of Data Definition Language statements are:

CREATE – To create an object.
ALTER – To alter the structure of the existing object·
DROP – It used to delete objects.
TRUNCATE – Remove all records from a table, including spaces.
MSCK REPAIR TABLE – Refresh metadata information.

The DML statement define as INSERT statement, it involves metadata that must be have nodes, and is also damage the SYNC_DDL query option.

What is hive ?

Editor — Tue, 13 Jul 2021 21:09:15 +0000

Hive

Hive is a component of Hortonworks Data Platform(HDP).
Hive provides a SQL interface to data stored in HDP.
Hive has 3 main functions:

- Data Summarization
- Query
- Analysis.

It supports queries expressed language called HiveQL, which automatically translates SQL like queries into MapReduce jobs executed on Hadoop.
It also enables data serialization and increases flexibility in schema architecture including a system catalog called Hive Metastore.

Architecture of hive :

Features of hive:

Different storage such as plain text, RCFile, ORC, HBase, and others.
Pre-defined functions (UDFs) to manipulate dates, strings, and other data-mining tools.
Hive supports extending the UDF set to handle use-cases not supported by built-in functions.

Limitations of Hive:

Hive supports overwriting or hold data, but not updates and deletes.
In Hive, sub queries are not supported