Iris Software interview questions and answers - Wikitechy

What is the advantages of VB.NET ?

Editor — Thu, 22 Jul 2021 07:43:21 +0000

VB.NET offers code execution that runs under the Common Language Runtime (CLR), resulting in robust, stable and secure applications. Both features of the .NET framework are eagerly obtainable in VB.NET.
VB.NET is object oriented. This is a major addition that VB6 and other earlier releases didn’t have.
The .NET framework comes with ADO.NET, which follows the disconnected paradigm, i.e. once the required records are fetched the connection no longer exists.
It also retrieves the records that are expected to be accessed in the immediate future.
This enhances Scalability of the application to a great extent.
VB.NET uses XML to transfer data between the various layers in the DNA Architecture i.e. data are passed as simple text strings.
Error handling has changed in VB.NET. A new Try-Catch-Finally block has been introduced to handle errors and exceptions as a unit, allowing appropriate action to be taken at the place the error occurred thus discouraging the use of ON ERROR GOTO statement. This again credits to the maintainability of the code.

What is the difference between vb and vb.net ?

Editor — Thu, 22 Jul 2021 07:41:06 +0000

VB	VB.NET
VB stands for Visual Basic. It is a high level programming language that is used for the fast development of Windows based programs.	VB.NET stands for Visual Basic Network Enabled Technology. It is one of the languages for the .NET framework by Microsoft. This language has been created specifically for VB developers. It belongs to the category of object oriented computer programming languages that is implemented on the .NET framework.
Interpreter based Language	Compiled language
VB Runtime	Common Language Specification
This type of language is not safe.	This language is safe compared to visual basic.
Multithreaded applications Cannot be developed.	Multithreaded applications Can be developed.
Data is handled using DAO protocol, then RDO, followed by ADO (ActiveX Data Objects)	Data is handled using ADO.net
An object based language.	A true object oriented language.
Most of the parameters passed by reference.	Parameters passing by Default value.
Objects support default property	Objects does not support default property
Replaced by elements in Collection QBColor, RGB DateAdd, DateDiff, DatePart, DateSerial, DateValue, TimeSerial, TimeValue	System.Collection System.Drawing.Color Replaced by elements in System.DateTime
End Type structure can be used in declare function.	Declared using the Structure…End Structure construct.

What is the difference between sqoop and pig ?

Editor — Thu, 22 Jul 2021 06:59:29 +0000

Pig:

Pig is a platform for managing large data.
It uses high level programming to analyse the data.
The programming is done using the language pig latin.
It is simple and highly specific.
The pig Metastore stores all info about the tables. And we can execute spark sql queries because spark can interact with pig Metastore

Sqoop:

Sqoop is a simple tool used for transferring large amount of data.
It is used for transferring data from hadoop to relational database.
sqoop can import data from Oracle,MYSQL,SQl.
Sqoop metastore is a tool for using hosts in a shared metadata repository. Multiple users and remote users execute saved jobs in metastore.

What is staging table in sqoop ?

Editor — Thu, 22 Jul 2021 06:53:48 +0000

Staging table in sqoop:

Staging table is used to stage the exported data.
Sqoop breaks down method into multiple transactions. A failing job may create partial knowledge.
This could result in job failing because it could cause duplication of data or even collision while insertion.
Staging tables are simple database tables that contain business details.
Staging table acts as an auxiliary table that’s used to transferred data.
Sqoop breaks down export method into multiple transactions, it’s possible that a failing export job may lead to partial data being committed to the database

What is update command in sqoop ?

Editor — Thu, 22 Jul 2021 06:52:07 +0000

Update command in sqoop:

There are two approaches to update in sqoop.
- The records inserted are to be added to the target table.
- The sqoop provides an update mode by providing “update-key ” command.
This command is used to generate the SQl update statement.
If the value in column 1 matches the data is updated else the data is inserted into the table.
This technique is also refered to as upsert where the default action is update but insertion takes place when matching data is not available.

 
$ sqoop export (Generic Arguments)
  --table target-relational-table 
  --update-key column1
  --export-dir /user/my-hdfs-file
  ...
Generates => UPDATE target-relational-table SET
                    column2=1000,column3=2000
		          WHERE column1=100;

What is the use of sqoop ?

Editor — Thu, 22 Jul 2021 06:49:42 +0000

Sqoop uses:

Sqoop is a simple tool that is used for moving data.
It is generally used move data from non hadoop data stores.
These non hadoop data stores are the relational databases and data warehouses .
Sqoop allows the users to specify the target (i.e) the place where the data is to be moved inside of hadoop from relational databases like Oracle,Teredata.
Data ingestion is important in any big data project because the volume of data is generally in petabytes or exabytes.
Hadoop Sqoop and Hadoop Flume are the two tools in Hadoop which is used to gather data from different sources and load them into HDFS.

What is sqoop metastore ?

Editor — Thu, 22 Jul 2021 06:47:16 +0000

Sqoop Metastore:

Sqoop metastore is an internally implemented database that can hold the definition of a given job.
It includes details like which database,which table,which approach,what query to extract from,what target to write to and so on.
Consider an array of SQL databases that have the same structure, but are partitioned by for performance reasons.
Thus we have the same sqoop job running against multiple database instances at the same time, all unified at a single table in Hadoop.
In order to get parallelization, we generate a job that manages the Sqoop sub-workflows. This results in 500 distinct sqoop jobs.
Metastore is used to handle these complexities.

What is sqoop ?

Editor — Thu, 22 Jul 2021 06:22:24 +0000

Sqoop:

Apache Sqoop is efficiently transferring data between Relational databases and Apache Hadoop.
It is used to import data from relational databases such as MySQL, Oracle to Hadoop HDFS, and export from Hadoop file system to relational databases.
It improves the efficiency of data analysis by combining structured data and unstructured data.
Sqoop stands for SQL for Hadoop.SQL + HADOOP =SQOOP

What are the differences between Impala and Apache Drill ?

Editor — Mon, 19 Jul 2021 06:41:05 +0000

Difference between Impala and Apache Drill

Index	Cloudera impala	Apache Drill
Description	Analytic DBMS for Hadoop	Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud Storage
Primary database modele	relational DBMS	document store relational DBMS
Developer	Cloudera	Apache Software Foundation
License	open source	open source
Server operating systems	Linux	Linux,OSX, Windows
Data scheme	yes	schema-free
Secondary indexes	yes	no
SQL	SQL-like DML and DDL statements	SQL SELECT statement is SQL:2003 compliant
Supported programming languages	All languages supporting JDBC/ODBC	C++

What is the difference between Impala and Apache Hive ?

Editor — Mon, 19 Jul 2021 06:40:04 +0000

Difference between Impala and Apache Hive

Impala	Apache hive
Impala does runtime code generations for “big loops ” using llvm.	Apache hive generates query expressions at compile time.
Hadoop 2.6.0	Hadoop 2.7.3
Runtime Filtering Optimization Enabled	All queries run through LLAP
Parquet format with snappy compression	ORCFile format with zlib compression
Impala avoids startup overhead as daemon processes are started at boot time itself, always being ready to processes a query.	Every hive query has this problem of “cold start”.
Impala is meant for interactive computing.	Apache Hive might not be ideal for interactive computing .
Impala is more like MPP database.	Hive is batch based Hadoop MapReduce .
Impala does not support complex types.	Hive supports complex types .
Impala does not support fault tolerance	Apache Hive is fault tolerant

What is the difference between Apache Hadoop and Cloudera in big data ?

Editor — Mon, 19 Jul 2021 06:38:24 +0000

Difference between Apache Hadoop and Cloudera in big data

Apache Hadoop is the Hadoop distribution from Apache group.
Cloudera Hadoop has its own supply of Hadoop which is designed on top of Apache Hadoop. so it does not have latest release of Hadoop.
Cloudera Hadoop contains extra tools. while Hadoop distributions like Cloudera Search, Impala, Cloudera Navigator and Cloudera Manager these are not involved in Cloudera Hadoop.
These additional tools turns out Cloudera Hadoop to be slightly open source and proprietary.

Apache Hadoop is very important to use to cope up with any data, no matter how large or what type or complex it is. It works like a fluid way or can say it is very faster, cheaper and reliable analytic tool than others.
Even, it can easily work on unstructured, and schemaless data and people can easily ground their data into a format without reformating or putting any other efforts.
Apart from this, via Apache Hadoop various programming model has been simplified to allow you to quickly write and test the software in distributed systems.
This is an open-source Apache Hadoop distribution usually, focus on enterprise-class deployments.
Cloudera Inc is a very popular American software company, known for providing Apache Hadoop-based software, services, and full support for the same.
With Cloudera, it is said that Hadoop is just a starting to create your data management strategy, and you can easily use the same to add the security and other various functions to create an enterprise-grade foundation for your data.

What is the architectural design and features of Cloudera Impala ?

Editor — Mon, 19 Jul 2021 06:35:32 +0000

Components of the Impala Server

The impala server consist of distributed and massively parallel processing info engine.
It run on particular hosts among your CDH cluster with a various daemon processes.

The Impala Daemon

The core impala element may be a daemon method that runs on every node of the cluster, and its physically described by the impalad method.
Then read and write a informative files; which accepts questions transmitted from the impala-shell command, Hue, JDBC, or ODBC; parallelizes the questions can seperate work to alternative nodes within the impala cluster; and transmits intermediate question results back to the central organizer node.
The impala daemon running on any node, and that node is the organizer node for that question when You can submit a question.
The other nodes transmit partial results back to the organizer, that constructs the ultimate result set for a question.
You may continuously connect with a similar impala daemon for convenience.When running experiments with possibility through the impala-shell command.
For clusters running production workloads, you would possibly load-balance between the nodes by submitting every question to a distinct impala daemon in round-robin vogue, exploitation the JDBC or ODBC interfaces.
Its verify that nodes and may settle for new work. Through the impala daemons are in constant communication with the statestore.

The Impala Statestore

The impala element called the statestore checks on the health of impala daemons on all the nodes during a cluster, and unendingly relays its findings to every of these daemons.
It is physically described by a daemon method named statestored; you simply would like such a method on one node within the cluster.
If an impala node goes offline because of hardware failure, network error, software package issue, or alternative reason, the statestore informs all the opposite nodes so future queries will avoid creating requests to the unreached node.

New options in Impala:

Performance and quantifiability enhancements.
Integration with Apache koodoo.
Security.
The REFRESH statement currently updates data regarding HDFS block locations.
[IMPALA-1654] many varieties of DDL operations will currently work on a spread of partitions.

What is Cloudera’s technology stack ?

Editor — Mon, 19 Jul 2021 06:33:22 +0000

Cloudera’s technology stack

Here are some products that make up Cloudera’s tech stack:

- Adobe Experience Manager,
- Amazon EC2,
- Apache (Ant, HBase, Kafka, Maven, Mesos, Solr, Spark, Web Server),
- Atlassian JIRA,
- Bugsnag,
- C++,
- Cask Data Applicaiton Platform,
- Dyn DNS,
- Fastly,
- Git,
- GitHub,
- Google Fonts,
- Impala,
- Java,
- Jenkins,
- Mirantis,
- Piston,
- Python,
- RBCommons,
- Red Hat Enterprise Linux,
- Scala, and
- VMware vSphere.

Additionally, here’s a list of other software products that Cloudera is using internally,

Marketing: AdRoll, AppNexus, BrightFunnel, Demandbase
Sales and Support: Infer, MindTickle, Outreach, Salesforce Sales Cloud
Analytics: Adobe Analytics, Amplitude, Chartbeat, Tableau Software
HR: AngelList Jobs, Codility, RolePoint, The Muse
Finance: Concur, Ombud, SteelBrick
Productivity: Box, Google Apps for Work, LibreOffice, Microsoft Office 365

What is cloudera impala ?

Editor — Mon, 19 Jul 2021 06:30:25 +0000

Cloudera’s Impala

Impala was the first to bring SQL querying to the public in April 2013. Impala comes with a bunch of interesting features:

Impala can query many file format such as Parquet, Avro, Text, RCFile, SequenceFile
Impala supports data stored in HDFS, Apache HBase and Amazon S3
Impala supports multiple compression codecs:
- Snappy (Recommended for its effective balance between compression ratio and decompression speed),
- Gzip (Recommended when achieving the highest level of compression),
- Deflate (not supported for text files), Bzip2, LZO (for text files only);
Impala provides security through authorization based on Sentry (OS user ID)
- Defining which users are allowed to access which resources,
- What operations are they allowed to perform authentication based on Kerberos + ability to specify Active Directory username/password,
- How does Impala verify the identity of the users to confirm that they are allowed exercise their privileges assigned to that user auditing,
- What operations were attempted,
- Did they succeed or not, allowing to track down suspicious activity; audit data are collected by Cloudera Manager;
Impala supports SSL network encryption between Impala and client programs, and between the Impala-related daemons running on different nodes in the cluster;
Impala allows to use UDFs and UDAFs;
Impala orders the joins automatically to be the most efficient;
Impala allows admission control – prioritization and queueing of queries within impala;
Impala allows multi-user concurrent queries;
Impala caches frequently accessed data in memory;
Impala computes statistics (with COMPUTE STATS);
Impala provides window functions (aggregation OVER PARTITION, RANK, LEAD, LAG, NTILE, and so on) – to provide more advanced SQL analytic capabilities (since version 2.0);
Impala allows external joins and aggregation using disk (since version 2.0) – enables operations to spill to disk if their internal state exceeds the aggregate memory size;
Impala allows subqueries inside WHERE clauses;
Impala allows incremental statistics – only run statistics on the new or changed data for even faster statistics computations;
Impala enables queries on complex nested structures including maps, structs and arrays;
Impala enables merging (MERGE) in updates into existing tables;
Impala enables some OLAP functions (ROLLUP, CUBE, GROUPING SET);
Impala allows use of impala for inserts and updates into HBase.