apache spark - Wikitechy https://www.wikitechy.com/interview-questions/tag/apache-spark/ Interview Questions Tue, 14 Sep 2021 10:04:46 +0000 en-US hourly 1 https://wordpress.org/?v=6.9.4 https://www.wikitechy.com/interview-questions/wp-content/uploads/2025/10/cropped-wikitechy-icon-32x32.png apache spark - Wikitechy https://www.wikitechy.com/interview-questions/tag/apache-spark/ 32 32 What is Shark ? https://www.wikitechy.com/interview-questions/apache-spark/what-is-shark/ https://www.wikitechy.com/interview-questions/apache-spark/what-is-shark/#respond Mon, 12 Jul 2021 16:16:52 +0000 https://www.wikitechy.com/interview-questions/?p=254

What is Shark ?

  • Shark is a tool, developed for people who are from a database background – to access Scala MLib capabilities through Hive like SQL interface.
  • Shark tool helps data users run Hive on Spark – offering compatibility with Hive metastore, queries and data.
  • Like Hive, Spark queries are written using a SQL-like language called HiveQL, which Spark translates into Spark Directed Acyclic Graphs (DAGs) that are executed on the Hadoop cluster.
  • More complex queries are supported through User Defined Functions (UDFs) that can be written in Java and referenced by a HiveQL query.
What is Shark
]]> https://www.wikitechy.com/interview-questions/apache-spark/what-is-shark/feed/ 0 What is RDD ? https://www.wikitechy.com/interview-questions/apache-spark/what-is-rdd/ https://www.wikitechy.com/interview-questions/apache-spark/what-is-rdd/#respond Mon, 12 Jul 2021 15:56:56 +0000 https://www.wikitechy.com/interview-questions/?p=250

What is RDD ?

  • Resilient Distributed Datasets (RDD) is a fundamental data structure of Spark. It is an immutable distributed collection of objects.
  • Each dataset in RDD is divided into logical partitions, which may be computed on different nodes of the cluster.
  • Formally, an RDD is a read-only, partitioned collection of records.
  • RDDs can be created through deterministic operations on either data on stable storage or other RDDs.
  • RDD is a fault-tolerant collection of elements that can be operated in parallel.
what is RDD
]]> https://www.wikitechy.com/interview-questions/apache-spark/what-is-rdd/feed/ 0