{"id":135,"date":"2021-07-11T15:37:08","date_gmt":"2021-07-11T15:37:08","guid":{"rendered":"https:\/\/www.wikitechy.com\/interview-questions\/?p=135"},"modified":"2021-09-15T05:39:35","modified_gmt":"2021-09-15T05:39:35","slug":"what-is-the-difference-between-flume-and-kafka","status":"publish","type":"post","link":"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/","title":{"rendered":"What is the difference between flume and Kafka ?"},"content":{"rendered":"
\n
\n

Difference between flume and Kafka<\/h2>\n<\/div>\n<\/div>\n\n\n\n\n\n\n\n\n\n\n\n
Flume<\/th>\nKafka<\/th>\n<\/tr>\n
The Flume is a Distributed reliable system for collecting,
\naggregating and moving large amount of data to
\ncentralized datastore like HDFS or Hbase<\/td>\n		General purpose publish \u2013 subscribe
\nmodel messaging system<\/td>\n<\/tr>\n
Adding more consumers means to change the
\ndesign of flume pipeline and replicating the channel to
\ndeliver messages to new sink which needs
\ndowntime<\/td>\n		Easy to add more consumers without downtime<\/td>\n<\/tr>\n
Supports many built-in sources and sinks
\nout of box<\/td>\n		Sometimes need to write own producer and consumer code though Spark and Storm have now come up with built-in integrations to Kafka<\/td>\n<\/tr>\n
Flume pushes data into sink and hence
\nconsumers do not have to maintain offset<\/td>\n		Subscribers are responsible for pulling data
\nand also maintaining pointer to offset<\/td>\n<\/tr>\n
Events are lost in case the agent goes down<\/td>\nProvides fault tolerance<\/td>\n<\/tr>\n
Does not support partitioning<\/td>\nSupports partitioning<\/td>\n<\/tr>\n
Flume pushes data to the sink because of
\nwhich writes to sink can overwhelm data reads from sink<\/td>\n		Since kafka does not push data, writes from producer to broker and reads from broker to consumers can happen at their own pace<\/td>\n<\/tr>\n
It is tightly integrated with Hadoop<\/td>\nGeneral purpose<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
\n
\"what<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Answer-Kafka can support data streams for multiple applications<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[369],"tags":[418,416,414,413,417,422,415,412,411,420,421,410,364,419],"class_list":["post-135","post","type-post","status-publish","format-standard","hentry","category-apache-flume","tag-apache-flume-disadvantages","tag-apache-flume-limitations","tag-apache-flume-vs-sqoop","tag-apache-flume-vs-storm","tag-disadvantages-of-flume","tag-flume-interview-question","tag-flume-kafka-integration","tag-flume-vs-kafka-vs-spark","tag-flume-vs-spark-streaming","tag-flume-vs-streaming","tag-kafka-vs-flume-vs-scribe","tag-kafka-vs-sqoop","tag-wells-fargo-interview-questions-and-answers","tag-which-type-of-channel-will-provide-high-throughput"],"yoast_head":"\nWhat is the difference between flume and Kafka ?<\/title>\n<meta name=\"description\" content=\"What is the difference between flume and Kafka ? - flume interview questions - Distributed reliable system for collecting aggregating\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"What is the difference between flume and Kafka ?\" \/>\n<meta property=\"og:description\" content=\"What is the difference between flume and Kafka ? - flume interview questions - Distributed reliable system for collecting aggregating\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/\" \/>\n<meta property=\"og:site_name\" content=\"Wikitechy\" \/>\n<meta property=\"article:published_time\" content=\"2021-07-11T15:37:08+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-09-15T05:39:35+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/cdn.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka.png\" \/>\n<meta name=\"author\" content=\"Editor\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Editor\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka\\\/\"},\"author\":{\"name\":\"Editor\",\"@id\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/#\\\/schema\\\/person\\\/4d5a581fb5470d1560324bddc5e8b757\"},\"headline\":\"What is the difference between flume and Kafka ?\",\"datePublished\":\"2021-07-11T15:37:08+00:00\",\"dateModified\":\"2021-09-15T05:39:35+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka\\\/\"},\"wordCount\":201,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/cdn.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka.png\",\"keywords\":[\"apache flume disadvantages\",\"apache flume limitations\",\"apache flume vs sqoop\",\"apache flume vs storm\",\"disadvantages of flume\",\"flume interview question\",\"flume kafka integration\",\"flume vs kafka vs spark\",\"flume vs spark streaming\",\"flume vs streaming\",\"kafka vs flume vs scribe\",\"kafka vs sqoop\",\"Wells Fargo interview questions and answers\",\"which type of channel will provide high throughput\"],\"articleSection\":[\"Apache Flume\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka\\\/\",\"url\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka\\\/\",\"name\":\"What is the difference between flume and Kafka ?\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/cdn.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka.png\",\"datePublished\":\"2021-07-11T15:37:08+00:00\",\"dateModified\":\"2021-09-15T05:39:35+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/#\\\/schema\\\/person\\\/4d5a581fb5470d1560324bddc5e8b757\"},\"description\":\"What is the difference between flume and Kafka ? - flume interview questions - Distributed reliable system for collecting aggregating\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka\\\/#primaryimage\",\"url\":\"https:\\\/\\\/cdn.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka.png\",\"contentUrl\":\"https:\\\/\\\/cdn.wikitechy.com\\\/interview-questions\\\/apache-flume\\\/what-is-the-difference-between-flume-and-kafka.png\"},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/#website\",\"url\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/\",\"name\":\"Wikitechy\",\"description\":\"Interview Questions\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/#\\\/schema\\\/person\\\/4d5a581fb5470d1560324bddc5e8b757\",\"name\":\"Editor\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e9531079fe7e07841b7b156c04d65e5f39d4adfd18b6ffe3edfff8ca5aab85b5?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e9531079fe7e07841b7b156c04d65e5f39d4adfd18b6ffe3edfff8ca5aab85b5?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e9531079fe7e07841b7b156c04d65e5f39d4adfd18b6ffe3edfff8ca5aab85b5?s=96&d=mm&r=g\",\"caption\":\"Editor\"},\"url\":\"https:\\\/\\\/www.wikitechy.com\\\/interview-questions\\\/author\\\/editor\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"What is the difference between flume and Kafka ?","description":"What is the difference between flume and Kafka ? - flume interview questions - Distributed reliable system for collecting aggregating","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/","og_locale":"en_US","og_type":"article","og_title":"What is the difference between flume and Kafka ?","og_description":"What is the difference between flume and Kafka ? - flume interview questions - Distributed reliable system for collecting aggregating","og_url":"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/","og_site_name":"Wikitechy","article_published_time":"2021-07-11T15:37:08+00:00","article_modified_time":"2021-09-15T05:39:35+00:00","og_image":[{"url":"https:\/\/cdn.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka.png","type":"","width":"","height":""}],"author":"Editor","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Editor","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/#article","isPartOf":{"@id":"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/"},"author":{"name":"Editor","@id":"https:\/\/www.wikitechy.com\/interview-questions\/#\/schema\/person\/4d5a581fb5470d1560324bddc5e8b757"},"headline":"What is the difference between flume and Kafka ?","datePublished":"2021-07-11T15:37:08+00:00","dateModified":"2021-09-15T05:39:35+00:00","mainEntityOfPage":{"@id":"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/"},"wordCount":201,"commentCount":0,"image":{"@id":"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/#primaryimage"},"thumbnailUrl":"https:\/\/cdn.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka.png","keywords":["apache flume disadvantages","apache flume limitations","apache flume vs sqoop","apache flume vs storm","disadvantages of flume","flume interview question","flume kafka integration","flume vs kafka vs spark","flume vs spark streaming","flume vs streaming","kafka vs flume vs scribe","kafka vs sqoop","Wells Fargo interview questions and answers","which type of channel will provide high throughput"],"articleSection":["Apache Flume"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/","url":"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/","name":"What is the difference between flume and Kafka ?","isPartOf":{"@id":"https:\/\/www.wikitechy.com\/interview-questions\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/#primaryimage"},"image":{"@id":"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/#primaryimage"},"thumbnailUrl":"https:\/\/cdn.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka.png","datePublished":"2021-07-11T15:37:08+00:00","dateModified":"2021-09-15T05:39:35+00:00","author":{"@id":"https:\/\/www.wikitechy.com\/interview-questions\/#\/schema\/person\/4d5a581fb5470d1560324bddc5e8b757"},"description":"What is the difference between flume and Kafka ? - flume interview questions - Distributed reliable system for collecting aggregating","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka\/#primaryimage","url":"https:\/\/cdn.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka.png","contentUrl":"https:\/\/cdn.wikitechy.com\/interview-questions\/apache-flume\/what-is-the-difference-between-flume-and-kafka.png"},{"@type":"WebSite","@id":"https:\/\/www.wikitechy.com\/interview-questions\/#website","url":"https:\/\/www.wikitechy.com\/interview-questions\/","name":"Wikitechy","description":"Interview Questions","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.wikitechy.com\/interview-questions\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.wikitechy.com\/interview-questions\/#\/schema\/person\/4d5a581fb5470d1560324bddc5e8b757","name":"Editor","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/e9531079fe7e07841b7b156c04d65e5f39d4adfd18b6ffe3edfff8ca5aab85b5?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/e9531079fe7e07841b7b156c04d65e5f39d4adfd18b6ffe3edfff8ca5aab85b5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e9531079fe7e07841b7b156c04d65e5f39d4adfd18b6ffe3edfff8ca5aab85b5?s=96&d=mm&r=g","caption":"Editor"},"url":"https:\/\/www.wikitechy.com\/interview-questions\/author\/editor\/"}]}},"_links":{"self":[{"href":"https:\/\/www.wikitechy.com\/interview-questions\/wp-json\/wp\/v2\/posts\/135","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.wikitechy.com\/interview-questions\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.wikitechy.com\/interview-questions\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.wikitechy.com\/interview-questions\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.wikitechy.com\/interview-questions\/wp-json\/wp\/v2\/comments?post=135"}],"version-history":[{"count":5,"href":"https:\/\/www.wikitechy.com\/interview-questions\/wp-json\/wp\/v2\/posts\/135\/revisions"}],"predecessor-version":[{"id":3793,"href":"https:\/\/www.wikitechy.com\/interview-questions\/wp-json\/wp\/v2\/posts\/135\/revisions\/3793"}],"wp:attachment":[{"href":"https:\/\/www.wikitechy.com\/interview-questions\/wp-json\/wp\/v2\/media?parent=135"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.wikitechy.com\/interview-questions\/wp-json\/wp\/v2\/categories?post=135"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.wikitechy.com\/interview-questions\/wp-json\/wp\/v2\/tags?post=135"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}