Difference between flume and Kafka<\/h2>\n<\/div>\n<\/div>\n\n\n\n\n\n\n\n\n\n\n\n

Flume<\/th>\n	Kafka<\/th>\n<\/tr>\n
The Flume is a Distributed reliable system for collecting, \naggregating and moving large amount of data to \ncentralized datastore like HDFS or Hbase<\/td>\n	General purpose publish – subscribe \nmodel messaging system<\/td>\n<\/tr>\n
Adding more consumers means to change the \ndesign of flume pipeline and replicating the channel to \ndeliver messages to new sink which needs \ndowntime<\/td>\n	Easy to add more consumers without downtime<\/td>\n<\/tr>\n
Supports many built-in sources and sinks \nout of box<\/td>\n	Sometimes need to write own producer and consumer code though Spark and Storm have now come up with built-in integrations to Kafka<\/td>\n<\/tr>\n
Flume pushes data into sink and hence \nconsumers do not have to maintain offset<\/td>\n	Subscribers are responsible for pulling data \nand also maintaining pointer to offset<\/td>\n<\/tr>\n
Events are lost in case the agent goes down<\/td>\n	Provides fault tolerance<\/td>\n<\/tr>\n
Does not support partitioning<\/td>\n	Supports partitioning<\/td>\n<\/tr>\n
Flume pushes data to the sink because of \nwhich writes to sink can overwhelm data reads from sink<\/td>\n	Since kafka does not push data, writes from producer to broker and reads from broker to consumers can happen at their own pace<\/td>\n<\/tr>\n
It is tightly integrated with Hadoop<\/td>\n	General purpose<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n \n <\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":" Answer-Kafka can support data streams for multiple applications<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"passster_activate_protection":false,"passster_protect_child_pages":"","passster_protection_type":"password","passster_password":"","passster_activate_overwrite_defaults":"","passster_headline":"","passster_instruction":"","passster_placeholder":"","passster_button":"","passster_id":"","passster_activate_misc_settings":"","passster_redirect_url":"","passster_hide":"no","passster_area_shortcode":"","gtb_hide_title":false,"gtb_wrap_title":false,"gtb_class_title":"","gtb_remove_headerfooter":false,"footnotes":""},"categories":[369],"tags":[418,416,414,413,417,422,415,412,411,420,421,410,364,419],"class_list":["post-135","post","type-post","status-publish","format-standard","hentry","category-apache-flume","tag-apache-flume-disadvantages","tag-apache-flume-limitations","tag-apache-flume-vs-sqoop","tag-apache-flume-vs-storm","tag-disadvantages-of-flume","tag-flume-interview-question","tag-flume-kafka-integration","tag-flume-vs-kafka-vs-spark","tag-flume-vs-spark-streaming","tag-flume-vs-streaming","tag-kafka-vs-flume-vs-scribe","tag-kafka-vs-sqoop","tag-wells-fargo-interview-questions-and-answers","tag-which-type-of-channel-will-provide-high-throughput"],"yoast_head":"\n