What are the steps to deploy a big data solution ?



What are the steps to deploy a big data solution ?


    There are three steps to deploy a Big Data Solution
Deploying Big Data solution

Data Ingestion

  • The first step for deploying a big data solution is the data ingestion i.e. extraction of data from various sources.
  • The data source may be a CRM like Salesforce, Enterprise Resource Planning System like SAP, RDBMS like MySQL or any other log files, documents, social media feeds etc.
  • The data can be ingested either through batch jobs or real-time streaming. The extracted data is then stored in HDFS.
Data Ingestion in Big Data

Data Ingestion in Big Data

Data Storage

  • After the data ingestion, the next step is to store the extracted data. The data either be stored in HDFS or NoSQL database (i.e. HBase).
  • The HDFS storage works well for sequential access whereas HBase for random read/write access.
Data Storage

Data Storage in Big Data

Data Processing

  • The final step in deploying a big data solution is the data processing.
  • The data is processed through one of the processing frameworks like Spark, MapReduce, Pig, etc.
Data Processing in Big Data

Data Processing in Big Data



Related Searches to What are the steps to deploy a big data solution ?