• VB.NET offers code execution that runs under the Common Language Runtime (CLR), resulting in robust, stable and secure applications. Both features of the .NET framework are eagerly obtainable in VB.NET.
• VB.NET is object oriented. This is a major addition that VB6 and other earlier releases didn’t have.
• The .NET framework comes with ADO.NET, which follows the disconnected paradigm, i.e. once the required records are fetched the connection no longer exists.
• It also retrieves the records that are expected to be accessed in the immediate future.
• This enhances Scalability of the application to a great extent.
• VB.NET uses XML to transfer data between the various layers in the DNA Architecture i.e. data are passed as simple text strings.
• Error handling has changed in VB.NET. A new Try-Catch-Finally block has been introduced to handle errors and exceptions as a unit, allowing appropriate action to be taken at the place the error occurred thus discouraging the use of ON ERROR GOTO statement. This again credits to the maintainability of the code.

• Pig is a platform for managing large data.
• It uses high level programming to analyse the data.
• The programming is done using the language pig latin.
• It is simple and highly specific.
• The pig Metastore stores all info about the tables. And we can execute spark sql queries because spark can interact with pig Metastore

• Sqoop is a simple tool used for transferring large amount of data.
• It is used for transferring data from hadoop to relational database.
• sqoop can import data from Oracle,MYSQL,SQl.
• Sqoop metastore is a tool for using hosts in a shared metadata repository. Multiple users and remote users execute saved jobs in metastore.

• Staging table is used to stage the exported data.
• Sqoop breaks down method into multiple transactions. A failing job may create partial knowledge.
• This could result in job failing because it could cause duplication of data or even collision while insertion.
• Staging tables are simple database tables that contain business details.
• Staging table acts as an auxiliary table that’s used to transferred data.
• Sqoop breaks down export method into multiple transactions, it’s possible that a failing export job may lead to partial data being committed to the database

• Sqoop is a simple tool that is used for moving data.
• It is generally used move data from non hadoop data stores.
• These non hadoop data stores are the relational databases and data warehouses .
• Sqoop allows the users to specify the target (i.e) the place where the data is to be moved inside of hadoop from relational databases like Oracle,Teredata.
• Data ingestion is important in any big data project because the volume of data is generally in petabytes or exabytes.
• Hadoop Sqoop and Hadoop Flume are the two tools in Hadoop which is used to gather data from different sources and load them into HDFS.

• Sqoop metastore is an internally implemented database that can hold the definition of a given job.
• It includes details like which database,which table,which approach,what query to extract from,what target to write to and so on.
• Consider an array of SQL databases that have the same structure, but are partitioned by for performance reasons.
• Thus we have the same sqoop job running against multiple database instances at the same time, all unified at a single table in Hadoop.
• In order to get parallelization, we generate a job that manages the Sqoop sub-workflows. This results in 500 distinct sqoop jobs.
• Metastore is used to handle these complexities.

• Apache Sqoop is efficiently transferring data between Relational databases and Apache Hadoop.
• It is used to import data from relational databases such as MySQL, Oracle to Hadoop HDFS, and export from Hadoop file system to relational databases.
• It improves the efficiency of data analysis by combining structured data and unstructured data.
• Sqoop stands for SQL for Hadoop.SQL + HADOOP =SQOOP

• Here, some of the key advantages of using splunk which will help you in understanding key features of splunk.

• Identify and resolve issues upto 70% faster.
• Reduces costly escalations by upto 90%.
• Splunk change difficult logs to visual graphs and reports resulting simplified analysis,reporting and troubleshooting

• No separate database requirements like oracle or SQL as splunk stores all data in its index
• Supports any format and any amount of data -enables centralized log management

• Simple to implement and scale
• Continually index all of your IT data in real time.
• Automatically search helpful information inserted in your data, therefore you can’t to spot yourself.
• Search your physical and virtual IT infrastructure for virtually something of interest and find results in seconds.
• Save searches and tag helpful information, to create your system smarter.
• Set up alerts to modify the observation of your system for specific continual events.
• Provide analytical reports with interactive charts, graphs, and tables.
• Share saved searches and reports with fellow Splunk users, and distribute their results to team members and project stakeholders via email.
• Design specialised, information-rich views and dashboards that work the wide-ranging desires of your enterprise.

• The application of splunk are Monitoring, Analyzes machines, security and business performance.
• With intuitive analysis features, machine learning, packaged applications and open APIs, Splunk Enterprise is a comfortable environment that scales from focused use cases to an enterprise-wide analytics backbone.

• A Splunk application is not much more than a structured set of configurations and assets help to achieve an end goal of data collection, indexing, and visualization.
• In order to make a valid Splunk application, you must include the ability to navigate. Without navigation in the application, you would be working on an add-on.
• The performance of application can be monitor vendors Dynatrace and Extrahop have developed apps for Splunk software to help the users to visualize their deep-dive application performance data in the event of all other application data like logs, events, infrastructure performance.

• Proxy logs : It’s better for C2 analysis of files, domains, downloads of DLL/EXE files.
• Anti‐virus logs : It’s good for analysis of malware, vulnerabilities of hosts, laptops, servers, monitor for suspicious file paths.
• Server Operating System logs : These logs are good for analysis of server activities such as users, runaway services, security logs.
• Firewall logs : Logs for network traffic of source/destination ip addresses, ports, protocols.
• Mail logs : Logs for inbound/outbound mail for malicious links, targeted recipients, unauthorized file out bound, data loss, bad attachments.
• Custom apps logs : Logs could be analyze for possible buffer overflow, code injection, SQL injection analyses.

• Intrusion Prevention System logs : To alert on signatures firing off, COTS signatures, threat analysis of bad network packets.
• Database logs : It’s can be capture for authorized access to critical data tables, authorized logons, op ports, admin accounts.
• Virtual Private Network(VPN) logs : Capture logs to analyze users coming into network for situational awareness, monitored foreign ip subnets, compliance monitoring of browsers/apps of connected hosts.
• Authentication logs : To monitor authorized/unauthorized users, times of day of connection, how often, logons/logoffs, BIOS analysis.
• Vulnerability Scan Data : Import data about assets, vulnerabilities, patch data, etc.
• Web Application logs : External facing logs to monitor suspicious SQL keywords, text patterns, REGEX for threats coming in through browser.
• DNS logs: To relate IP address what domain in a client level.
• DHCP logs : To monitor what systems are assign what IP address and how long, how often.

• Active Directory/Domain Controller logs : Monitor user accounts for AD admins, privilege accounts, remote access, multiple admins across the domain, new account creation, event ID’s.
• Badge Access logs : Logs to capture to correlate insider threat, situational awareness, correlate data with authentication logs.
• Router/Switch data (net-‐flow) : Capture this critical data source for APT monitoring, network monitoring, data exfiltration, flow analysis are very important data source.
• Packet Capture logs(PCAP): Very difficult data source to capture for Advanced Persistent Threats, packet analysis, deep packet inspection, malware analysis, etc.
• FW + AV : Will help detect and respond to viruses, worm propagation.
• IPS + AV + FW : Detect/alert on network based attacks such as buffer overflow, reconnaissance scans, code injection.
• PROXY : The web based/application layer is a majority attacks to monitor like: cross-site scripting, session hacking, browse redirects.
• AV + PROXY : Monitor/detect/respond to download of bad files, remote code execution…web-based attacks.
• FW + PROXY : Detect outbound data exfiltration, detect potentially misconfig fw rules.
• IPS + FW : Network packet of signature threats to be monitored.
• AD Server : All user/group modifications, deletes, updates for administrators to be monitored.
• AD + PROXY : Monitor/Detect/Alert on post compromise analysis, lateral movement.

• Splunk is a software environment to give search, analyze and event to machine-generated data collect from the websites, applications, sensors,devices etc.,It create for your IT environment and business.
• Splunk is an advanced,scalable and potent technology that system can be stored indexes and searches log files.It also search the machine-generated data to give operational intelligence.
• The below image will help you how to relate Splunk collects data.

• To report real time health condition.
• The patient’s health record and analyze patterns in the Delve deeper.
• When the patient’s health degrades to be Alarms/Alerts for both the doctor and patient.

• It work can be moved as the ‘Google’ for log files.
• Simpler terms to be searched with the use of Search Processing Language (SPL).
• If the Splunk file system to be store the data directly that means it does not require any back-end or any database
• It doesn’t against any single point of failure to be happened.

Some of the additional components that separately and collectively perform the major operations:

• Strong project management skills.
• Good Communication.
• SQL is easy to learn a most of analytics skills.
• Decent grasp Python-Pandas or R to do data crunching and manipulation.
• Based on Good visualization skills (start by checking out CSS color palettes).
• The Tableau or Power BI come at the very end of the analytics pipeline, it’s good you have an overall grasp.

• Power BI stores all the uploaded data into Azure BLOB and Azure SQL Database.
• Both databases are only accessible by an Admin user, which means that end-users cannot access the database, eliminating user-level security.
• Now this data is on the Microsoft cloud and when it comes to their security, there isn’t much to explain except to say that the data is 100% secured.

• • Power BI Service
  • Power BI Desktop

• Cloud SaaS is called the Power BI service for accessing the cloud service.

• Power BI Desktop was an free application that works in joining together with the Power BI Service.
• Power Bi Desktop allows you to do the following.

• • Get the data from collection of unlike sources.
  • Create relationships between your data and to end with fine or desirable qualities of your data model.
  • Then Create and save your reports.
  • And Upload or publish your reports.

• The reports created using Power BI Desktop then published in Power BI Service right from the Power BI Desktop application.
• Using both Power BI Service and Power BI Desktop the cost is free.

• Power BI will not accept files larger than 1GB.
• Power BI reports and dashboards cannot accept or pass user, account or other entity parameters.
• This makes it impossible to create entity specific dashboards such as a dashboard for an account, opportunity, case, or campaign.
• Very few data sources that permit real-time connections to Power BI reports and dashboard.
• Users have the same email domains listed with your office 365 tenant can only be shared dashboards and reports.
• The integration to Dynamics CRM is limited.