Elasticsearch Tutorial

 

Getting Started with Elasticsearch

Elasticsearch is an open-source, highly scalable, full-text search, and analytics engine. You can crawl through big volume of data rapidly with the help of Elasticsearch. Generally, it is used in applications where complex search is required. It is developed in Java and licensed under Apache license version 2.0. Nowadays, many big companies around the world are using it for their growth.

What is Elasticsearch?

Firstly, let us understand why Elasticsearch was invented. Consider one example where customers are looking for some product information from huge product volume. But the system is taking too much time for information retrieval due to large volume of data. This in turn leads to poor user experience and there may be chances to lose the potential customer due to the same. RDBMS (Relational Database Management System) works slow when it comes to large amount of data. To overcome this problem, Elasticsearch was invented.

Elasticsearch is a document-based system which stores, manages, and retrieves document-oriented or semi-structured data. Data is stored in JSON document format in Elasticsearch. It is also schema-less. It is a NoSQL database which uses Lucene search engine

Elasticsearch uses Query Domain Specific Language to interact with data. Here, queries are written in JSON format. With the help of Query DSL, we can accommodate all the complex logic in a single query. Query DSL is designed to handle all real-world complex logics in a single query.

Let us explore Elasticsearch features to understand what it offers.

Elasticsearch Features

Below are features offered by Elasticsearch:

• Elasticsearch is best suitable for structured and unstructured data.
• Elasticsearch is an alternative document store for MongoDB and RavenDB.
• Elasticsearch has implemented denormalization to improve the performance of search.
• Many big organizations like Wikipedia, Github, StackOverflow use Elasticsearch for their search engine.
• It is an open source technology.
• It is an easy to use and developer-friendly environment.
• Elasticsearch community is very active and always tries to ensure that Elasticsearch is compatible with everything.

Elasticsearch Architecture

Elasticsearch is not a data store primarily. But, technically yes, we can make it a data store. Elasticsearch stores documents and its versions. If two processes simultaneously start writing to a document, latest version will be kept. It doesn’t support ACID (Atomicity, Consistency, Isolation, and Durability) properties like a database.

Let us understand its architecture by exploring below concepts.

Nodes and Clusters

Node is defined as a single instance of Elasticsearch. Usually, it runs one instance for each machine. Clusters are termed as a collection of nodes which communicate with each other to read/write to an index. Cluster requires a unique name to avoid unnecessary nodes to join the cluster. There is a master node which manages the whole cluster. Master node is responsible for any changes to clusters like adding a node, removing a node, creating or deleting indices, etc. Each cluster and node has a unique name.

Each node in a cluster contributes to the searching and indexing capabilities of cluster. For example, if we have run some search query, each node will execute that to search through the data it stores. Each node supports searching, indexing, manipulating of existing data.

Documents and Indices

Whatever data item we store in cluster is nothing but the document. Document is a JSON object here and we can relate it to rows in database terminology. For example, if you want to store a student, then you will add one object having name and standard as its properties. As we are aware that data will be spread across all the nodes, but do we know how to organize it? These documents are stored under indices. Index is defined as the collection of documents having similar properties or we can say logically related. For instance, an index for orders’ data, products’ data and customers’ data.

Documents have their unique ID, which can be assigned by Elasticsearch or by users while adding them to index. Any document is uniquely identified by its ID and index. There is no limit to number of documents being added to index.

Indices are also identified by their name. Their names can be used to search for any document.

Shards and replicas

Elasticsearch uses Lucene technology for faster retrieval of data. It uses the power of Lucene index in distributed system to retrieve data extremely fast. Shards are termed as individual instances of Lucene index. As data volume increases, index performance also slows down. To overcome this, Elasticsearch uses shards to divide indexes and multiple pieces. Shards are important due to below two reasons.

• Shards enable us to divide the content horizontally
• Shards allow parallel operations across multiple nodes which in turn increases performance.

Replicas are invented to avoid any unexpected network failure. Replica shards, as their name implies, are replicas of index’s shards. Replicas are important in Elasticsearch architecture for below 2 reasons.

• In case of shard or node failure, it will act as a life savior for Elasticsearch. Replica shard is never associated to that node on which primary shard is defined
• Due to replica shards, we can increase the throughput and performance as parallel search can happen on replica shards as well.

While creating index, we can choose number of shards and its replicas. However, we can change number of replicas dynamically anytime.

Elasticsearch Advantages

Below are few advantages of Elasticsearch:

• Elasticsearch is built on Lucene – a full-featured information retrieval library. So, it gives the most efficient and powerful full-text search capabilities of an open source product. It will be great as it is widely known by developers.
• Elasticsearch has implemented a lot of features like Faceted search, customized stemming, customized splitting text into words, etc.
• Elasticsearch supports fuzzy search. As you can find even though there are spelling mistakes in search text.
• Elasticsearch supports intellisense feature which autocompletes your search text by predicting your search based on your search history or completing your text with existing tags. For example, Google search.
• As Elasticsearch is API driven, any action can be performed using a RESTful API.
• Elasticsearch stores any changes in data in transaction loss which reduces the risk of data loss.
• As Elasticsearch is distributed in nature, it is very easy to scale and integrate Elasticsearch in any organization.
• Elasticsearch supports faceted search which is like having multiple filters on data along with classification system over them. This search is more robust in nature than normal text-search.
• Elasticsearch implements multi-tenancy in a better way as a large Elasticsearch index.
• Using Elasticsearch query DSL, it is very easy to prepare complex queries and tune them precisely. Moreover, query DSL provides a way to rank and group the results.
• As Elasticsearch uses JSON objects, it is very easy to communicate with other various programming languages.

Elasticsearch Use-cases

Below are few use-cases for Elasticsearch:

• An online store which allows its customers to explore all the products they sell. In this case, you can use Elasticsearch to store the whole product inventory and catalog. It also allows user to search and use autocomplete option.
• Consider a scenario where you need to store log or transactions which you can use to analyze trends, summarizations, anomalies or statistics. In this case, you can use Logstash, a part of ELK Stack (Elasticsearch/Logstash/Kibana), to store and parse your data. Logstash helps you to feed data into Elasticsearch.
• Have you seen the button “Notify me if item in stock” or “Notify me if price of this item falls down” on e-commerce sites? This feature can be achieved with the help of Elasticsearch. Using Elasticsearch, you can reverse-search and have a watch on price movements or stock movements and send the alerts to customers once conditions are satisfied.
• Consider the requirement where you need to quickly analyze the data and visualize it. In this case, Kibana can be best used with Elasticsearch. Elasticsearch is used to store data and Kibana can visualize that data in various custom dashboards. Kibana is a part of ELK Stack (Elasticsearch, Logstash, Kibana).

Elasticsearch Vs. RDBMS

Elasticsearch is a No SQL database. It doesn’t have any joins, relations, constraints, or any transactional behaviour. Elasticsearch is easier to scale if we compare it with RDBMS. To know in detail, let us compare how Elasticsearch is different from RDBMS.

Elasticsearch Vs. MongoDB

Like Elasticsearch, MongoDB is also a document oriented database management system. They have many features in common like Document-oriented system, schema free, sharding, replicas, high availability, etc. But still they cater to different set of users. Following table depicts the comparison between Elasticsearch and MongoDB.

Elasticsearch Vs. Solr

Solr is also a search-text engine which is built on top of Apache Lucene platform like Elasticsearch. Due to the same platform, they have many similar features. But still they have many differences when it comes to ease of deployment, scalability, and many more functionalities. Below is the comparison between Elasticsearch and Solr.

Current Demand and Future of Elasticsearch

Elasticsearch is the most popular, open source, distributed, cross-platform, and scalable search engine. Elasticsearch is growing exponentially since 2010 and making a remarkable impression all over the IT industry. Due to its exponentia

l growth, there is a very high demand of talents having Elasticsearch skills. IT professionals having knowledge of Elasticsearch are hired with a great salary and are valued more. It is trending in IT industry as it has a very bright future due to its capabilities to handle large amount of data and faster search.

Checkout Elasticsearch Interview Questions

Logstash Interview Questions And Answers

Kibana Interview Questions And Answers

Conclusion

Elasticsearch stands out from all its competitors as it is highly scalable and widely distributed in nature. If you have a large volume of data and you want a faster search, then there is no way you can find anything which is as good as Elasticsearch.