OpenSearch 2.8 has just been released, so we thought it would be useful to some highlights of the new release as well as some useful links for more information. Licensed under Apache 2.0, the open source search engine will reach it’s second official “birthday” on 13 July 2023.
OpenSearch 2.8 Themes
OpenSearch 2.8 is a fairly pragmatic release featuring a handful of new features and enhancements.
These features and enhancements aim to improve the query performance, index management tasks, version upgrade support, and search application functionality in OpenSearch (achieved with some experimental functionality).
There are also the usual number of minor improvements supporting improved performance when using OpenSearch for Security Analytics and Observability use cases.
OpenSearch 2.8 Key New Features
Query Data Across Clusters Using Piped Processing Language (PPL)
You can now query data across your OpenSearch clusters using PPL with cross-cluster support. With PPL cross-cluster querying, you have more flexibility and efficiency in accessing and analyzing data across your OpenSearch environment.
Enhanced Index Management
OpenSearch streamlines index management, simplifying the administration and maintenance of your OpenSearch clusters. From the index management user interface, you can now apply operations such as refresh, flush, and clear cache with ease.
Automatic Notifications for Index Operations
Index operations, such as reindex, split, or shrink, may take a long time to complete. OpenSearch now provides the ability to set up automatic notifications to inform you when long-running index operations have finished. This feature allows you to automatically monitor the progress of these operations, saving you time and improving your DevOps efficiency.
Simplified Upgrades with Segment Replication
To simplify the process of upgrading to a new version of OpenSearch, the segment replication feature now supports mixed cluster versions. This enables smoother transitions when upgrading your OpenSearch clusters, as it allows for compatibility between different cluster versions. You can now confidently upgrade your OpenSearch environment with minimal disruption to your operations.
Default Index Replication Strategy
OpenSearch now offers the option to set segment replication as the default index replication strategy, ensuring consistent and efficient replication of indexes across your clusters. This also simplifies the configuration process, enabling you to establish a standardized replication strategy for your OpenSearch deployment.
Improved Query Latency with Searchable Snapshots
The searchable snapshots feature in OpenSearch has been enhanced to improve query latency. When querying indexes, the feature now defaults to the primary shard, resulting in better performance and reduced latency. The result is faster, more responsive searches.
Expanded Node.js Support for OpenSearch Dashboards
OpenSearch Dashboards now supports a wider range of Node.js versions (14, 16 and 18) based on your preferences and compatibility requirements. This added flexibility and compatibility with your Node.js environment allows seamless integration of OpenSearch Dashboards into your existing infrastructure.
Optimized k-Nearest Neighbors (k-NN) Functionality
The k-nearest neighbors (k-NN) functionality in OpenSearch has been updated with optimizations for native memory allocations. These optimizations enhance stability and performance when dealing with large workloads. Improved k-NN functionality improves efficiency and reliability in use cases such as similarity searches and recommendation systems.
Amazon OpenSearch Serverless Support for Multiple Data Sources
OpenSearch now supports Amazon OpenSearch Serverless with the SigV4 authorization type in its multiple data sources functionality. With this extended support, you can easily incorporate Amazon OpenSearch Serverless into your data ecosystem, enabling efficient data management and analysis.
Experimental Features
Highlights of OpenSearch 2.8 include a new experimental feature: search pipelines, which is disabled by default. To enable this feature, you can refer to the documentation for instructions.
The new search pipelines functionality provides a way for search application developers to construct a sequence of search processors within an OpenSearch cluster. These search processors serve as building blocks that can be combined to integrate various components into the search workflow. The three built-in search processors included in this release are:
- Filter_query: This processor allows you to apply filters to the search query. It helps refine or limit the results returned by the search based on specific criteria or conditions defined in the filter.
- Rename_field: With this processor, you can modify the names of fields in the search results. It enables you to customize the field names to match your requirements or conventions.
- Script_request: This processor enables the execution of custom scripts or code during the search request. You can use it to perform additional operations or calculations on the search results using your own scripting logic.
These search processors offer flexibility and customization options for manipulating the search query, refining the search results, and incorporating additional functionality into the search process.
Where will OpenSearch go next?
Though it doesn’t guarantee exact release features, the OpenSearch Roadmap gives a clue for what’s coming with the open source search engine.
There are a lot of candidate features, but OpenSearch 3.0 is still vague at this time, based on publicly available information. You can expect one or two minor releases before 3.0, with likely improvements for neural search / k-NN, machine learning models, and anomaly visualization (for log analytics use cases).
It is also unclear how OpenSearch will respond to Elastic’s further investments in integrating AI and Elasticsearch, with some of these developments being proprietary to Elastic.
Wrapping Up
We hope you enjoyed this summary of OpenSearch 2.8 highlights. If you have any further questions, need any support or advice with your OpenSearch implementation we would be happy to help! CONTACT US.
Relevant resources
From OpenSearch:
Pureinsights Services:
- Search Application Consulting | Pureinsights
- OpenSearch Consulting | Pureinsights
- Amazon OpenSearch Service Consulting |Pureinsights
- Pureinsights Achieves AWS Select Tier Services Partner Status | Pureinsights
Blogs: