Graham Gillen
OpenSearch 2.0 has just been released, so we thought it would be useful to some highlights of the new release as well as some useful links for more information.
OpenSearch 2.0 Major Features
Performance and Modularity Enhancements with Lucene 9.1 Upgrade
The most significant feature in OpenSearch 2.0 is the upgrade to Lucene 9.1. Similarly to Elastic 8.0, which incorporates Lucene 9.0, this upgrade billed as a significant performance boost, promising 10-15% faster indexing and faster I/O.
The Lucene 9.1 upgrade also includes Java Jigsaw module support, meaning Lucene JARs are now proper Java Module. This aligns with OpenSearch’s intent to make sure the toolset is modular and extensible.
Security Enhancement: Document-Level Alerting
Security is an oft-ignored capability in search applications, especially down to the document level (versus folder or repository level). OpenSearch 2.0 adds the capability to create monitors that generate alerts on a per-document level. This enhances the overall document-level security of the platform.
Better OpenSearch Plugin Management: Notifications Plugin
OpenSearch 2.0 includes a Notifications Plugin system that allows developers / admin users to configure and manage Plugin notification channels in one interface. We note the irony of a plugin for monitoring plugins. This is particularly useful for complex deployments using numerous plugins.
New Algorithms for Machine Learning: MLCommons Upgrade
The MLCommons plugin introduced in OpenSearch 1.3 helps users build and train machine learning models. OpenSearch 2.0 upgrades the plugin two add new algorithms for linear regression and localization. Linear regression is a must-have for numerical predictive analysis in ML models. With localization, users get help for developing ML models used for anomaly detection and visualizations for root cause analysis and similar use cases.
OpenSearch 2.0 vs Elastic 8.0: No Approximate Nearest Neighbor?
The major announcement about Elastic 8.0 is summarized in this Press Release. Like the OpenSearch 2.0 announcement, the details mention the Lucene 9.x upgrade and the resulting performance enhancements.
However, the Elastic 8.0 makes a big deal about improved vector search using an Approximate Nearest Neighbour (ANN) capability, making it possible to do vector-based comparisons of text at speed and scale. Vector search turns text-like queries into mathematical representations that can be used to match against text in documents being searched. ANN allows developers to trade off accuracy for speed when using vector search. This is important when trying to use vector search in massive text repositories.
By comparison, OpenSearch 2.0 does not make any direct mention of ANN. Instead, there is information on k-NN, which is more of an exact match approach. The documentation does give advice on how to use k-NN for approximate matches.
At this time, and without further evaluation and testing, we cannot make a judgement on whether OpenSearch or Elasticsearch has the better ANN capability. We simply note that vector search in an increasingly important capability in any search application, and that Elasticsearch has emphasized ANN more in its most recent release.
Wrapping Up
We hope you enjoyed this summary of OpenSearch 2.0 highlights. If you have any further questions, need any support or advice with your OpenSearch implementation we would be happy to help! CONTACT US