Ricardo León
Curated Practical Advice from Online Resources
In previous blogs, we discussed the Elastic license change and the emergence of OpenSearch as a pure open source alternative. While we don’t expect this development to impact Elastic’s core business from paying customers, a considerable number of companies are likely to be researching the implications of migrating from Elasticsearch to OpenSearch.
In this blog, we curated what we think is the most practical advice we found online on this topic – assuming that you are seriously considering migrating. We will not cover whether or not you should migrate, as that is a complex decision based on interpreting the Elastic license and how you leverage the software in your business.
Migrating from Elasticsearch to OpenSearch is easy, right? It depends.
When OpenSearch 1.0 was released in July 2021, some quick articles were written making it seem that migrating would be simple and easy. Just take a snapshot of your Elasticsearch index and make it visible to your OpenSearch server. You may have to re-index if you were running older versions of Elasticsearch.
Simple, right? Well, then answer is, it depends. There still is the question as to what it will take for your application built on Elasticsearch to work with OpenSearch.
This short OpenSearch blog does provide a better overview of migration considerations, and we elaborate on some of these considerations below.
Before You Start: Versions Matter
OpenSearch launched in September of 2021 as an Apache 2.0 licensed branch of Elasticsearch 7.10.2. As of the date of this blog, OpenSearch 1.3 is the latest version. The version of Elasticsearch that you are on is the most important factor in determining your migration path.
- Elasticsearch 7.10.2 is compatible with OpenSearch 1.x
- Elasticsearch 5.x and 6.x will require straightforward migration to 7.10.2
- Elasticsearch versions 7.11+ may have a migration path depending on community tools that emerge
- Elasticsearch 8.0 or later has no proven migration path currently exists at this time
We expect that migration will be increasingly difficult over time. As the Elasticsearch and OpenSearch code diverges, there are no guarantees that the OpenSearch community will want to invest in developing migration paths and scripts.
The Most Critical Compatibility Concerns
When migrating from one search platform to another compatibility is critical in two key areas, and the same applies when migrating from Elasticsearch to OpenSearch.
- Index compatibility – can OpenSearch read your latest Elasticsearch index?
- Wire compatibility – will your application built on Elasticsearch APIs and client libraries be able to communicate with OpenSearch servers? (i.e. will they work)
Since OpenSearch branched from Elasticsearch, index compatibility is usually a minor issue to address, unless Elastic makes significant changes to the index in versions 8.x and beyond.
Wire compatibility is a much more complex issue. OpenSearch had to do considerable work to create sanitized versions of the Elasticsearch client libraries prior to the release of OpenSearch 1.0. AWS has announced their commitment to OpenSearch and the work required to maintain API compatibility and the client libraries necessary for popular development languages.
Best Current Methods for Migrating from Elasticsearch to OpenSearch
Reputable online articles summarize the migration methods as follows:
- Snapshots
- Rolling upgrades
- Reindex API
- OpenSearch upgrade tool
If you are running Elasticsearch 7.10.2 to 7.11.2 the OpenSearch Snapshot API is the most direct migration path. This API makes a copy of the Elasticsearch repository available and restorable as an OpenSearch 1.x repository. Snapshots include cluster and node settings and index metadata.
If you are running Elasticsearch versions 5.x or 6.x, rolling upgrades provide a path to get to 7.10.2 and enable you to use snapshots to migrate to OpenSearch. The upgrade process may require reindexing, which can take time if you have a lot of content.
The Reindex API from OpenSearch offers a way to migrate from 7.12+ to OpenSearch 1.x. You can also use this method for Elasticsearch versions 5.x or 6.x if you want more fine grain control over the content you want to migrate and reindex.
OpenSearch provides basic instructions on migrating from Elasticsearch. Starting in OpenSearch 1.1, the OpenSearch upgrade tool automates some of these steps, including some of the error-prone manual steps in the process.
For a more detailed overview of upgrade methods, you can also check out this article from Opster.
Final Considerations in Migration Planning
We’ve only scratched the surface of what it takes to ensure that an application running on Elasticsearch will run seamlessly on OpenSearch. If you have a production application, we all know this is like changing tires on a moving bus. Other things to consider for your search application when migrating from Elasticsearch to OpenSearch:
- Should you migrate? In some situations, it may NOT be in your best long-term interest to switch
- If you do migrate, this may be an opportunity to “clean out the garage” and only migrate and index the most relevant content
- Testing migrations with a subset of your data and infrastructure will reduce migration risk
- A full QC cycle is recommended, including configuration checks and application testing
- Upgrading and testing on staged infrastructure that duplicates production environments also reduces risk
CONTACT US for a free 1:1 consultation on migrating from Elasticsearch to OpenSearch, or for any other questions about our search application technology and services.