/

Elasticsearch vs OpenSearch : The Elastic View (Part 2)

Elasticsearch vs OpenSearch : The Elastic View (Part 2)

In early 2021, on the heels of a major licensing change by Elastic, Amazon announced the OpenSearch project, a code branch of Elasticsearch and Kibana under the Apache 2.0 open-source license. In the second of a three-part Elasticsearch vs OpenSearch blog series, we will examine Elastic’s point-of-view and reactions. Other blogs in the series cover the User point-of-view and the Amazon point-of-view.

Elasticsearch vs OpenSearch: Origins and Milestones

To understand Elastic’s reaction to the Elasticsearch vs. OpenSearch saga, you have to start with the company’s origins and early cult-like following. And we do mean “cult” in the best possible sense.

Shay Banon created Elasticsearch in 2010 and formed Elasticsearch Inc. in 2012, near the height of the enterprise search market. At the time, Apache Solr reigned as the open-source search engine king. But how did Elasticsearch usurp the throne in such short order? 

Elasticsearch (the product) was the foundation upon which Elastic (the company) was built. Elastic features a colorful origin story on their website and we will cover some important milestones illustrated in the timeline below.

Elasticsearch vs OpenSearch Key Milestones

Elasticsearch Crashes Three Parties: Search, Analytics, and Security

There were a lot of very good search engines available in 2012 (and we had worked with most of them). Autonomy was the most hyped, Google’s brand recognition translated into success for the Google Search Appliance, and SharePoint benefited from Microsoft’s huge user base.

But Elasticsearch made an early big impression in two ways:

  • First, it ticked a lot of checkboxes. It was very good, very scalable, very fast, and relatively easy to deploy. Elasticsearch downloads quickly reached millions. You have to credit Shay Banon with masterful coding in the early versions.
  • Second – and very important – Elasticsearch paired itself with two other very good open-source projects, Logstash and Kibana (ELK). The “ELK stack” or “Elastic Stack” use cases went beyond enterprise search to log analytics, just when “Big Data” hype was about to explode.

Our experience at the first Elasticon in 2015 was a revelation. Shay Banon received a rockstar-like reception at the keynote and took the time to introduce us to a prospect from Disney. There was palpable excitement in the air that we were all at the early stage of something big.

From an enterprise search perspective, we were also surprised at how many use case presentations there were about search and analysis of log data, including one about the Mars Rover. But despite Elasticsearch’s millions of downloads, neither Gartner nor Forrester paid much attention to Elastic in their research on the enterprise search space. Elastic didn’t seem to care.

After the summit, it dawned on us: “they’re going after Splunk.”

Enterprise search is a multi-billion-dollar industry, but the “Big Data” / log analytics market that Splunk dominated was worth perhaps ten times that. This market incorporates analytics for business transactions, the emerging IoT (Internet of Things) market, and security-based use cases like SIEM (Security Information and Event Management).

Big Profits from Big Data

This was the first time that David was taking on Goliath, though you will never hear Elastic say this explicitly. They were taking on Splunk, the log analytics “gorilla”, armed with a great technology stack and the halo effect of being open source. Better, sexier, and cheaper.

Fast-forward six years, and a lot has changed. Elastic received $70m in funding, launched an IPO, and in 2021, annual revenues exceeded $600 million, a staggering 48% increase over 2020. With the growth, some of the halo effect wore off. In the open-source community, there’s almost an unwritten commandment that says “thou shalt not make TOO much money.” Elastic’s success certainly came with growing pains, and managing a public company with demanding shareholders is no picnic.

But a disruptive trend in IT loomed: everything was moving to “the cloud”. Technology executives everywhere scrambled to develop a cloud strategy, including how to deal with the leading vendors in the market. If you liken the cloud market to our solar system, Alibaba, Oracle, IBM would be smaller planets and asteroids, Google would be Saturn, Microsoft is Jupiter. And shining bright and smack in the middle is the Sun – Amazon.

Elastic, The Cloud and World Domination

Everything in IT is now about the cloud. Everything.

Elastic offers many of its services as cloud-based offerings, and while some of those services are proprietary, the dominant Elastic Stack is still freely available. Most companies today look to deploy managed services versions of Elasticsearch and Kibana with one of the three top cloud providers: Google, Microsoft, or Amazon Web Services (AWS).

Looking to capitalize on this, Elastic announced the Elasticsearch Service for Google Cloud and Elastic Stack for Microsoft Azure partner solutions. Presumably, Elastic benefits from some revenue stream and visibility into the deployment of their technologies with these partners. Conspicuously absent was any collaboration with AWS.

AWS has always leveraged open-source technologies to improve the range of services available on its cloud.  Amazon launched CloudSearch in 2012, starting with its own A9 technology, but quickly switching to Solr, the leading open-source search engine at the time. In 2015, with the increasing popularity of Elasticsearch, AWS launched the Amazon Elasticsearch Service, without collaborating with Elastic. Banon and Elastic management were probably incensed, but Elastic was focused on other things, and the AWS relationship took a back seat.

In 2016 Elastic announced the launch of several commercial packages of its technology, including Elastic Cloud and X-Pack. X-Pack was a commercial packaging of Elasticsearch, Logstash, Kibana, and Beats. To justify its pricing, X-Pack included proprietary features like security and management, to help Elastic monetize their open-source assets. Elastic revenues grew to $160 million culminating in a successful IPO in 2018.

Elasticsearch vs OpenSearch : The Last Straw

The straw that broke the camel’s back came in February 2019. Amazon announced support for Open Distro, the open-source community’s response to X-Pack. Open Distro was a branch of Elasticsearch and Kibana, and Amazon contributed security and management enhancements, making it a direct competitor to X-Pack. As documented in our first blog in this series, things got ugly quickly with Elastic launching a lawsuit claiming copyright infringement.

In early 2021, Elastic finally felt compelled to protect its business and shareholder interests by changing its licensing model. Amazon responded by morphing Open Distro into the OpenSearch project. Elasticsearch vs OpenSearch and the rift between Elastic and Amazon became very public, and blog posts and tweets from Shay Banon told you this didn’t feel like business. It felt personal.

From Banon’s s point of view, it must feel like you opened a great restaurant and even shared your best recipes. Your restaurant became a hit, and some chefs came to dine. Then one day, you find out that one of the chefs has added your recipes to their menu without asking. That would make anyone go on an expletive-filled Gordon Ramsey rant, wouldn’t it? Maybe not a perfect analogy, but probably close.

The last questions to examine relate to Elastic’s licensing change. What is the licensing model, and who is impacted? Is Elastic still an open-source technology company?

What is SSPL and Elastic License v2?

Elastic seemed to be targeting Amazon’s “misbehavior” when it changed its licensing model in 2021. Instead of the permissive Apache 2.0 license, Elastic is giving users the option of choosing Elastic License 2.0 (ELv2) or Server-Side Public License 1.0 (SSPL). Elastic explains the licensing changes on a purpose-built FAQ page on their website.

ELv2 is a brief license that clearly spells out that Elastic IP cannot be used to provide managed services that include Elastic software functionality to third parties (customers). This clearly targets SaaS and IaaS providers, including dominant platforms like AWS.

SSPL is a “source-available” (vs. “open-source”) software license model introduced by MongoDB in 2018 to protect its ability to profit from its intellectual property while facilitating a community model for code base contributions. SSPL stipulates that any third-party service provider that incorporates SSPL-licensed software must also release the entirety of their source code under SSPL.

If you are a SaaS or IaaS provider that embeds Apache 2.0 Elastic code or products, you face two unpleasant choices. ELv2 forces you to migrate away from Elasticsearch, and SSPL would force you to open up your entire solution’s code base. Negotiating a business relationship with Elastic is also a viable, if complicated option.  

As pointed by Technology Analyst Christopher Tozzi in this article in ITPro Today:

“That’s a big deal for public cloud providers like AWS, which offers a managed service based on Elasticsearch. The licensing change means that AWS can no longer use Elastic’s version of the Elasticsearch platform on its cloud.”

To protect its now substantial business, Elastic was willing to suffer any short-term adverse PR while inconveniencing many service providers that represent little if any revenue. One gray area is large corporations like Netflix, Expedia, and Capital One, who use Elasticsearch and may be broadly considered as SaaS providers. They probably consider AWS a more strategic vendor, and the Elastic licensing changes seems to have pushed them further into the Amazon / OpenSearch camp.

Is Elasticsearch still Open Source?

Technically, no. The real question, though, is “does it matter”?

SSPL is not recognized by the Open Source Initiative (OSI), and Apache 2.0 is the accepted standard. No matter how Elastic tries to spin it, that’s a fact. But now that business is booming, Elastic no longer needs the “halo” of open source. With a successful IPO, it also no longer needs to appeal to VC firms looking for “the next Red Hat”.

Elastic is doing what most successful companies that started as open source are doing: evolving. An article in the COSS Community site outlines the evolution of companies from fully open source to more protected community-driven models. Anyone can join and contribute to the “community”, but you have to abide by the rules. And Elastic owns and writes the rules for its community.

Rationalizing Elastic’s Actions

Someone once said that in politics, you can question someone’s actions, but not their motives. Taken as a whole, Elastic’s actions are easy to rationalize.

  • Elasticsearch was created using a fully open-source model that made it wildly popular, both with developers and VC firms. When its assets grew to encompass Logstash and Kibana, the company changed its name to Elastic, and targeted the huge emerging “Big Data” market.
  • The expanded target market helped Elastic raise over $70 million in funding and gave it much larger revenue potential (on the scale of Splunk). Elastic conducted a successful IPO and grew revenues substantially by offering training, support, and value-added products and services to their open-source assets.
  • Like every technology company today, Elastic had to address the “megatrend” of everything moving to cloud-based service models. They established relationships with Google and Microsoft, offering the Elastic technology stack as cloud services. There was no deal with AWS, but given Elastic’s incredible recent growth rate, perhaps they feel they don’t need Amazon.
  • Amazon’s support for Open Distro (and now OpenSearch) clearly threatened Elastic’s growth potential and fueled the Elasticsearch vs OpenSearch feud. Elastic changed its software licensing model simply to defend the business. After serving Elastic during its early growth period, the Apache 2.0 licensing model had outlived its usefulness.

Follow The Money

In the end, if you put aside the altruistic intentions of any open-source project, Elasticsearch vs. OpenSearch comes down to clashing business models on how to make money.

Amazon has always taken their cues from Walmart: offer quality products and beat the competition on volume and price. For AWS, this strategy includes leveraging IP from successful free open-source projects. The challenge for any open-source company like MongoDB or Elastic is to grow profitably in the long run. Both companies had to evolve from a pure open-source model to an open community-based model. This strategy is meant to please the majority of users, but most importantly, to please shareholders.

Wrapping Things Up…

In the next and last blog on Elasticsearch vs OpenSearch, we’ll cover the Amazon (and OpenSearch) point-of-view. If you enjoyed this post, please consider sharing it using the social media buttons below.

If you are struggling with deciding whether to use Elasticsearch or OpenSearch for your search-based applications, please CONTACT US for a free 1:1 consultation or explore our range of search application services.

Additional Resources

If you missed any part of the series, you can read them all here:

Twitter
LinkedIn

Stay up to date with our latest insights!