What is ManticoreSearch ?
Open source text search engine for big data and stream filtering
- Filtering multi-language user generated data streams in real time
- Indexing and searching over hundreds of terabytes of data
- Search results grouping and faceting
- High-quality search results ranking following best industry practices
- Backend for search UI (eg. “Did you mean”, search phrase autocompletion, spelling correction etc.)
More in depth
Manticore Search is an open source high performance full-text search oriented engine. It was started in May 2017 as a fork of Sphinx Search by a number of former Sphinx employees with help of few big companies that suffered from lack of support from Sphinx Technologies. Since the beginning the key priorities for the new company were:
- Provide to existing users of Sphinx as good support services as possible. We do 24x7 support and have multiple plans from free and very cheap for those who just need some assistance to enterprise where all areas and aspects of dealing with the engine are covered.
- Increase stability:
- Starting as a fork of then latest available version Sphinx 2.3.2-beta the first month was fully devoted to fixing most known bugs that existed then.
- The next step was to improve automated testing, so more than 100 new tests have been added since then and the project migrated to Google Test framework, we started doing regular static analysis and solved most found issues.
- The 3rd key component of higher stability is pre-release testing. Working closer as never with the busiest users allows to discover and fix faster issues that can only be seen on big setups.
- Continue development:
- Since the end of 2016 Sphinx development and support was suspended and there was a risk that it would never continue (again, that was the key reason of starting the company), but even before that the release schedule was not stable and many bugfixes were waiting to be released for months. In Manticore we make monthly releases and always prioritize bugfixing over new features development and deployment.
- We try to be as close to the community as possible to understand what are the real flaws in the product and adapt our roadmap to cover that. We’re actively reviewing and answering questions on StackOverflow and other similar sites, accept pull requests on GitHub, send monthly newsletter etc. We even have an open Slack anyone can join and share his problems and suggestions.
- Besides bugfixing we’ve already made many improvements and new functionality among which a new type of indexes — percolate and a new JSON interface. Many more can be found in release notes.
Differences from Sphinx
Manticore is not a new software. While “under the hood” a lot of changes have been made since forked, little to none changes are needed regarding usage if you upgrade from Sphinx. Manticore can still load indexes made with Sphinx (however this will change soon because of upcoming changes in index structure, yet it will not require any changes to search queries).
As well as Sphinx, Manticore Search is written in C++. It means speed and low resource consumption, it means you don’t have to worry about a garbage collector that suddenly makes a trouble.
In December Sphinx development was resumed and v3.0.1 was released with few interesting features (docstore, A-indexes and few others), but unfortunately it’s not open source any more. The biggest problem with this kind of distribution is if the author stops offering support again, no one can fix potential bugs (e.g. significantly degraded performance). For an open-source project the sources are public, and even if the project is abandoned someone can always understand and fix issues eventually as it already happened previously with Sphinx for what our customers are grateful to us.
For Linux distributions we continue to distribute the software using official packages as DEB and RPM. We know how important is this for devops. We currently support all Long Term Support versions of Ubuntu, Debian, CentOS and RedHat and provide packages for OSX and Windows as well.
As Docker became last years a popular choice for distributing software, we offer an official Docker image.
Somehow big change, yet transparent for most users, was switching from ‘configure’ tool to Cmake. Cmake is great for us. It can be configured to download dependencies (like RE2) library and create different build types. Compiling in Windows is a cumbersome task, but not anymore — you just need to have Visual Studio and CMake will take care of everything.
Manticore Search versioning
We use a semantic version system as x.y.z which means:
- Maintenance release increases z number
- Release with new big features or upgrades to the API increases y number
- Release that breaks index compatibility increases x number
Regarding how Manticore is distributed is simple: since it’s a fork of a GPLv2 project, it can only be distributed under GPLv2 license. Manticore Search is 100% open-source, we love it and we currently don’t see any reason to make it not open source as it happened with Sphinx Search.