Introduction to Document Storage

In earlier versions, Manticore Search would store only the indexed data for the full-text fields. In 3.2 a docstore engine was added that allowed to also store the original content of full-text fields.

In 3.4 the full-text fields can also be just stored, without being indexed.

Let's take a demo table having full-text fields in different storing modes:

mysql -P9306 -h0

CREATE TABLE testrt ( defaultTitle text, indexedTitle text indexed,storedonlyTitle text stored );

In the CREATE TABLE syntax, full-text fields have 'text' type. If no other option is specified, the field will be indexed and stored by default.

Inserting documents

Let's add a text to each field and see how we can search them and how they are returned in results.

INSERT INTO testrt (id,defaultTitle) VALUES (1,'Samsung A50');

INSERT INTO testrt (id,indexedTitle) VALUES (2,'Samsung A50');

INSERT INTO testrt (id,storedonlyTitle) VALUES (3,'Samsung A50');

Executing search queries

let's run a search to see what result we get:

SELECT * FROM testrt WHERE MATCH('samsung');

As we can see, only 2 documents are returned:

For document with id 1, we see the text we inserted, as the defaulttitle field is both indexed and stored.

For document with id 2, we get the document in the result set, but there is no text. This is because we only set the text in 'indexedtitle' field which indexes the text, but it's not stored, therefor doesn't return anything in the result set.

The document with id 3, is not found in the result set because we added the text to the storedonly field, which only stores the text, but not table, so it's not possible to be found when performing a full-text match.

A stored-only field is useful when we have a text that is not used in searching, but we need it in the result set.

INSERT INTO testrt(id,defaulttitle,storedonlytitle) VALUES (4,'Samsung A50', ' Galaxy A series');

Running the search query again:

SELECT * FROM testrt WHERE MATCH('samsung');

Highlighting

Without docstore, highlighting required getting the original text from it's source and adding it in the query to return back highlighted text.

If the field has the document stored, we can use HIGHLIGHT() to retrieve highlighted snippets:

SELECT HIGHLIGHT() FROM testrt WHERE MATCH('samsung')\G