Write to Search Index
The Write to Search Index stage enables you to create a full-text index based on the data coming into the stage. Having this data in a dedicated search index results in quicker response time when you conduct searches against the index from other Spectrum Technology Platform stages.
Full-text-search indexes are preferable to relational databases when you have a great deal of free-form text data that needs to be searched or categorized or if you support a high volume of interactive, text based queries.
- Create an Index: In the Index field, select the
Create Index
button to create an index. For more information, see Creating an Index. - View the schema of the fields: The Fields section allows you to view the schema of the fields for the selected index. You need to configure the source fields for all the fields listed as necessary.
- Apply Write to Search Index: Click Apply at the top-right corner of the page to apply the configuration changes to the write to search index stage in the dataflow. For more information, see Applying Configuration Changes to the DataFlow.
How Write to Search Index functions
Write to Search Index uses analyzers to break input text into small indexing elements called tokens. It then extracts search index terms from those tokens. The type of analyzer used—how input text is broken into tokens—determines how you will then be able to search for that text.
For example, the Keyword analyzer always does an exact match and tokenizes the whole string as a single token, while the Standard analyzer breaks the string to create tokens. The Stop Word analyzer is more sophisticated and removes articles, such as "a" or "the" from the string before tokenizing, thus shrinking the index size. Search indexes support the near real-time feature, allowing indexes to be updated almost immediately, without the need to close and rebuild the stage using the search index.