Avalanche

Avalanche, a collection of Snowballs.

Avalanche is a collection of stemmers (currently from the Snowball project, but not technically limited to those) that help improve Drupal 8 database search by reducing words in searches into their stems. More stemmers can be added as plugins.

Improvements to the base Snowball stemmers will also be considered, though these require new tests and test data.

Supported languages

The following languages are currently supported natively with custom-made stemmers. These stemmers are a work in progress, and more will be added in the future.

• Finnish
• Swedish

PECL stem package

If the PECL stem package is installed, any languages that it supports are also supported by Avalanche, though their configurability is limited. As of version 1.5.1, the PECL stemmer can support any combination of languages from the following list:

• Danish
• Dutch
• English
• Finnish
• French
• German
• Hungarian
• Italian
• Norwegian
• Portuguese
• Romanian
• Russian
• Spanish
• Swedish
• Turkish

To install the PECL stem package, run pecl install stem as root. You will be able to choose which stemmers to enable. If the installation fails with a compile error and you are running PHP 7, try the following:

1. Download the stemmer tarball and unpack it.
2. Patch stem.c with this patch.
3. Check the md5sum of the patched stem.c and update it to package.xml.
4. Recreate the stem-1.5.1.tgz archive (tar -czvf stem-1.5.1.tgz stem-1.5.1/ package.xml)
5. Install the package with pecl install stem-1.5.1.tgz.

Future plans

• Add support for more than one stemmer per language. done
• Add support for all current stemmer configuration options (tokenization, stopwords).
• Add support for Search API.
• Add Snowball stemmers for Norwegian and Danish.
• Add Snowball stemmer for German.
• Add Snowball stemmer for English.
• Add other stemmers.

Current issues

• Core search currently removes all diacritical marks from search input text before hook_search_preprocess implementations see it. This prevents some stemming steps that rely on their existence from working properly (e.g. step 3 of the Swedish and Danish stemmers, steps 1 and 2 of the Italian stemmer and pretty much the entire Spanish and French stemmers). See issue 2858337.
  • This is circumvented by default by overriding the core transliteration service with a near-identical version that only disables the function that removes the diacritical marks. Whether this is needed depends on the languages in use. It's not critical for English, but if your alphabet includes letters with umlauts or other diacritical marks, there's a high chance you'll need keep the override enabled. You can disable this service override on the Avalanche settings page.
• The configuration interface is not yet fully implemented and is subject to change. Some options, while configurable, are not yet respected by the stemmers.