Auto completion length

[Vux]

I'm running in to issues with the autocomplete not displaying queries that are longer than 43 characters (including spaces).

We are deriving our spelling suggestions from a single metadata field and using the query completion source as spelling suggestions.

If we add longer terms into the spelling-whitelist.cfg they appear in the autocomplete dropdown but cut off after hitting 43 characters (again, including spaces).

Any ideas on how we can increase the character limit to pick up longer metadata values?

[gordongrace]

Hi @Vux -

It sounds like you're hitting a default limit on word length (20 characters), controlled by Indexer Options:

https://docs.funnelback.com/15.10/collections/collection-configuration/indexer_options_collection_cfg.html#c-controlling-how-things-are-indexed

Note particularly:
- SORTSIG
- dilw

Is this truncation also appearing in the raw output of suggest.json? Are you seeing truncation appearing on multi-term suggestions as well as single-term suggestions?

[Vux]

Hi Gordon,

Thanks for the suggestion. We increased the SORTSIG but this did not resolve our issue.

The truncation is also appearing in the raw suggest.json and when clicked on searches the truncated query. For example, "education and something that makes this really long" would cut off at "education and something that makes this rea" and the query would be "education and something that makes this rea"

However this only occurs for terms in the spelling-whitelist.cfg, longer terms that should have been picked up from the crawler don't even show up.

We have noted that in the Step-AnnieAPrimaryCollection.log there is a line that states ++Param: Maximum annnotation length: 42 but not sure if this is related, nor how to modify it?

[gordongrace]

We have noted that in the Step-AnnieAPrimaryCollection.log there is a line that states ++Param: Maximum annnotation length

Further down in that same log file, you'll also note:

++Rejections of annotations which were too short or too long:

How many rejections have been occurring?

[gordongrace]

Indexing also triggers an annotations step - a list of annotation options are described at:

https://docs.funnelback.com/15.10/more/extra/padre_usage.html#4-annie-a

You can influence these at index-time using the collection.cfg value:

https://docs.funnelback.com/more/extra/annie_index_opts_collection_cfg.html

Example:
annie.index_opts=-maxwds 10 -maxlen 60

[Vux]

Thanks for the response again Gordon. None of the annotations were rejected, however further down there was the following:

+++ Param: Maximum postings length for annotations: 970
+++ Param: After annotation postings list reaches max, fraction to be purged: 0.3333

Could this be what is culling it out?

I just tried adding in those config changes to the collection.cfg however it doesn't seem to be working still.

[gordongrace]

Could you investigate the output from Step-AnnieAPrimaryCollection.log? It'll confirm whether the settings were applied as expected, and highlight any issues encountered.

[Vux]

The settings appear to have been applied, here is the excerpt from the logs after a recrawl with the new config settings:
++Param: Maximum annnotation length: 60
++Param: Maximum no. words in an annotation: 10
++Number of annotations which were truncated: 0

Additionally there is a line up the top:
Command line: /opt/funnelback/bin/annie-a /opt/funnelback/data/open-access-web/one/idx/index -build_lcache=off -build_annou=on -maxwds 10 -maxlen 60

I have tried increasing the annotation length more as the entry in question actually contains 61 characters including spaces, however that didn't work either.