Query multiple date fields

[Dhiraj]

We have a requirement to query funnelback using multiple date fields, wherein there will be a start date field and an end date field to fetch the current live event XMLs. When we try to use the dates as number fields we are not able to get response using numeric operators in the query. Is there any way to achieve the multiple date query apart from the events functionality in funnelback.

Thanks

[plevan]

Funnelback currently only supports a single 'date' type metadata field. This 'date' field is treated specially for ranking and also has dedicated date sorting, faceting and query parameters.

If your multiple date requirement is just for search purposes then you should be able to define these metadata fields a numeric ones (as you've suggested you already did). For this to work you need to ensure that the value you index in these fields is stored in a format that will sort alphabetically (e.g. 14 Mar 2020 must be represented with something like 20200314). Funnelback won't convert these automatically - you'll need the dates to be represented that way in the XML or write a custom filter to do the conversion fo you. Numeric fields have their own query CGI parameters that you can make use of https://docs.funnelback.com/15.24/customise/metadata-customisation/metadata-class-types.html#searching-numeric-metadata

You'd also need to ensure that when you run your query that it's restricted to only return fully matching results (you can set the fmo=true query processor option to ensure this happens.

This way if you have two fields startDate and endDate and want things that match a "anniversary" event type (set in the eventType metadata field) between the dates of 1 Jan 2019 and 25 Feb 2019 you could then run a query similar to:

http://example.com/s/search.html?collection=events&query=eventType:anniversary&ge_startDate=20190101&le_endDate=20190225&fmo=true

Note you would normally want to set the fmo option as a query processor option rather than as a CGI parameter.\

I hope that this helps you to achieve what you want.

[Dhiraj]

Many thanks Plevan for the prompt reply and your suggestion worked for me, i am able to execute a date query by converting the date into numeric field as YYYYMMDD. However, out of curiosity, i would like to ask why does funnelback not have Date as custom input type to simplify date-based queries, which is very common for search engine based requirements.

[plevan]

Hi, it's a feature request that comes up from time to time. There's no specific reason as far as I know why we wouldn't want to support arbitrary date types at some point - it's just not that high on the priority list at the moment in terms of the overall product development.

[Dhiraj]

Hi Plevan,

I have the following queries related to the usage of dates in events,

1) Can we have a date along with Timestamp which can be used as a criteria for query?

2) Can the date with timestamp be used to sort the data.

3) Where do we need to change the config to support date format with timestamp

4) For query with events i am unable to get response using the following http://example.com/s/search.html?collection=events&query=eventType:anniversary % O>20190101<20190225. It gives 400 error, unable to parse URI query.

5) Inorder to use Date meta tags supported out of the box, which elements in XML can be used for representing the date tags: dc.date,dc.date.valid,dc.date.valid,dc.date.modified,etc.

6) Is the event supported for xml tag? will the following xml element work as events < dateOn>20200301|20200302|20200303< / dateOn>.

Thanks in advance

[plevan]

1) Can we have a date along with Timestamp which can be used as a criteria for query?

The date field in Funnelback ignores any timestamp information. There is an open ticket about extending support for this. If you require any timestamp information you'll need to use a different type of metadata field (either a numeric one carrying something like an epoch value, or use a text field).

2) Can the date with timestamp be used to sort the data.

If you use something like an epoch date or a date format that text sorts correctly (like YYYY-MM-DD HH:mm:ss) you should be able to sort the date fine. You just can't use the date field explicitly.

3) Where do we need to change the config to support date format with timestamp

As mentioned above there is no support in terms of date metadata for timestamp information. If you use an alternate metadata field for the date it's still a good idea to have the day level date mapped to metadata so you get date information into the ranking algorithm (dates mapped to other metadata fields are not treated as a date).

4) For query with events i am unable to get response using the following http://example.com/s/search.html?collection=events&query=eventType:anniversary % O>20190101<20190225. It gives 400 error, unable to parse URI query.

% is a special character in a URL and needs to be URL encoded.

5) Inorder to use Date meta tags supported out of the box, which elements in XML can be used for representing the date tags: dc.date,dc.date.valid,dc.date.valid,dc.date.modified,etc.

dc.date is probably the safest bet. Other variants can be used but if there are multiple values in different fields there is an order in which Funnelback determines the 'date' to assign to the record when it is indexed. See: https://docs.funnelback.com/customise/metadata-customisation/metadata-class-types.html#metadata-class-types-date and specifically the bit on date precedence order

6) Is the event supported for xml tag? will the following xml element work as events < dateOn>20200301|20200302|20200303< / dateOn>.

Yes as long as you are using events mode when you run the query. You could also specify that in multiple elements

<dateOn>20200301|20200302|20200303</dateOn>

could also be specified with something like

<dateOn>20200301</dateOn>
<dateOn>20200302</dateOn>
<dateOn>20200303</dateOn>

Note: events mode is incompatible with a heap of other Funnelback features so you need to be aware of this if you consider using it. https://community.funnelback.com/knowledge-base/implementation/search-interface/events-search-limitations

[Dhiraj]

Thanks Peter for the update.

I was going through the showcase of funnelback events on the below url: http://showcase.funnelback.com/s/search.html?collection=showcase-events&profile=_default&query=!showallresults&event_start=2017-07-28&event_end=2017-08-04.

Was just navigating one of the url which is of https://www.sydneyoperahouse.com/events/whats-on.html and came accross the day and time of the day functionality on their current site. We are looking to implement something similar to the events based on the date, day and time of the day functionality.

Can you please confirm if https://www.sydneyoperahouse.com events functionality is based on funnelback events and if yes, how can we implement events with the day, time of the day functionality.

Thanks,
Dhiraj

[plevan]

That showcase demo does use events mode for the search.

The custom CGI parameters are converted into the event query using a pre process hook script that maps the custom CGI parameters into the events query:

// Transform 'event_start', 'event_end' and 'event_days' CGI parameters into
// occurrence-style operators for event search, appending to original query.
//
// Refer: http://docs.funnelback.com/event_search.html
if (((transaction.question.inputParameterMap["event_start"]) && (transaction.question.inputParameterMap["event_start"] != null))
   || ((transaction.question.inputParameterMap["event_end"]) && (transaction.question.inputParameterMap["event_end"] != null))
   || ((transaction.question.inputParameterMap["event_days"]) && (transaction.question.inputParameterMap["event_days"] != null))) {

  def occurrenceMetadataField = "O" // Default to O, as per documented example
  def occurrences = " %"
  // Handle one or more single day occurrences
  if (transaction.question.inputParameterMap["event_days"]) {
    event_day_array = transaction.question.inputParameterMap["event_days"].split("\\|")
    if (event_day_array.size() > 0) {
      event_day_array.each {
        occurrences += " " + occurrenceMetadataField + "=" + (it =~ /-/).replaceAll("")
      }
    } else {
      occurrences += " " + occurrenceMetadataField + "=" + (transaction.question.inputParameterMap["event_days"] =~ /-/).replaceAll("")
      }
  } else {
    // Handle date ranges
    occurrences += " " + occurrenceMetadataField
    if (transaction.question.inputParameterMap["event_start"]) {
      occurrences += ">" + (transaction.question.inputParameterMap["event_start"] =~ /-/).replaceAll("")
    }
    if (transaction.question.inputParameterMap["event_end"]) {
      occurrences += "<" + (transaction.question.inputParameterMap["event_end"] =~ /-/).replaceAll("")
    }
  }
  // Add occurrences logic back to original query
  transaction.question.query += occurrences
}

I would urge you to really consider if you need to use events mode. It's a feature that is highly incompatible with other features within Funnelback and is only really useful for very specific use cases.

There are other ways you can search events (e.g. using numeric metadata fields for the start and end dates) that work in a way that doesn't break other functionality. What events mode gives you is the ability for a single search result to appear multiple times in the list of results, and also be grouped by the day.