Json endpoint - retrieving all metadata field names/values

Developers Search Interface
1

Hi all,

I need to create a custom advanced search page and was going to do this outside of the freemarker templates (although if its possible in there then great!).

The advanced search would take the “query constructor” format, where users can pick fields to search with certain fields being from a pre-defined dropdown list. A bit like this: https://cdn.dribbble.com/users/1798715/screenshots/6066303/advanced_search_4x.jpg?compress=1&resize=1000x750.

My questions are:

1.How do I get out a list of all metadata fields/classes I have in funnelback - preferably via the json enpoint or similar. I don’t want to add them all as facets have to query all records to return them via json.
2.How would I then perform a query to retrieve all values of a certain field so I could pre-populate a dropdown in certain cases?

I could probably hard-code number 1 above, but would really prefer not to..

EDIT - found this , but is there any easier way? Redirect Notice

Thanks!

1 Like
2

I hope that I’ve understood your question and answered it sensible way. FWIW I found Freemarker pretty good. It takes a bit of work if (like me) you don’t work in Java. There is already advanced search in the default template so it might be worth taking another look.

In response to your question 1 about which fields are exposed. The list of fields for a collection is defined in collection.cfg using the query processor options. The -SF=[c,d,t,…] option exposes fields in the results.

For question 2, the results JSON will return the facet keys and the number of values. It’s instructive to take the default search results and replace the .html with .json:
https://example.ac.uk/s/search.html?collection=my-collection&profile=_default&query=!showall
to
https://example.ac.uk/s/search.json?collection=my-collection&profile=_default&query=!showall

If you’ve defined facets and the exposed metadata then you’ll find the values in the JSON response.

3

That’s great thankyou!

I’m trying to work through this:

For the first (less important) query, I already have SF=[.*], where would these appear in the json endpoint? I can see the ListMetadata array for results, but I don’t want to have to traverse all results to build a list of the available metadata classes? Example of a result set is here: https://collections.st-andrews.ac.uk/search/?form=grid&mode=query&query=monk&skin=0&json=1 . I’m probably going to hardwire these anyway so this is probably not that important.

For the second query - I didn’t want to create facets for every single metadata class (there’s 80!), and was hoping for a way to do a json endpoint query to retrieve a list of all available metadata options for a given class, so I can dynamically populate a select2-style dropdown box. I wanted to avoid having to do this: Redirect Notice

Thanks again!

4

The SF=[.*] populates the listMetadata for each result record: response > resultPacket > results > listMetadata
viz:

"response": {
  "resultPacket": {
    "results": [{
      "rank": 1,
      "score": 1000,
      ....
      "listMetadata": {
        "MultimediaWidth": ["800", "800"],
        "IRN": ["1000516"],
        "OriginalDateLatest": ["1904"],
        "SimpleName": ["sculpture"],
        "MultimediaHeight": ["1200", "1200"],

The list of all the metadata values appears in response > resultPacket > results > rmcs

viz:

"response": {
  "resultPacket": {
      "rmcs": {
          "Format:negatives (photographs)": 7,
          "ArtistFullName:Cuthbert Glass": 1,
          "ArtistFullName:Mr George A. Little": 3,
          "LocationContinent:Europe": 19,

There is more about the datamodel in the ModernUI documentation

5

Correct - rmcs will hold the counts for every metadata value.

If you’re using this to populate a dropdown I would recommend creating a separate profile to use for these queries that sets a number of query processor options in the padre_opts.cfg to run an optimal query (e.g. setting the number of results to return to 1, turning off lots of features like the ranking algorithm - see: Redirect Notice for some suggestions of what can be set). This will ensure the query that needs to run to populate the form is as low impact as possible. It will also mean that these queries don’t find their way into your search analytics.

6

Hi both

Thank you for the help so far! I didn’t spot rmcs which I’m off to do some testing with to see if it suits! If it does, I think the plan would be to write a small script that would do a showall/nullquery to the service overnight, then parse the json response into a nicer json file that can be used during the day to auto-populate the dropdown boxes.

Thanks again!