Adding gscopes.cfg in meta colection

gnana 2020-05-21 18:08:41 UTC #1

I adding gscopes.cfg in collection type meta(my top parent collection). Whatever adding in this gscopes.cfg is not working the same content working in when I add in child web collection gscopes.cfg

We could not add gscopes.cfg in meta collection?

gnana 2020-05-21 19:06:24 UTC #2

we have 100 child collection So want to maintain gscopes.cfg in meta collection.So that if we have need to update anything in gscopes.cfg it will be easy and maintain same code all sub collection is very difficult.

plevan 2020-05-21 21:31:38 UTC #3

gscopes need to be applied on the component collection as they are applied as part of the index build. They will have no effect if defined on a meta collection.

It is generally not a good idea to have so many component collections - you lose ranking information when web collections are split up, the can be much harder to manage and also risk having duplicate URLs in the search index. Have you thought about using a web collection with multiple profiles instead?

I also have a feeling that meta collections are limited to a maximum of 64 component collections though I believe you receive an error when you reach the limit.

If you must reference the gscopes config from the meta collection you could copy this in to your component collection using pre-index workflow as part of it's collection update. However note that this may not work in future versions if the way configuration for gscopes is handled changes.

gnana 2020-05-22 13:16:05 UTC #4

Thank you for the response.

The reason we choose multiple web collections instead of multiple profiles, We got the issue in crawling all 100+ domains. It's not crawling all the web documents from 100+ domains. So we create one collection for one domain and it's crawling all web documents in a domain.

Is it possible to crawl by profile instead of a collection.? I know the profile is in meta collection and domain added in the component collection. We want to crawl domain by domain.

We have 100+ domains with 100K+ web documents in web collection. So when we add a new domain in a web collection and re-start crawling this takes more than a day to finish crawling. This case will not work for us. So we start this multi collection methodology.

Are you sure we could not add 64+ component collections?