In large systems that experience heavy load, you can optimize the performance of AgentBoolean queries by creating optimized agents or categories. This optimization is most useful when you use IDOL Server to:
Return agents that match a non-indexed document, for example to alert users to new content that matches their agents.
Return categories that match a non-indexed document, for example to categorize a document and add the category information to a field before indexing.
When IDOL Server matches query text against agents or categories, it uses the following matching order:
It matches the query text against the Index
fields in the agents or categories (for example the TRAINING
or DRECONTENT
fields).
For agents that match in Step 1, it matches the query text against any Boolean restrictions in the agents or categories.
For agents that match in Step 2, it matches the query text against any FieldText
restrictions in the agents or categories.
IDOL Server checks the Boolean restriction only if the agent or category content matches the document.
To optimize performance, choose your category or agent content carefully to ensure that it matches only query text that is also likely to match the Boolean expression.
For many Boolean restrictions, you can select one or two terms that queries must contain to match the Boolean expression. If you select the rarest of these combinations, fewer documents match the agent content. This process reduces the number of Boolean expressions that IDOL Server must match.
For example, if your Boolean restriction is:
“New Zealand” AND “French Champagne”
you can set the agent content to:
Zealand
In this Boolean expression, the query text must contain all four terms, so you can choose any of them as the agent content. Zealand
is the rarest term, so fewer documents match this agent.
For other expressions, you can often similarly choose a minimal list of terms to reduce the number of documents that match against the Boolean restriction.
This action returns a minimal list of terms that text must contain to match the Boolean expression. It provides the rarest terms where possible. The action returns the following information:
<autnresponse> <action>DOCUMENTSTATS</action> <response>SUCCESS</response> <responsedata> <autn:wildrequired>false</autn:wildrequired> <autn:numberrequired>1</autn:numberrequired> <autn:required>ZEALAND</autn:required> </responsedata> </autnresponse>
Using this method to choose the agent or category content can improve performance, especially in systems that receive a large number of queries.
For some Boolean expressions that include Wildcards and DREFUZZY
or SOUNDEX
expressions, the minimal list of terms must include a Wildcard value.
For these expressions, the DocumentStats
action with QueryStats
set to True
returns the following line:
<autn:wildrequired>true</autn:wildrequired>
The agent or category content cannot contain a Wildcard value, so you must use a different approach to ensure that IDOL Server checks the Boolean expression.
You can ensure that IDOL Server checks any documents that contain one of these expressions by configuring an AlwaysMatchType
field in the Agentstore component and adding it to the agents.
IDOL Server always matches the content of an agent or a category that contains an AlwaysMatchType
field. It then attempts to match the query text against the Boolean restriction, and returns any agents or categories that match at this stage.
Open the IDX or XML files that contain your AgentBoolean documents.
Decide which fields you want to always match. For a document to always match, the AlwaysMatchType
field must have a non-empty string.
Note: For IDOL Server categories, you must use the NOCONTENTCAT
field.
Open the Agentstore configuration file in a text editor.
In the [FieldProcessing]
section, add a new process to the list of processes. This process identifies the fields in the IDX or XML document that you want to always match. For example:
[FieldProcessing] 0=SetIndexFields ... 15=SetAlwaysMatchFields
Create a new section for the process that you added. In this section, create a property for the process (you define the property, later by setting one or more applicable configuration parameters). Set PropertyFieldCSVs
to a list of fields to associate with the process. For example:
[SetAgentBooleanFields] Property=AlwaysMatchFields PropertyFieldCSVs=*/MARKERFIELD,*/NOCONTENTCAT
Create a new section for the property that you added, in which you set AlwaysMatchType
to True
. This property indicates that IDOL Server must always match the associated fields in queries. For example:
[AlwaysMatchFields] AlwaysMatchType=True
Save and close the configuration file. Restart IDOL Server for your changes to take effect.
After you configure the AlwaysMatchType
field, you must manually create agents as IDX or XML documents.
For each agent, send the Boolean restriction in a DocumentStats
action to IDOL Server to return the minimal list of terms.
If the result does not contain a Wildcard value, you can create agents in the normal way.
If the result contains a Wildcard, add the AlwaysMatchType
field to the agent IDX. For example:
#DREREFERENCE WineAgent #DRETITLE My Wine Agent #DREFIELD BOOLEANRESTRICTION=""Champ*" AND "Merlot"" #DREFIELD MARKERFIELD="1" #DRECONTENT #DREENDDOC
You can index your agent IDX or XML document into the Agentstore component agent database by using a DREADD
or DREADDDATA
index action. For example:
http://localhost:9051/DREADD?C:\data\Agents.idx&DREDBName=Agent
|