This option enables you to assess the performance and accuracy of an Eduction grammar against a set of pre-tagged examples.
You must supply a text file with one phrase on each line; the Assess
feature checks whether each line contains a match.
You must specify at least one input file, using the -v
parameter or the -w
parameter. If required, you can specify both of these parameters.
You can use wildcard expressions in the -e
and -g
parameters; see
Wildcard Expressions in edktool for more information.
-l <licensefile>
|
The file containing a valid license key for Eduction. If you do not specify a license key at the command line, |
-c <configfile>
|
A configuration file controlling the assessment. The configuration file can be either an IDOL Server style .CFG configuration file or an XML configuration file. See Configuration Files for Eduction Settings. You can specify one or more grammar files and one or more entities in place of a configuration file. Specifying a configuration file overrides the grammar or entity parameters. |
-g <grammarfile>
|
A grammar file to use when If you provide a grammar file but you do not specify any entities with |
-e <entity>
|
The entities to extract when |
-x
|
(Optional) Modifies the behavior so that the Assess feature checks for exact matches. |
-m <matched entities>
|
(Optional) This parameter does not change the extraction behavior, but enables you to check which entities are producing the matches. |
-v <valid_input>
|
A file of phrases where a match would be valid. |
-w <invalid_input>
|
A file of phrases where a match would be invalid. |
-a
|
(Optional) The output includes explanations of each failure, and statistics such as recall, precision, and F1 (depending on the type of input file you provided). Include the -a parameter to display additional output, including the results for every phrase in your input files. |
-o <outputfile>
|
(Optional) By default, Eduction sends output to the console. To send the output to a file, use the The output is a list of all phrases that failed. For valid input this would be a phrase that contained no match; for invalid input this would be a phrase that contained a match. |
-q
|
(Optional) Sets “Quiet Mode” so that descriptive messages are removed, and the output consists only of a list of examples that failed, in the form "FAIL: "text" is matched by "entity"" or similar, depending on the test specifications. If you also set the -a parameter, examples that pass are also included in the output.
|
For more information on how to use the Assess feature to check the effectiveness and performance of your grammar files, refer to IDOL Expert.
edktool a -l <license> -c <configuration_file> [-a] [-o <output_file>]
Run several assessments from a single Eduction configuration file.
The configuration file must contain a numbered [assessmentN]
section for each assessment you want to run. You must specify the input files, the entities to match, and whether matching should be exact. For example:
[assessment0] valid=data.txt
[assessment1] entities=entity1,entity2 valid=match.txt invalid=should_not_match.txt exact=true
You can specify multiple entities either by separating them with commas, or by using wildcard expressions. You can use the * wildcard to match any number of characters, or the ? wildcard to match a single character. For example, set Entities
to org/soccer/*
to use the entities org/soccer/us
, org/soccer/gb
, org/soccer/de
, and so on without having to type a lengthy comma-separated list.
|