Configure Generative Models in QMS

To use QMS to generate summaries, you must configure at least one generative model in the QMS configuration file.

You can configure QMS to use the following methods to generate embeddings:

  • a generative LLM. You create a generative LLM model by using the supplied script to convert an available third-party model into a format that QMS can use.

  • a Lua script model. You write a script that generates the summaries, for example by using a HTTP endpoint.

Create a Generative Model

The generative models use LLMs from Hugging Face. The QMS installation includes a Python script that you can use to generate model files in a format that QMS can use to generate your summaries.

You can choose generative t5 models. The script provides a list of suggested models.

The script export_transformers_model.py is installed in your QMS installation vector-generator/tools directory. This directory also includes a requirements.txt file to allow you to install the necessary dependencies for the script.

To create your model

  1. Install the requirements for the export_transformers_model.py script by using pip with the requirements.txt file. For example:

    pip install -r requirements.txt
  2. Run the export_transformers_model.py script with the following arguments:

    model

    The model to download from Hugging Face.

    model-type The type of model to create. For generative summaries, set this argument to generative.

    You can also optionally set the following arguments:

    output The file name to use for the generated model file. The default value is model.pt.
    output-spiece The file name to use for the sentencepiece tokenizer file. The default value is spiece.model.
    cache The location for the cache for model downloads. The default value is .cache.

    When the script finishes, it outputs the name and location of the model and tokenizer files that it creates. You use these values in your QMS configuration (see ModelPath and TokenizerPath).

Create a Lua Script Model

You can create your own Lua script model, using any method that you choose to generate the embedding data. You can use the Lua model to interact or access third party API summary generation.

The script must define a function called summarize. This function must accept a single parameter, which is a string representing the text to generate summaries for. It must return the output summary as a string.

For example:

function summarize(text)
   return string.format("Here is your summary: %s, text")
end

Configure QMS with the Generative Model

You configure embeddings in the [Generative] section of the QMS configuration file. See Generative Configuration Parameters.

To configure an embedding model

  1. Open your configuration file in a text editor.

  2. Find the [Generative] section, or create one if it does not exist.

  3. Add an entry for your embedding model by using a zero-based index number. Set this value to the name of the configuration section to use to define your model. For example:

    [Generative]
    0=Summarizer
  4. Create a configuration section with the name that you defined.

    [Summarizer]
  5. Set the Type parameter to the type of model you want to use. For example:

    [Summarizer]
    Type=GenerativeLLM
  6. Set additional parameters for your model. The required parameters depend on the type of model:

  7. Save and close the configuration file.