Configure Embeddings in QMS

To use QMS to generate embeddings, you must configure at least one embedding model in the QMS configuration file.

You can configure QMS to use the following methods to generate embeddings:

  • a sentence transformer model. You create a sentence transformer model by using the supplied script to convert an available third-party model into a format that QMS can use.

  • a Lua script. You write a script that generates the embeddings, by whatever method you want to use.

Create a Transformer Model

The sentence transformer models use vector models from Hugging Face. The QMS installation includes a Python script that you can use to generate model files in a format that QMS can use to generate your embeddings.

You can choose any sentence-transformer sentencepiece model.

TIP: To find models appropriate for embedding generation, you can use the hugging face tags for sentence similarity. For example: https://huggingface.co/models?pipeline_tag=sentence-similarity&sort=trending

The script export_transformers_model.py is installed in your QMS installation vector-generator/tools directory. This directory also includes a requirements.txt file to allow you to install the necessary dependencies for the script.

To create your model

  1. Install the requirements for the export_transformers_model.py script by using pip with the requirements.txt file. For example:

    pip install -r requirements.txt
  2. Run the export_transformers_model.py script with the following arguments:

    model

    The model to download from Hugging Face.

    model-type The type of model to create. For embedding generation, set this argument to sentence-tranformer.

    You can also optionally set the following arguments:

    output The file name to use for the generated model file. The default value is model.pt.
    output-spiece The file name to use for the sentencepiece tokenizer file. The default value is spiece.model.
    cache The location for the cache for model downloads. The default value is .cache.

    When the script finishes, it outputs the name and location of the model and tokenizer files that it creates. You use these values in your QMS configuration (see ModelPath and TokenizerPath). For example:

    ModelPath: sentence-t5-large.pt
    TokenizerPath: spiece.model

Create a Lua Script Model

You can create your own Lua script model, using any method that you choose to generate the embedding data. You can use the Lua model to interact or access third party API embedding generation, such as the Hugging Face Inference API, or Inference Endpoints service.

The script must define a function called generateembeddings. This function must accept a single parameter, which is a string representing the text to generate embeddings for.

The function must return a table of tables, where each inner table contains floating point numbers. Each inner table therefore corresponds to a vector for that text.

You can optionally include a second table of tables, which returns offsets for the embeddings in the format:

{{embedding_1_start_offset, embedding_1_end_offset}, {embedding_2_start_offset, embedding_2_end_offset}, {embedding_3_start_offset, embedding_3_end_offset}, ...},

NOTE: If you want to use offset information in IDOL (for indexing or querying), you must use UTF-8 byte offsets.

For example:

function generateembeddings(text)
   return {{1,2,3,4}, {5,6,7,8}, {9,10,11,12}}, {{34,48}, {48, 124}, {124, 156}}
end

In this example, the function returned three embeddings in the first table (with three nested tables). The second table is the offset values. The vector {1,2,3,4} starts at offset 34 and ends at 48. The second vector {5,6,7,8} starts at 48 and ends at 124, and so on.

Configure QMS with the Embeddings Model

You configure embeddings in the [Embeddings] section of the QMS configuration file. See Embeddings Configuration Parameters.

To configure an embedding model

  1. Open your configuration file in a text editor.

  2. Find the [Embeddings] section, or create one if it does not exist.

  3. Add an entry for your embedding model by using a zero-based index number. Set this value to the name of the configuration section to use to define your model. For example:

    [Embeddings]
    0=SentenceTransformer
  4. Create a configuration section with the name that you defined.

    [SentenceTransformer]
  5. Set the Type parameter to the type of model you want to use. For example:

    [SentenceTransformer]
    Type=Transformer
  6. Set additional parameters for your model. The required parameters depend on the type of model:

  7. Save and close the configuration file.