ml

Reference API related to ml applications

ML tasks involving text inputs

source

Task

 Task (model_id:str)

Base class for ML Tasks.

Type Details
model_id str Id used to identify the model on Vespa applications.

source

TextTask

 TextTask (model_id:str, model:str, tokenizer:Optional[str]=None,
           output_file:<class'IO'>=<_io.StringIO object at
           0x7fb07610f550>)

Base class for Tasks involving text inputs.

Type Default Details
model_id str Id used to identify the model on Vespa applications.
model str Id of the model as used by the model hub.
tokenizer typing.Optional[str] None Id of the tokenizer as used by the model hub.
output_file IO <_io.StringIO object at 0x7fb07610f550> Output file to write output messages.

source

TextTask.export_to_onnx

 TextTask.export_to_onnx (output_path:str)

Export a model to ONNX

Type Details
output_path str Relative output path for the onnx model, should end in ‘.onnx’
Returns None

source

TextTask.predict

 TextTask.predict (text:str)

Predict using a local instance of the model

Type Details
text str text input for the task.
Returns typing.List Predictions.

source

SequenceClassification

 SequenceClassification (model_id:str, model:str,
                         tokenizer:Optional[str]=None,
                         output_file:<class'IO'>=<_io.StringIO object at
                         0x7fb0124a5790>)

Sequence Classification task.

It takes a text input and returns an array of floats depending on which model is used to solve the task.

Type Default Details
model_id str Id used to identify the model on Vespa applications.
model str Id of the model as used by the model hub. Alternatively, it can also be the path to the folder containing the model files, as long as the model config is also there.
tokenizer typing.Optional[str] None Id of the tokenizer as used by the model hub. Alternatively, it can also be the path to the folder containing the tokenizer files, as long as the model config is also there.
output_file IO <_io.StringIO object at 0x7fb0124a5790> Output file to write output messages.

Model config for Vespa applications

source

ModelConfig

 ModelConfig (model_id)

Base model configuration for Vespa applications.

Type Details
model_id Unique model id to represent the model within a Vespa application.
Returns None

source

BertModelConfig

 BertModelConfig (model_id:str, query_input_size:int, doc_input_size:int,
                  tokenizer:Union[str,os.PathLike],
                  model:Union[str,os.PathLike,NoneType]=None)

BERT model configuration for Vespa applications.

bert_config = BertModelConfig( … model_id=“pretrained_bert_tiny”, … query_input_size=32, … doc_input_size=96, … tokenizer=“google/bert_uncased_L-2_H-128_A-2”, … model=“google/bert_uncased_L-2_H-128_A-2”, … ) # doctest: +SKIP BertModelConfig(‘pretrained_bert_tiny’, 32, 96, ‘google/bert_uncased_L-2_H-128_A-2’, ‘google/bert_uncased_L-2_H-128_A-2’)

Type Default Details
model_id str Unique model id to represent the model within a Vespa application.
query_input_size int The size of the input vector dedicated to the query text.
doc_input_size int The size of the input vector dedicated to the document text.
tokenizer typing.Union[str, os.PathLike] The name or a path to a saved BERT model tokenizer from the transformers library.
model typing.Union[str, os.PathLike, NoneType] None The name or a path to a saved model that is compatible with the tokenizer. The model is optional at construction since you might want to train it first. You must add a model via :func:add_model before deploying a Vespa application that uses this class.
Returns None 
#bert_config = BertModelConfig(
#    model_id="pretrained_bert_tiny",
#    query_input_size=32,
#    doc_input_size=96,
#    tokenizer="google/bert_uncased_L-2_H-128_A-2"
#)
#bert_config = BertModelConfig(
#    model_id="pretrained_bert_tiny",
#    query_input_size=32,
#    doc_input_size=96,
#    tokenizer="google/bert_uncased_L-2_H-128_A-2",
#    model="google/bert_uncased_L-2_H-128_A-2",
#)

source

BertModelConfig.predict

 BertModelConfig.predict (queries, docs)

Predict (forward pass) given queries and docs texts

Type Details
queries A List of query texts.
docs A List of document texts.
Returns typing.List Logits

source

BertModelConfig.add_model

 BertModelConfig.add_model (model:Union[str,os.PathLike])

Add a BERT model

Type Details
model typing.Union[str, os.PathLike] The name or a path to a saved model that is compatible with the tokenizer.
Returns None

source

BertModelConfig.doc_fields

 BertModelConfig.doc_fields (text:str)

Generate document fields related to the model that needs to be fed to Vespa.

Type Details
text str The text related to the document to be used as input to the bert model
Returns typing.Dict Dict with key and values as expected by Vespa.

source

BertModelConfig.query_tensor_mapping

 BertModelConfig.query_tensor_mapping (text:str)

Maps query text to a tensor expected by Vespa at run time.

Type Details
text str Query text to be used as input to the BERT model.
Returns typing.List[float] Input ids expected by Vespa.

source

BertModelConfig.create_encodings

 BertModelConfig.create_encodings (queries:List[str], docs:List[str],
                                   return_tensors=False)

Create BERT model encodings.

Create BERT encodings following the same pattern used during Vespa serving. Useful to generate training data and ensuring training and serving compatibility.

Type Default Details
queries typing.List[str] Query texts.
docs typing.List[str] Document texts.
return_tensors bool False Return tensors
Returns typing.Dict Dict containing input_ids, token_type_ids and attention_mask encodings.

source

BertModelConfig.export_to_onnx

 BertModelConfig.export_to_onnx (output_path:str)

Export a model to ONNX

Type Details
output_path str Relative output path for the onnx model, should end in ‘.onnx’
Returns None

source

BertModelConfig.onnx_model

 BertModelConfig.onnx_model ()

source

BertModelConfig.query_profile_type_fields

 BertModelConfig.query_profile_type_fields ()

source

BertModelConfig.document_fields

 BertModelConfig.document_fields (document_field_indexing)

source

BertModelConfig.rank_profile

 BertModelConfig.rank_profile (include_model_summary_features, **kwargs)

Model Server

source

ModelServer

 ModelServer (name:str, tasks:Optional[List[__main__.Task]]=None)

Create a Vespa stateless model evaluation server.

A Vespa stateless model evaluation server is a simplified Vespa application without content clusters.

Type Default Details
name str Application name.
tasks typing.Optional[typing.List[main.Task]] None List of tasks to be served.

Add ranking model

source

add_ranking_model

 add_ranking_model (app_package:vespa.package.ApplicationPackage,
                    model_config:__main__.ModelConfig, schema=None,
                    include_model_summary_features=False,
                    document_field_indexing=None, **kwargs)

Add ranking profile based on a specific model config.

Type Default Details
app_package ApplicationPackage Application package to include ranking model
model_config ModelConfig Model config instance specifying the model to be used on the RankProfile.
schema NoneType None Name of the schema to add model ranking to.
include_model_summary_features bool False True to include model specific summary features, such as inputs and outputs that are useful for debugging. Default to False as this requires an extra model evaluation when fetching summary features.
document_field_indexing NoneType None List of indexing attributes for the document fields required by the ranking model.
kwargs
Returns None Further arguments to be passed to RankProfile.