LLMSimilarityScorer#

classDiagram Monitorable <|-- ScorerBase ScorerBase <|-- LLMSimilarityScorer
class council.scorers.LLMSimilarityScorer(llm: LLMBase, expected: str)[source]#

Bases: ScorerBase

Using an LLM to compute a similarity score between two messages.

__init__(llm: LLMBase, expected: str) → None[source]#

Initialize a new instance

Parameters:

  • llm (LLMBase) – the LLM to be used

  • expected (str) – the expected text message

render_as_dict(include_children: bool = True) → Dict[str, Any]#

returns the graph of operation as a dictionary

render_as_json() → str#

returns the graph of operation as a JSON string

score(context: ScorerContext, message: ChatMessage) → float#

Score the given message

Parameters:
Returns:

similarity score. The greater the value to higher the similarity

Raises:

ScorerException – an unexpected error occurs

to_dict() → Dict[str, Any][source]#

Serialize the instance into a dictionary. May need to be overridden in derived classes