OllamaLLMConfiguration#

class council.llm.OllamaLLMConfiguration(model: str, keep_alive: float | str | None = None, json_mode: bool = False)[source]#

Bases: LLMConfigurationBase

__init__(model: str, keep_alive: float | str | None = None, json_mode: bool = False) → None[source]#

Initialize a new instance

Parameters:

model (str) – model name to use from https://ollama.com/library
keep_alive (Optional[float | str]) – ollama keep_alive parameter
json_mode (bool) – whenever to use json mode, default False

property model: Parameter[str]#: Ollama model.

property keep_alive: Parameter[str]#: Number of seconds / duration string to keep model in memory. See https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-keep-a-model-loaded-in-memory-or-make-it-unload-immediately

property keep_alive_value: float | str | None#: Convert keep_alive parameter to a format expected by ollama.

property json_mode: Parameter[bool]#: Whenever to return json. Will be converted into ollama format parameter.

property format: Literal['', 'json']#: The format to return a response in.

property mirostat: Parameter[int]#: Enable Mirostat sampling for controlling perplexity.

property mirostat_eta: Parameter[float]#: Learning rate for Mirostat sampling.

property mirostat_tau: Parameter[float]#: Controls balance between coherence and diversity.

property num_ctx: Parameter[int]#: Context window size.

property repeat_last_n: Parameter[int]#: Look back size for repetition prevention.

property repeat_penalty: Parameter[float]#: Penalty for repetition.

property temperature: Parameter[float]#: The temperature of the model.

property seed: Parameter[int]#: Random seed.

property stop: Parameter[str]#: Stop sequence.

property stop_value: List[str] | None#: Format stop parameter. Only single value is supported currently.

property tfs_z: Parameter[float]#: Tail free sampling parameter.

property num_predict: Parameter[int]#: Maximum number of tokens to predict.

property top_k: Parameter[int]#: Only sample from the top K options for each subsequent token. Used to remove “long tail” low probability responses.

property top_p: Parameter[float]#: Use nucleus sampling. In nucleus sampling, we compute the cumulative distribution over all the options for each subsequent token in decreasing probability order and cut it off once it reaches a particular probability specified by top_p.

property min_p: Parameter[float]#: Minimum probability for token consideration

params_to_options() → Dict[str, Any][source]#: Convert parameters to options dict