LLMMessage#
- class council.llm.LLMMessage(role: LLMMessageRole, content: str, name: str | None = None, data: Sequence[LLMMessageData] | None = None)[source]#
Bases:
object
Represents chat messages. Used in the payload
- Parameters:
role (LLMMessageRole) – the role/persona the message is coming from. Could be either user, system or assistant
content (str) – the message content
name (str) – name of the author of this message
data (Sequence[LLMMessageData]) – the data associated with this message
- __init__(role: LLMMessageRole, content: str, name: str | None = None, data: Sequence[LLMMessageData] | None = None) None [source]#
Initialize a new instance of LLMMessage
- static system_message(content: str, name: str | None = None, data: Sequence[LLMMessageData] | None = None) LLMMessage [source]#
Create a new system message instance
- Parameters:
content (str) – the message content
name (str) – name of the author of this message
data (Sequence[LLMMessageData]) – list of data associated with this message
- static user_message(content: str, name: str | None = None, data: Sequence[LLMMessageData] | None = None) LLMMessage [source]#
Create a new user message instance
- Parameters:
content (str) – the message content
name (str) – name of the author of this message
data (Sequence[LLMMessageData]) – list of data associated with this message
- static assistant_message(content: str, name: str | None = None) LLMMessage [source]#
Create a new assistant message instance
- Parameters:
content (str) – the message content
name (str) – name of the author of this message
- property data: Sequence[LLMMessageData]#
Get the list of data associated with this message
- add_data(data: LLMMessageData) None [source]#
Add data to the message.
- add_content(*, path: str | None = None, url: str | None = None) None [source]#
Add content to the message.
- property content: str#
Retrieve the content of this instance
- property name: str | None#
Retrieve the name authoring the content of this instance
- property role: LLMMessageRole#
Retrieve the role of this instance
- property has_data: bool#
Check if this message has data associated with it
- is_of_role(role: LLMMessageRole) bool [source]#
Check the role of this instance
- static from_chat_message(chat_message: ChatMessage) LLMMessage | None [source]#
Convert
ChatMessage
intoLLMMessage
LLMMessageData#
- class council.llm.llm_message.LLMMessageData(content: str, mime_type: str)[source]#
Bases:
object
Represents the data of a message.
- classmethod from_file(path: str) LLMMessageData [source]#
Add data from file to the message.
- classmethod from_uri(uri: str) LLMMessageData [source]#
Add an uri to the message.
LLMCacheControlData#
- class council.llm.llm_message.LLMCacheControlData(content: str)[source]#
Bases:
LLMMessageData
Data class to hold cache control information for Anthropic prompt caching.
- static ephemeral() LLMCacheControlData [source]#
Returns ephemeral cache type
Code Example#
Here’s how you can use Anthropic prompt caching with council.
import os
# !pip install council-ai==0.0.24
from council.llm import AnthropicLLM
from council.llm.llm_message import LLMMessage, LLMCacheControlData
from council.contexts import LLMContext
os.environ["ANTHROPIC_API_KEY"] = "sk-YOUR-KEY-HERE"
os.environ["ANTHROPIC_LLM_MODEL"] = "claude-3-haiku-20240307"
# Ensure that the number of tokens in a cacheable message exceeding
# the minimum cacheable token count, which is 2048 for Haiku;
# otherwise, the message will not be cached.
HUGE_STATIC_SYSTEM_PROMPT = ""
# Create a system message with ephemeral caching
system_message_with_cache = LLMMessage.system_message(
HUGE_STATIC_SYSTEM_PROMPT,
data=[LLMCacheControlData.ephemeral()]
)
# Initialize the messages list with cachable system message
messages = [
system_message_with_cache,
LLMMessage.user_message("What are benefits of using caching?")
]
llm = AnthropicLLM.from_env()
result = llm.post_chat_request(LLMContext.empty(), messages)
print(result.first_choice)
print(result.raw_response["usage"])