Welcome to Languru
The general-purpose LLM app stacks deploy AI services quickly and (stupidly) simply.
_
| | __ _ _ __ __ _ _ _ _ __ _ _
| | / _` | '_ \ / _` | | | | '__| | | |
| |__| (_| | | | | (_| | |_| | | | |_| |
|_____\__,_|_| |_|\__, |\__,_|_| \__,_|
|___/
Documentation: Github Pages
Install Languru
pip install languru
# Install For LLM deployment.
pip install languru[all]
# Install development dependencies.
poetry install -E <extras> --with dev
# Or just install all dependencies.
poetry install -E all --with dev --with docs
OpenAI Clients
Supported OpenAI clients:
openai.OpenAI
openai.AzureOpenAI
languru.openai_plugins.clients.anthropic.AnthropicOpenAI
languru.openai_plugins.clients.google.GoogleOpenAI
languru.openai_plugins.clients.groq.GroqOpenAI
languru.openai_plugins.clients.pplx.PerplexityOpenAI
languru.openai_plugins.clients.voyage.VoyageOpenAI
OpenAI Server
languru server run # Remember set all needed `api-key` for OpenAI clients.
Query LLM service, which is fully compatible with OpenAI APIs.
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8682/v1")
res = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello!"},
],
)
for choice in res.choices:
print(f"{choice.message.role}: {choice.message.content}")
# assistant: Hello! How can I assist you today?
Chat streaming:
client = OpenAI(base_url="http://localhost:8682/v1")
res = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello!"},
],
stream=True,
)
for chunk in res:
for choice in chunk.choices:
if choice.delta.content:
print(choice.delta.content, end="", flush=True)
# Hello! How can I assist you today?
OpenAI plugins clients:
client = OpenAI(base_url="http://localhost:8682/v1")
res = client.chat.completions.create(
model="google/gemini-1.5-flash",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello!"},
],
stream=True,
)
for choice in res.choices:
print(f"{choice.message.role}: {choice.message.content}")