LLM Chat Server

Example Use Cases

• 1

  A hosted API layer to send user messages to Ollama or OpenAI models with configurable model, provider, and request parameters.

• 2

  A built-in tool approval workflow to require human approval before executing external tool calls or actions.

• 3

  Session management and optional streaming responses to maintain context across interactions and deliver incremental outputs.

Description

This repository provides a configurable FastAPI server for chat interactions with LLM providers (Ollama or OpenAI). It exposes endpoints to start sessions, send user messages, handle tool invocation approvals, and retrieve session state, with configuration via environment variables or CLI arguments. The server supports streaming responses and is designed to integrate LLM capabilities into applications while controlling tool calls through an approval process.