AI Integration

Quadkit ships a modular AI stack built on the same contract-first foundation as the rest of the framework. You program against protocols (LLMClientProtocol, RAGPipelineProtocol, …), so providers and models are swappable through configuration alone.

The AI layer is composed of focused, independently installable packages:

Package	Purpose
`an AI extension`	Orchestration layer — discovers and wires the AI subsystems below
`quadkit-ai-llm`	Multi-provider LLM client (OpenAI, Anthropic, Gemini, Ollama, Groq, Mistral, …)
`a RAG extension`	Retrieval-augmented generation pipeline
`a vector-store extension`	Vector store backends (pgvector, Qdrant, Pinecone, in-memory)
`quadkit-ai-agents`	Agents with tools and strategies (ReAct, plan-and-execute)
`an AI memory extension`	Episodic, semantic, and working memory
`an AI extension`	Conversation sessions — branching, checkpointing, multi-agent
`an AI skills extension`	Skill/tool registry and executor
`quadkit-ai-mcp`	Model Context Protocol server and client
`an AI workers extension`	Background AI work — batch embedding, document ingestion
`an AI extension`	Tracing, metrics, and health checks for AI calls
`an AI evaluation extension`	Feedback collection and processing
`an AI extension`	LLM output evaluation (Q&A, string/embedding distance, trajectory, criteria) and reproducible experiment tracking

1. Configuring the LLM Client

quadkit-ai-llm exposes a single LLMClientProtocol and selects the concrete provider from configuration. Wire it through the published LLM module:

from quadkit import Application
from quadkit.ai.llm import LLMModule, ClientConfig


def create_app() -> Application:
    app = Application(name="my-ai-app")
    app.add_module(
        LLMModule.configure(
            ClientConfig(provider="anthropic", model="claude-sonnet-4-6")
        )
    )
    return app

Equivalent YAML — providers are an ordered list under the ai_llm section (the first is highest priority):

ai_llm:
  enabled: true
  strategy: sequential          # sequential | parallel_race | cost_optimized | latency_optimized
  providers:
    - name: primary
      model: claude-sonnet-4-6
      api_key: "${ANTHROPIC_API_KEY}"
  defaults:
    temperature: 0.2

2. Calling the LLM

Inject LLMClientProtocol and call complete(). It returns a Result — there are no exceptions for expected failures (rate limits, provider errors):

from quadkit.contracts.ai.llm import LLMClientProtocol
from quadkit.result import Result


class ChatService:
    def __init__(self, llm: LLMClientProtocol) -> None:
        self._llm = llm

    async def reply(self, prompt: str) -> str:
        result = await self._llm.complete(
            messages=[{"role": "user", "content": prompt}],
        )
        if result.is_err():
            return f"LLM error: {result.unwrap_err()}"
        return result.unwrap().content

complete() accepts a plain message list and supports model, temperature, max_tokens, tools, and stop_sequences overrides. For token-by-token output, use stream_chat(...), which returns an async stream of chunks.

3. Thinking Suppression

Some models (Qwen3, Gemma, and other reasoning models served via LM Studio / vLLM / SGLang) emit chain-of-thought tokens by default, adding 20–30s of latency. Quadkit can suppress this at the provider level via ThinkingConfig:

from quadkit.contracts.ai.thinking import ThinkingConfig
from quadkit.ai.llm import ClientConfig

ClientConfig(
    provider="lmstudio",
    model="qwen3",
    thinking=ThinkingConfig(suppress=True),   # inject `enable_thinking: false`
)

Or per provider in the routing config / via env var:

QK_AI_LLM__PROVIDERS__PRIMARY__SUPPRESS_THINKING=true

ThinkingConfig also exposes budget_tokens (Anthropic, Gemini 2.5), effort (OpenAI o-series), and level (Gemini 3) for models where you want reasoning but with a bound.

4. Retrieval-Augmented Generation (RAG)

A RAG extension can coordinate chunking, embedding, vector retrieval, and synthesis behind RAGPipelineProtocol. A module-level RAG configuration API is not part of the published packages, so the example below focuses on the stable contract boundary.

Then query through the injected pipeline:

from quadkit.contracts.ai.rag import RAGPipelineProtocol, RAGContext


class DocsService:
    def __init__(self, rag: RAGPipelineProtocol) -> None:
        self._rag = rag

    async def ask(self, question: str) -> str:
        result = await self._rag.execute(RAGContext(query=question))
        answer = result.unwrap()
        return answer.answer  # plus citations / sources when enabled

The vector backend (pgvector, Qdrant, Pinecone, or in-memory for tests) is provided by a vector-store extension and selected via the vector config section — your RAG code never changes when you switch stores.

5. Agents, Memory & Sessions

For multi-step reasoning, quadkit-ai-agents provides agents that call tools and follow strategies such as ReAct and plan-and-execute. Pair them with:

an AI skills extension — a registry of callable tools the agent can invoke.
an AI memory extension — episodic / semantic / working memory across turns.
an AI extension — durable conversations with branching and checkpointing.

These compose through the container like any other Quadkit services. See the per-package guides under the ecosystem for the exact tool-registration and executor APIs.

6. Observability

an AI extension adds tracing, metrics, and health checks around AI calls — giving you visibility into latency, token usage, and retrieval steps without changing your service code:

ai_observability:
  enabled: true
  metrics_enabled: true
  tracing_enabled: true
  health_checks_enabled: true

Next Steps

The Ecosystem — every public package at a glance
Configuration — sections, profiles, and env-var overrides
Result Pattern — how complete() and query() report failures