Get the weekly radar
One short email with the freshest products, every week.
Discover anthropic products found across trusted public sources.
The AI starterkit with everything you need to build & launch AI apps quickly, without all the headaches and frustration. Includes 9+ ready-to-use AI demo applications.
Observability and DevTool platform for AI Agents agentops_demo.mp4 AgentOps helps developers build, evaluate, and monitor AI agents. From prototype to production. Open Source The AgentOps app is open source under the MIT license. Explore the code in our app directory. Key Integrations 🔌 📊 Replay Analytics and Debugging Step-by-step agent execution graphs 💸 LLM Cost Management Track spend with LLM foundation model providers 🤝 Framework Integrations Native Integrations with CrewAI, AG2 (AutoGen), Agno, LangGraph, & more ⚒️ Self-Host Want to run AgentOps on your own cloud? You're covered Quick Start ⌨️ pip install agentops Session replays in 2 lines of code Initialize the AgentOps client and automatically get analytics on all your LLM calls. Get an API key import agentops # Beginning of your program (i.e. main.py, __init__.py) agentops.init( < INSERT YOUR API KEY HERE >) ... # End of program agentops.end_session('Success') All your sessions can be viewed on the AgentOps dashboard Self-Hosting Looking to run the full AgentOps app (Dashboard + API backend) on your machine? Follow the setup guide in app/README.md: Run the App and Backend (Dashboard + API) Agent Debugging Session Replays Summary Analytics First class Developer Experience Add powerful observability to your agents, tools, and functions with as little code as possible: one line at a time. Refer to our documentation # Create a session span (root for all other spans) from agentops.sdk.decorators import session @session def my_workflow(): # Your session code here return result # Create an agent span for tracking agent operations from agentops.sdk.decorators import agent @agent class MyAgent: def __init__(self, name): self.name = name # Agent methods here # Create operation/task spans for tracking specific operations from agentops.sdk.decorators import operation, task @operation # or @task def process_data(data): # Process the data return result # Create workflow spans for tracking multi-operation workflows from agentops.sdk.decorators import workflow @workflow def my_workflow(data): # Workflow implementation return result # Nest decorators for proper span hierarchy from agentops.sdk.decorators import session, agent, operation @agent class MyAgent: @operation def nested_operation(self, message): return f"Processed: {message}" @operation def main_operation(self): result = self.nested_operation("test message") return result @session def my_session(): agent = MyAgent() return agent.main_operation() All decorators support: Input/Output Recording Exception Handling Async/await functions Generator functions Custom attributes and names Integrations 🦾 OpenAI Agents SDK 🖇️ Build multi-agent systems with tools, handoffs, and guardrails. AgentOps natively integrates with the OpenAI Agents SDKs for both Python and TypeScript. Python pip install openai-agents Python integration guide OpenAI Agents Python documentation TypeScript npm install agentops @openai/agents TypeScript integration guide OpenAI Agents JS documentation CrewAI 🛶 Build Crew agents with observability in just 2 lines of code. Simply set an AGENTOPS_API_KEY in your environment, and your crews will get automatic monitoring on the AgentOps dashboard. pip install 'crewai[agentops]' AgentOps integration example Official CrewAI documentation AG2 🤖 With only two lines of code, add full observability and monitoring to AG2 (formerly AutoGen) agents. Set an AGENTOPS_API_KEY in your environment and call agentops.init() AG2 Observability Example AG2 - AgentOps Documentation Camel AI 🐪 Track and analyze CAMEL agents with full observability. Set an AGENTOPS_API_KEY in your environment and initialize AgentOps to get started. Camel AI - Advanced agent communication framework AgentOps integration example Official Camel AI documentation Installation pip install "camel-ai[all]==0.2.11" pip install agentops import os import agentops from camel.agents import ChatAgent from camel.messages import BaseMessage from camel.models import ModelFactory from camel.types import ModelPlatformType, ModelType # Initialize AgentOps agentops.init(os.getenv("AGENTOPS_API_KEY"), tags=["CAMEL Example"]) # Import toolkits after AgentOps init for tracking from camel.toolkits import SearchToolkit # Set up the agent with search tools sys_msg = BaseMessage.make_assistant_message( role_name='Tools calling operator', content='You are a helpful assistant' ) # Configure tools and model tools = [*SearchToolkit().get_tools()] model = ModelFactory.create( model_platform=ModelPlatformType.OPENAI, model_type=ModelType.GPT_4O_MINI, ) # Create and run the agent camel_agent = ChatAgent( system_message=sys_msg, model=model, tools=tools, ) response = camel_agent.step("What is AgentOps?") print(response) agentops.end_session("Success") Check out our Camel integration guide for more examples including multi-agent scenarios. Langchain 🦜🔗 AgentOps works seamlessly with applications built using Langchain. To use the handler, install Langchain as an optional dependency: Installation pip install agentops[langchain] To use the handler, import and set import os from langchain.chat_models import ChatOpenAI from langchain.agents import initialize_agent, AgentType from agentops.integration.callbacks.langchain import LangchainCallbackHandler AGENTOPS_API_KEY = os.environ['AGENTOPS_API_KEY'] handler = LangchainCallbackHandler(api_key=AGENTOPS_API_KEY, tags=['Langchain Example']) llm = ChatOpenAI(openai_api_key=OPENAI_API_KEY, callbacks=[handler], model='gpt-3.5-turbo') agent = initialize_agent(tools, llm, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True, callbacks=[handler], # You must pass in a callback handler to record your agent handle_parsing_errors=True) Check out the Langchain Examples Notebook for more details including Async handlers. Cohere ⌨️ First class support for Cohere(>=5.4.0). This is a living integration, should you need any added functionality please message us on Discord! AgentOps integration example Official Cohere documentation Installation pip install cohere import cohere import agentops # Beginning of program's code (i.e. main.py, __init__.py) agentops.init(<INSERT YOUR API KEY HERE>) co = cohere.Client() chat = co.chat( message="Is it pronounced ceaux-hear or co-hehray?" ) print(chat) agentops.end_session('Success') import cohere import agentops # Beginning of program's code (i.e. main.py, __init__.py) agentops.init(<INSERT YOUR API KEY HERE>) co = cohere.Client() stream = co.chat_stream( message="Write me a haiku about the synergies between Cohere and AgentOps" ) for event in stream: if event.event_type == "text-generation": print(event.text, end='') agentops.end_session('Success') Anthropic ﹨ Track agents built with the Anthropic Python SDK (>=0.32.0). AgentOps integration guide Official Anthropic documentation Installation pip install anthropic import anthropic import agentops # Beginning of program's code (i.e. main.py, __init__.py) agentops.init(<INSERT YOUR API KEY HERE>) client = anthropic.Anthropic( # This is the default and can be omitted api_key=os.environ.get("ANTHROPIC_API_KEY"), ) message = client.messages.create( max_tokens=1024, messages=[ { "role": "user", "content": "Tell me a cool fact about AgentOps", } ], model="claude-3-opus-20240229", ) print(message.content) agentops.end_session('Success') Streaming import anthropic import agentops # Beginning of program's code (i.e. main.py, __init__.py) agentops.init(<INSERT YOUR API KEY HERE>) client = anthropic.Anthropic( # This is the default and can be omitted api_key=os.environ.get("ANTHROPIC_API_KEY"), ) stream = client.messages.create( max_tokens=1024, model="claude-3-opus-20240229", messages=[ { "role": "user", "content": "Tell me something cool about streaming agents", } ], stream=True, ) response = "" for event in stream: if event.type == "content_block_delta": response += event.delta.text elif event.type == "message_stop": print("\n") print(response) print("\n") Async import asyncio from anthropic import AsyncAnthropic client = AsyncAnthropic( # This is the default and can be omitted api_key=os.environ.get("ANTHROPIC_API_KEY"), ) async def main() -> None: message = await client.messages.create( max_tokens=1024, messages=[ { "role": "user", "content": "Tell me something interesting about async agents", } ], model="claude-3-opus-20240229", ) print(message.content) await main() Mistral 〽️ Track agents built with the Mistral Python SDK (>=0.32.0). AgentOps integration example Official Mistral documentation Installation pip install mistralai Sync from mistralai import Mistral import agentops # Beginning of program's code (i.e. main.py, __init__.py) agentops.init(<INSERT YOUR API KEY HERE>) client = Mistral( # This is the default and can be omitted api_key=os.environ.get("MISTRAL_API_KEY"), ) message = client.chat.complete( messages=[ { "role": "user", "content": "Tell me a cool fact about AgentOps", } ], model="open-mistral-nemo", ) print(message.choices[0].message.content) agentops.end_session('Success') Streaming from mistralai import Mistral import agentops # Beginning of program's code (i.e. main.py, __init__.py) agentops.init(<INSERT YOUR API KEY HERE>) client = Mistral( # This is the default and can be omitted api_key=os.environ.get("MISTRAL_API_KEY"), ) message = client.chat.stream( messages=[ { "role": "user", "content": "Tell me something cool about streaming agents", } ], model="open-mistral-nemo", ) response = "" for event in message: if event.data.choices[0].finish_reason == "stop": print("\n") print(response) print("\n") else: response += event.text agentops.end_session('Success') Async import asyncio from mistralai import Mistral client = Mistral( # This is the default and can be omitted api_key=os.environ.get("MISTRAL_API_KEY"), ) async def main() -> None: message = await client.chat.complete_async( messages=[ { "role": "user", "content": "Tell me something interesting about async agents", } ], model="open-mistral-nemo", ) print(message.choices[0].message.content) await main() Async Streaming import asyncio from mistralai import Mistral client = Mistral( # This is the default and can be omitted api_key=os.environ.get("MISTRAL_API_KEY"), ) async def main() -> None: message = await client.chat.stream_async( messages=[ { "role": "user", "content": "Tell me something interesting about async streaming agents", } ], model="open-mistral-nemo", ) response = "" async for event in message: if event.data.choices[0].finish_reason == "stop": print("\n") print(response) print("\n") else: response += event.text await main() CamelAI ﹨ Track agents built with the CamelAI Python SDK (>=0.32.0). CamelAI integration guide Official CamelAI documentation Installation pip install camel-ai[all] pip install agentops #Import Dependencies import agentops import os from getpass import getpass from dotenv import load_dotenv #Set Keys load_dotenv() openai_api_key = os.getenv("OPENAI_API_KEY") or "<your openai key here>" agentops_api_key = os.getenv("AGENTOPS_API_KEY") or "<your agentops key here>" You can find usage examples here!. LiteLLM 🚅 AgentOps provides support for LiteLLM(>=1.3.1), allowing you to call 100+ LLMs using the same Input/Output Format. AgentOps integration example Official LiteLLM documentation Installation pip install litellm # Do not use LiteLLM like this # from litellm import completion # ... # response = completion(model="claude-3", messages=messages) # Use LiteLLM like this import litellm ... response = litellm.completion(model="claude-3", messages=messages) # or response = await litellm.acompletion(model="claude-3", messages=messages) LlamaIndex 🦙 AgentOps works seamlessly with applications built using LlamaIndex, a framework for building context-augmented generative AI applications with LLMs. Installation pip install llama-index-instrumentation-agentops To use the handler, import and set
Your AI bill can be quietly wrong. Providers shouldn't get to grade their own bills. Today, the company that charges you also decides what counts as a failure, what gets credited, and keeps the only detailed records. inferock-bench puts an independent, per-call receipt of what you were billed — and what failed — in your own hands. We built inferock-bench because we kept paying for answers that died mid-sentence, and nobody could tell us where the money went. Quickstart · Test your loss · What can go wrong · Key boundary · Docs Use it when you need to audit an AI/LLM bill, measure Claude or GPT token usage locally, or answer "was I billed for a failed API call?" It is a local LLM cost-tracking proxy for four measured provider planes: OpenAI, Anthropic, Gemini Developer API, and pinned OpenRouter endpoints spanning meta-llama, deepseek, mistral, moonshot/kimi, z-ai/glm, and qwen on observed hosts. Everything else is extensible-by-design, not measured today. It does not declare every mismatch an OpenAI overcharge or Anthropic billing error; it preserves token, cost, retry, and failure evidence so billing-integrity questions can be checked. Common cases it can help you inspect: a failed or timed-out request that still has usage, token counts that do not match the visible output, retries that may have amplified cost, and latency or model-version changes that need a trail. It cannot cap provider spend across calls it never sees, and it cannot explain traffic that bypassed the local proxy. The receipt headline Receipt word Plain-English meaning spent provider spend observed by the run for priced calls it saw. money loss bill-bounded dollar loss The Inferock Standard can tie to observed spend or charge evidence. time loss real wait or downtime measured as time, never added to dollars. invoice-check exposure an invoice-check amount, such as cache discount at risk; it is labeled "verify your invoice" and never summed into money loss. Real measured traffic, not fixture rows. Measured since 2026-07-09, the cumulative public ledger through run15 captured 1,268 measured calls, 565 failures/signals, $7.15 provider spend observed, $0.07 bill-bounded money loss (stored exact: $0.073875), ~2.9 min time loss, and $16.80 invoice-check exposure across 202 cache-discount-at-risk signals. The current-code cumulative receipt watches 12 of 13 surfaces and keeps invoice-check exposure separate from money loss. Run facts: sanitized public run card for 2026-07-09 and run15 public run card for 2026-07-10. The 2026-07-06 0.1.7 card remains published as a historical artifact; the current public receipt presentation ships with 0.1.10. ImportantThe receipt is spend-anchored. The headline is spent $X · money loss $Y · time loss Z · invoice-check exposure $E; bill-bounded money loss and recognition gap never include invoice-check exposure. CACHE_DISCOUNT_AT_RISK is still visible below the headline as a separate detail line that says "verify your invoice" rather than as money loss or a refund claim. Watch it run Share the receipt Quickstart Run it locally. We think you should be able to see exactly what a provider failure cost you, to the cent. Provider keys are not sent to Inferock; attached only to provider requests. Prerequisite: install Node.js 22+ with npm. node --version npm --version If either command is missing, install Node.js 22 or newer first. Run the local benchmark: npx inferock-bench The first run downloads the package and can take a minute or two before printing anything. Leave it running. You see lines like: inferock-bench listening at http://127.0.0.1:4318 Dashboard: http://127.0.0.1:4318/ Config: ~/.inferock-bench/config Save your provider key locally. Easiest path with the server from step 2 still running: open http://127.0.0.1:4318/ and save the provider key in the dashboard. Create that key in your provider account first; use a low-limit or development key while evaluating. It stays local under ~/.inferock-bench/, is saved with owner-only file permissions, and is shown back only in masked form. CLI path, before starting the server or after stopping it: npx inferock-bench setup <provider> The setup prompt hides your key while you type it. On a headless machine, pipe the key from your secret manager into the same command. To see the current supported provider names, run npx inferock-bench status or npx inferock-bench --help. A running server does not reload provider keys written by a separate CLI setup process; restart it after CLI setup. Any traffic you send through the benchmark is real provider usage. Start with a few short prompts and expect a small evaluation spend, controlled by your provider account limit. The built-in npx inferock-bench test flow shows estimated tokens, estimated dollars, and a spend cap before it makes any provider call. Get your local bench key: npx inferock-bench key reveal This prints the local ibl_ bench key to stdout, so it is pipe-friendly. It is a LOCAL-ONLY credential, not your provider key. To copy it to the clipboard instead, run: npx inferock-bench key copy If no clipboard is available, the copy command falls back to printing the key. You can also copy the local bench key from the dashboard. Check what is configured at any point: npx inferock-bench status It shows each provider's configured/masked state, the local store location, server state, and version. For a fast command list or package version without starting the server, use npx inferock-bench --help or npx inferock-bench --version. Point some traffic at the local benchmark. No app yet? Use one of these equal local targets after saving the matching provider key in step 3. Claude Code: npm i -g @anthropic-ai/claude-code ANTHROPIC_BASE_URL=http://127.0.0.1:4318 ANTHROPIC_API_KEY=ibl_your_local_bench_key claude -p "Draft a five-bullet checklist for reviewing an AI invoice." OpenAI SDK: import OpenAI from "openai"; const openai = new OpenAI({ apiKey: process.env.INFEROCK_BENCH_KEY ?? "ibl_your_local_bench_key", baseURL: "http://127.0.0.1:4318/v1", }); await openai.chat.completions.create({ model: "gpt-4o-mini-2024-07-18", messages: [{ role: "user", content: "Draft a five-bullet checklist for reviewing an AI invoice." }], }); Gemini: await fetch("http://127.0.0.1:4318/v1beta/models/gemini-2.5-flash:generateContent", { method: "POST", headers: { authorization: "Bearer " + (process.env.INFEROCK_BENCH_KEY ?? "ibl_your_local_bench_key"), "content-type": "application/json", }, body: JSON.stringify({ contents: [{ role: "user", parts: [{ text: "Draft a five-bullet checklist for reviewing an AI invoice." }] }], }), }); For these commands, the SDK API key is the local ibl_ bench key from step 4. Your provider key is not passed to Claude Code or your app; it is not sent to Inferock and is attached only to provider requests. If you configured OpenRouter, use the provider-specific SDK snippet in the dashboard's Local app connection panel or in the package README. Note: a Claude subscription (OAuth) login is not a supported mechanism for measuring calls. inferock-bench measures metered API traffic only — save a provider API key in the bench and point your SDK or agent at it with the local ibl_ key, as shown above. After the first successful proxied call, the terminal running inferock-bench prints: first call measured ✓ View your receipt from another terminal: npx inferock-bench receipt --compact If you installed inferock-bench globally, inferock-bench receipt --compact works too. This README uses the npx inferock-bench <cmd> form so one-time users are not stranded. Already have an app? Change exactly two SDK settings: apiKey and baseURL. const client = new YourProviderSdk({ apiKey: process.env.INFEROCK_BENCH_KEY ?? "ibl_your_generated_local_key", baseURL: "http://127.0.0.1:4318", }); Some SDKs use /v1 in the base URL. The dashboard shows the exact value for every configured provider; npx inferock-bench init prints OpenAI and Anthropic constructor snippets. Run npx inferock-bench init to detect OpenAI or Anthropic SDK usage and print the exact SDK change. npx inferock-bench init --patch path/to/client.ts --yes patches simple constructors only when it can update both apiKey and baseURL; otherwise it refuses with a clear message. For Gemini or OpenRouter, use the dashboard's Local app connection snippet or the package README's provider example. Run from source git clone https://github.com/inferock/inferock-bench.git cd inferock-bench pnpm install pnpm -r --workspace-concurrency=1 build node apps/inferock-bench/dist/index.js start Test your loss npx inferock-bench test runs the complete coverage battery through your configured provider scope, on your provider key, so the receipt can show what your provider cost you and which loss surfaces the run actually opened. The checked-in measured baseline powers the estimate, so a configured provider key and priced compatible model are enough to reach the consent step. For the exact formulas behind the receipt, see Paid-loss arithmetic. You see the estimated tokens, estimated dollars, model, suite, baseline, pricing source, and spend cap before any provider call is made. The copy states the price plainly: running the complete test set on the selected provider(s) will cost approximately the displayed amount. If you stop there, the command makes zero provider calls. Interactive runs require you to type RUN; automation must pass the displayed hash with --accept-estimate <hash> because --yes alone is not consent to a changed estimate. In the dashboard, open Advanced options, set Test driver to Agent test, then run the test to use a real coding agent. Agent test currently supports OpenAI and Anthropic runs; use the built-in generator for Gemini and OpenRouter coverage. If the pinned local agent is not installed, the dashboard names the exact npm tarballs, versions, SRI checksums, sizes, source URLs, and local install path before downloading. The agent receives only localhost and an ephemeral local ibl_ key, never your provider key. CLI equivalent: npx inferock-bench test --generator agent. The receipt is run-scoped. It reports spent $X · money loss $Y · time loss Z · invoice-check exposure $E, provider-recognized recovery, bill-bounded recognition gap, the separate invoice-check exposure detail line when applicable, and surfaces watched N of M surfaces your selected providers can open (M varies by provider), with every surface labeled watched-clean, signal, or not-openable. A zero only counts when the surface was watched; unopened surfaces are named as coverage debt, not silently claimed clean. For a priced call that fails the standard and is tied to observed spend or charge evidence, money loss is bill-bounded; provider-recognized can still be $0, and the gap is the difference inside that bill-bounded money ledger. The receipt opens with a one-line plain-English guide to spent dollars, bill-bounded money loss, time loss, and invoice-check exposure. A receipt "failure" is a measurement finding, not necessarily your app crashing. A signal is one finding the benchmark saw, such as a token cross-check. CACHE_DISCOUNT_AT_RISK is shown as invoice-check exposure with "verify your invoice" guidance; it is not summed into money loss or recognition gap. Provider-recognized is the part that appears likely to fit the provider's current credit rules; bill-bounded gaps stay visible instead of being hidden. If no provider key is configured, pricing is unknown, or the token baseline is ever absent or bootstrap-only, the CLI and dashboard fail closed and make zero provider calls. The baseline-degraded state is reported as baseline not measured yet: run `npx inferock-bench test --record-baseline` with explicit consent to produce a real per-task token baseline. The method details are in Coverage test methodology. What your provider doesn't tell you The gap is simple: providers give you totals. They usually do not give you the per-call receipt you would need to prove which answer broke, which retry ran, or which token count changed. We do not think you should have to trust a monthly total. We think every broken answer should leave a trail: what happened, how sure we are, and whether the provider would actually recognize the claim. Illustrative mechanism — not measured data. This is the kind of billing blind spot we built inferock-bench to catch on your own traffic.
Token Governor Re-engineer LLM prompts to spend the fewest tokens that still answer — a governing gateway with auth, budgets, caching, and streaming that works with Anthropic, OpenAI, Gemini, and any OpenAI-compatible provider. Plus a live A/B, saved constraint presets, and a batch triage that flags prompts which never needed a model. llm · tokens · cost-optimization · ai-gateway · llmops · openai · anthropic · gemini · streaming · prompt-engineering Single mode — build a constrained request, see projected savings, and get a verdict when the wrapper costs more than it saves. Presets — save/load constraint configs; they persist in the browser. Batch mode — score a list of prompts, flag deterministic ones, export CSV/Markdown, and get a suggested script for each flagged row. Live A/B — runs your prompt twice (raw vs. governed) through a server-side proxy and reports real token counts, using whichever provider you configure. The app A fixed one-screen dashboard (no page scroll — panels scroll internally). The top bar carries the mode tabs and pricing/scale inputs; below sit three panels: Prompt & constraints — the prompt plus succinctness, output shape, temperature, and toggles. Projected savings — a gauge, a "worth it / costs more than it saves" verdict, and a before/after cost receipt. Assembled request & live A/B — the generated system prompt with copy buttons, and a raw-vs-governed A/B that reports the real token counts returned. Switch to Batch mode and this becomes a triage table that flags deterministic prompts. (The image is a layout mockup, not a screenshot — run the app for the live, interactive version.) Requirements Node.js 18+ (uses the built-in fetch) An API key for one LLM provider (only needed for the live A/B panel) Setup npm install cp .env.example .env # then edit .env (see "Providers" below) Providers The server is provider-agnostic. Set these in .env: Variable Purpose LLM_PROVIDER anthropic or openai (openai = any OpenAI-compatible API) LLM_API_KEY your key (kept server-side; never sent to the browser) LLM_MODEL model id your key can access LLM_BASE_URL optional; override to point at Gemini, Groq, OpenRouter, Ollama, etc. The frontend never changes: it POSTs a normalized request to /api/messages, and the adapter in server/providers.js translates to/from the chosen provider and returns a normalized response. To add a native provider, edit that one file. Supported providers Almost every provider now offers an OpenAI-compatible API, so it works by setting a base URL and model — no code change. Only providers you want to hit on their own native protocol need an adapter. Provider Supported? How Anthropic Yes native (LLM_PROVIDER=anthropic, or gateway /v1/messages) OpenAI Yes LLM_PROVIDER=openai Gemini Yes OpenAI-compat base URL (below) Groq, OpenRouter, Together, Mistral, Fireworks, DeepSeek, xAI/Grok Yes OpenAI-compat base URL Local — Ollama, vLLM, llama.cpp Yes OpenAI-compat base URL Azure OpenAI Mostly OpenAI-compat, but needs a different auth header + api-version Cohere / Bedrock / Vertex native protocols Not yet would need a native adapter in server/providers.js Examples All OpenAI-compatible providers use LLM_PROVIDER=openai: # OpenAI LLM_MODEL=gpt-4o-mini # Gemini (Google AI Studio) — NOTE: no trailing slash (the code appends /chat/completions) LLM_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai LLM_MODEL=gemini-2.5-flash # use a model your key can access # Groq LLM_BASE_URL=https://api.groq.com/openai/v1 # OpenRouter LLM_BASE_URL=https://openrouter.ai/api/v1 # Local Ollama LLM_BASE_URL=http://localhost:11434/v1 LLM_API_KEY=ollama Notes: Gemini trailing slash. Google's own docs use .../v1beta/openai/ because the OpenAI SDK appends chat/completions. This server appends /chat/completions, so drop the trailing slash or you'll get a double-slash 404. Some newer OpenAI models expect max_completion_tokens — set OPENAI_MAX_TOKENS_PARAM=max_completion_tokens if you hit that error. Model availability changes and varies by key; if a model 404s, pick one from your provider's current list. npm run check:live confirms the wiring end to end. Native Gemini SDK (generateContent) is not supported. If your app uses the Google GenAI SDK's native API rather than its OpenAI-compat mode, either switch that client to OpenAI-compat mode or add a native Gemini adapter (a body-mapper plus a :streamGenerateContent passthrough, parallel to the Anthropic adapter). Develop npm run dev Vite frontend on http://localhost:5173 Express API on http://localhost:8787 (Vite proxies /api to it) Build and run in production npm run build # outputs dist/ npm start # Express serves dist/ AND /api on PORT (default 8787) Pre-deploy check Confirm the key and model are wired before you ship: npm run check # hits a running server's /api/health npm run check:start # boots the server itself, checks, stops npm run check:live # also sends a tiny real prompt (costs a few tokens) Check a deployed instance instead of localhost: HEALTH_URL=https://your-app.example.com/api/health npm run check It verifies the health endpoint responds, the key is present, and a model and base URL are configured; --live additionally confirms a real round-trip. Exit code is 0 on success, 1 on any failure — so it drops straight into CI. Runtime gateway (integrate with a live app) Beyond the analysis UI, the server is also a governing gateway. Point your existing SDK's base_url at it and every call is governed automatically — constraints injected, max_tokens capped, deterministic prompts optionally blocked — then streamed straight back. Your app code barely changes. Native endpoints: Endpoint Protocol Upstream POST /v1/messages Anthropic Anthropic (or ANTHROPIC_BASE_URL) POST /v1/chat/completions OpenAI-compatible OpenAI / OPENAI_BASE_URL GET /v1/governor/info — lists policies + wiring (no secrets) Per-request control via headers: Header Effect x-governor-policy: terse pick a named policy (default, terse, strict, passthrough) x-governor-detect: block override detect mode (off / warn / block) x-governor: off bypass governance for this call Policies live in server/policies.js (instructions, maxTokensCap, forceTemperature, detect). Edit or add your own. Point your SDK at it # OpenAI SDK from openai import OpenAI client = OpenAI(base_url="http://localhost:8787/v1", api_key="unused") client.chat.completions.create( model="gpt-4o-mini", stream=True, messages=[{"role": "user", "content": "..."}], extra_headers={"x-governor-policy": "terse"}, ) # Anthropic SDK from anthropic import Anthropic client = Anthropic(base_url="http://localhost:8787", api_key="unused") with client.messages.stream(model="claude-sonnet-5", max_tokens=1024, messages=[{"role": "user", "content": "..."}]) as s: for text in s.text_stream: print(text, end="") (The gateway supplies the real key from its env; the SDK's api_key is ignored.) Streaming stream: true is passed through as Server-Sent Events, unmodified, with backpressure preserved. The gateway sniffs a copy only to log token usage — it never alters the bytes your client receives. Try it: node examples/stream.mjs # OpenAI-protocol streaming demo curl -N http://localhost:8787/v1/chat/completions \ -H 'content-type: application/json' -H 'x-governor-policy: terse' \ -d '{"model":"gpt-4o-mini","stream":true,"messages":[{"role":"user","content":"hi"}]}' Notes Blocking is opt-in. Only the strict policy (or x-governor-detect: block) blocks deterministic prompts; it returns a provider-shaped 400 with a suggested script. The detector is a heuristic — run it in warn first. max_tokens is the only hard lever — instructions are advisory, the cap is enforced. Usage logging prints a [usage] … line per call. For OpenAI streaming, set stream_options.include_usage=true (the demo does) or usage will read as zero on streams. Same-protocol upstreams. Use the endpoint that matches your upstream; the gateway forwards natively rather than translating between protocols (that's what /api/messages is for). Resilience. Upstream calls have a connect timeout and retry with backoff on 429/5xx (GATEWAY_TIMEOUT_MS, GATEWAY_RETRIES). The timeout covers connect/headers, not the stream body, so long streams aren't cut off. Multi-instance. Set REDIS_URL (and npm i redis) to share budgets and cache across replicas; otherwise state is per-process. Gemini through the gateway. Point the OpenAI upstream at Gemini's OpenAI-compat base and call /v1/chat/completions: set OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai (no trailing slash) and OPENAI_API_KEY=<gemini key>, then use a gemini-* model. See the Providers table for the full compatibility list. Control plane: auth, budgets, cache The gateway is also a cost-control plane. All three are optional and layer onto the same /v1/* endpoints (the app's own /api/messages is unaffected). Auth. Define callers, each with a bearer key and optional pinned policy and budget, in gateway.callers.json (see gateway.callers.example.json): { "callers": [ { "id": "web-app", "key": "gw_live_...", "policy": "terse", "budget": { "tokens": 2000000, "usd": 20 } }, { "id": "worker", "key": "env:WORKER_KEY", "budget": { "tokens": 500000 } } ] } Callers send Authorization: Bearer <key>. A key value of env:VAR is pulled from the environment (so it can come from an injected secret). With no callers configured the gateway runs open (anonymous) — fine for local, not for production. A caller's pinned policy is authoritative: clients can't loosen it with x-governor-* headers. Budgets. Per-caller daily token and/or USD caps (UTC reset). When a caller is over, the next call gets 429 with Retry-After. Cache hits and the current count are free. USD caps need a pricing file (gateway.pricing.example.json → GATEWAY_PRICING_FILE); token caps need nothing. Check usage: curl -H 'Authorization: Bearer gw_live_...' http://localhost:8787/v1/governor/usage Enforcement is soft (checked before each call, recorded after), and the ledger is in-memory — single-instance. For multiple replicas, back server/budget.js with a shared store (e.g. Redis). Cache. Exact-match cache for non-streaming, deterministic calls (temperature <= GATEWAY_CACHE_MAX_TEMP, default 0). A hit returns the stored response with x-governor-cache: hit, skips the provider, and doesn't charge budget. Sampled (higher-temperature) and streaming calls are never cached, so you never get a stale non-deterministic answer. Bypass per request with x-governor-cache: off. Introspect the whole plane at GET /v1/governor/info. Tests npm test Dependency-free (node --test) coverage of the control plane — auth token checks, budget enforcement, and cache determinism/keying — runnable without a provider key. npm run check / check:live validate a running server's wiring. docker build -t token-governor . docker run -p 8787:8787 \ -e LLM_PROVIDER=anthropic -e LLM_API_KEY=sk-... -e LLM_MODEL=claude-sonnet-5 \ token-governor Secret externalization The API key is never hardcoded — it comes from the environment. There are two ways to source it from a managed secret store: 1. Platform injection (recommended, no code). Let your platform fetch the secret and inject it as an env var at deploy time. Nothing in the app changes: Cloud Run: --set-secrets=LLM_API_KEY=anthropic-key:latest ECS/Fargate: task-definition secrets from AWS Secrets Manager or SSM Kubernetes: External Secrets Operator / CSI driver → env var Azure App Service / Container Apps: Key Vault references → app setting 2. In-app fetch (built in). Point any *_API_KEY at a managed secret using a scheme, and the server resolves it at startup (see server/secrets.js):
A workspace to build, deploy and manage AI agents and workflows. Quickstart Cloud-hosted: sim.ai Self-hosted npx simstudio Open http://localhost:3000 Docker must be installed and running. Use -p, --port <port> to run Sim on a different port, or --no-pull to skip pulling the latest Docker images. Capabilities Connect 1,000+ integrations and every major LLM Add Slack, Notion, HubSpot, Salesforce, databases, and more Build agents visually, conversationally, or with code Ingest files, knowledge bases, and structured table data Monitor runs, logs, schedules, and workflow activity One workspace, every surface Chat and workflows are just the start — tables, files, knowledge, and scheduled tasks all live in the same workspace. Tables — a database, built in Files — one store for your team and every agent Knowledge — your agents' memory Scheduled tasks — runs on your schedule Self-hosting Docker Compose git clone https://github.com/simstudioai/sim.git && cd sim docker compose -f docker-compose.prod.yml up -d Open http://localhost:3000 Sim also supports local models via Ollama and vLLM. See the Docker self-hosting docs for setup details. Manual Setup Requirements: Bun, Node.js v20+, PostgreSQL 12+ with pgvector Clone and install: git clone https://github.com/simstudioai/sim.git cd sim bun install bun run prepare # Set up pre-commit hooks Set up PostgreSQL with pgvector: docker run --name simstudio-db -e POSTGRES_PASSWORD=your_password -e POSTGRES_DB=simstudio -p 5432:5432 -d pgvector/pgvector:pg17 Or install manually via the pgvector guide. Configure environment: cp apps/sim/.env.example apps/sim/.env # Create your secrets perl -i -pe "s/your_encryption_key/$(openssl rand -hex 32)/" apps/sim/.env perl -i -pe "s/your_internal_api_secret/$(openssl rand -hex 32)/" apps/sim/.env perl -i -pe "s/your_api_encryption_key/$(openssl rand -hex 32)/" apps/sim/.env # DB configs for migration cp packages/db/.env.example packages/db/.env # Edit both .env files to set DATABASE_URL="postgresql://postgres:your_password@localhost:5432/simstudio" Run migrations: cd packages/db && bun run db:migrate Start development servers: bun run dev:full # Starts Next.js app and realtime socket server Or run separately: bun run dev (Next.js) and cd apps/sim && bun run dev:sockets (realtime). Chat API Keys Chat is a Sim-managed service. To use Chat on a self-hosted instance: Go to https://sim.ai → Settings → Chat keys and generate a Chat API key Set COPILOT_API_KEY environment variable in your self-hosted apps/sim/.env file to that value Environment Variables See the environment variables reference for the full list, or apps/sim/.env.example for defaults. Tech Stack Next.js · Bun · PostgreSQL · Drizzle · Better Auth · Tailwind — and the rest of the stack Framework: Next.js (App Router) Runtime: Bun Database: PostgreSQL with Drizzle ORM Authentication: Better Auth Schema Validation: Zod UI: Shadcn, Tailwind CSS Streaming Markdown: Streamdown State Management: Zustand, TanStack Query Flow Editor: ReactFlow Docs: Fumadocs Monorepo: Turborepo Realtime: Socket.io Background Jobs: Trigger.dev Remote Code Execution: E2B Isolated Code Execution: isolated-vm Contributing We welcome contributions! Please see our Contributing Guide for details.
AI Pair Programming in Your Terminal Aider lets you pair program with LLMs to start a new project or build on your existing codebase. Features Cloud and local LLMs Aider works best with Claude 3.7 Sonnet, DeepSeek R1 & Chat V3, OpenAI o1, o3-mini & GPT-4o, but can connect to almost any LLM, including local models. Maps your codebase Aider makes a map of your entire codebase, which helps it work well in larger projects. 100+ code languages Aider works with most popular programming languages: python, javascript, rust, ruby, go, cpp, php, html, css, and dozens more. Git integration Aider automatically commits changes with sensible commit messages. Use familiar git tools to easily diff, manage and undo AI changes. Use in your IDE Use aider from within your favorite IDE or editor. Ask for changes by adding comments to your code and aider will get to work. Images & web pages Add images and web pages to the chat to provide visual context, screenshots, reference docs, etc. Voice-to-code Speak with aider about your code! Request new features, test cases or bug fixes using your voice and let aider implement the changes. Linting & testing Automatically lint and test your code every time aider makes changes. Aider can fix problems detected by your linters and test suites. Copy/paste to web chat Work with any LLM via its web chat interface. Aider streamlines copy/pasting code context and edits back and forth with a browser. Getting Started python -m pip install aider-install aider-install # Change directory into your codebase cd /to/your/project # DeepSeek aider --model deepseek --api-key deepseek=<key> # Claude 3.7 Sonnet aider --model sonnet --api-key anthropic=<key> # o3-mini aider --model o3-mini --api-key openai=<key> See the installation instructions and usage documentation for more details. More Information Documentation Installation Guide Usage Guide Tutorial Videos Connecting to LLMs Configuration Options Troubleshooting FAQ Community & Resources LLM Leaderboards GitHub Repository Discord Community Release notes Blog Kind Words From Users "My life has changed... Aider... It's going to rock your world." — Eric S. Raymond on X "The best free open source AI coding assistant." — IndyDevDan on YouTube "The best AI coding assistant so far." — Matthew Berman on YouTube "Aider ... has easily quadrupled my coding productivity." — SOLAR_FIELDS on Hacker News "It's a cool workflow... Aider's ergonomics are perfect for me." — qup on Hacker News "It's really like having your senior developer live right in your Git repo - truly amazing!" — rappster on GitHub "What an amazing tool. It's incredible." — valyagolev on GitHub "Aider is such an astounding thing!" — cgrothaus on GitHub "It was WAY faster than I would be getting off the ground and making the first few working versions." — Daniel Feldman on X "THANK YOU for Aider! It really feels like a glimpse into the future of coding." — derwiki on Hacker News "It's just amazing. It is freeing me to do things I felt were out my comfort zone before." — Dougie on Discord "This project is stellar." — funkytaco on GitHub "Amazing project, definitely the best AI coding assistant I've used." — joshuavial on GitHub "I absolutely love using Aider ... It makes software development feel so much lighter as an experience." — principalideal0 on Discord "I have been recovering from ... surgeries ... aider ... has allowed me to continue productivity." — codeninja on Reddit "I am an aider addict. I'm getting so much more work done, but in less time." — dandandan on Discord "Aider... blows everything else out of the water hands down, there's no competition whatsoever." — SystemSculpt on Discord "Aider is amazing, coupled with Sonnet 3.5 it's quite mind blowing." — Josh Dingus on Discord "Hands down, this is the best AI coding assistant tool so far." — IndyDevDan on YouTube "[Aider] changed my daily coding workflows. It's mind-blowing how ...(it)... can change your life." — maledorak on Discord "Best agent for actual dev work in existing codebases." — Nick Dobos on X "One of my favorite pieces of software. Blazing trails on new paradigms!" — Chris Wall on X "Aider has been revolutionary for me and my work." — Starry Hope on X "Try aider! One of the best ways to vibe code." — Chris Wall on X "Freaking love Aider." — hztar on Hacker News "Aider is hands down the best. And it's free and opensource." — AriyaSavakaLurker on Reddit "Aider is also my best friend." — jzn21 on Reddit "Try Aider, it's worth it." — jorgejhms on Reddit "I like aider :)" — Chenwei Cui on X "Aider is the precision tool of LLM code gen... Minimal, thoughtful and capable of surgical changes ... while keeping the developer in control." — Reilly Sweetland on X "Cannot believe aider vibe coded a 650 LOC feature across service and cli today in 1 shot." - autopoietist on Discord "Oh no the secret is out! Yes, Aider is the best coding tool around. I highly, highly recommend it to anyone." — Joshua D Vander Hook on X "thanks to aider, i have started and finished three personal projects within the last two days" — joseph stalzyn on X "Been using aider as my daily driver for over a year ... I absolutely love the tool, like beyond words." — koleok on Discord "Aider ... is the tool to benchmark against." — BeetleB on Hacker News "aider is really cool" — kache on X
Our AI finds products from public sources, then rotates exposure so useful newcomers have a fair chance to be seen.
Learn how discovery works
