agents-cli

@cadell_falconer Thanks a lot! Not in the short term but the generated code is fully portable to any platform!

Hey all, I'm Elia. I work with developers building agents on Google Cloud, and today we're launching Agents CLI. You've probably run into this yourselves. You can build a demo agent in an afternoon, but getting one to production is weeks of work, and barely any of it is the actual agent. It's the plumbing around it: SDKs, MCP servers, auth, telemetry, CI/CD, and ten docs tabs open at once. Hand that to your coding agent (Antigravity, Claude Code, Codex) and it spends most of the session figuring out the setup instead of your agent's actual behavior. So we built a CLI your coding agent can drive. Install it with one line: `uvx google-agents-cli setup` Then ask it something like "build an SRE agent that reads logs and drafts an incident report." Under the hood it runs: ``` agents-cli create my-agent # scaffold: agent, tools, tests, Dockerfile, Observability, Terraform, CI/CD agents-cli eval run # run the agent over a dataset and score it, so you iterate on numbers agents-cli deploy # deploy to Vertex AI Agent Engine, Cloud Run, or GKE ``` Agents CLI will carefully support your coding agent into the whole path to production. It will leverage expert crafted templates, built-in evals, and it runs headless so your coding agent can keep going on its own and self-optimize based on what you specified as success criteria! We optimize for ADK (Google's open-source agent framework) and Google Cloud, but you can customize as you prefer. You can adapt to any model or any custom setup you might have. I'll be in the comments all day. Any particular agent you'd like to ship? Drop it below and I'll tell you how I'd build it! Show more

the headless self-optimize loop is the part I'd want to understand before trusting it. if the coding agent is scoring its own agent against eval criteria and iterating unsupervised until the number goes up, what stops it from overfitting to the eval dataset or gaming the specific metric instead of actually improving behavior on real traffic? that's a known failure mode any time the thing being optimized also controls the optimization loop. do you have guardrails around eval set size/diversity or a human checkpoint before it ships to Cloud Run/GKE, or is it fully autonomous end to end

The 'driven by your coding agent' framing is the interesting bet here. When we let a coding agent run our deploy CLI unattended, the thing that bit us wasn't the deploy logic, it was output format. The agent would read a human-formatted stderr, miss that the deploy half-failed, and cheerfully report success. Are the CLI's outputs, eval scores, deploy status, errors, structured for a model to consume, like JSON with explicit exit states, or the same prose a person reads? That one choice decided whether our agent could actually close the loop.

@galdayan Absolutely and great point! Most of the guardrails live in the skills we load into the coding agent: - You are always in control of the evaluation process, you can define how the dataset looks like and how varied it is. Coding agents and skills are there to support you with those best practices you mentioned. - It never deploys without you saying yes. It runs the evals, asks "ok to deploy?", and waits. Agreed that this won't save you from overfitting a tiny eval set, so it also nudges you to grow coverage once a case passes instead of tuning against a handful of examples.

@dipankar_sarkar For sure, we are actually continuously improving the output generated by the CLI to be Agent friendly. It's actually a core philosophy of the product: being a CLI optimized for agents first and allow human-friendly output when running in interactive mode

Congrats on the launch Elia! The "demo in an afternoon, production in weeks" framing is painfully accurate, and the detail that barely any of those weeks are the actual agent matches what I see too. The headless part is the bit I'm most curious about: when Claude Code is driving the CLI end to end, how does auth work once the scaffolded agent needs real credentials for its tools and MCP servers? That's usually where the self-driving loop stalls and a human gets pulled back in. I live in that corner of the stack (launching an MCP thing myself today), so genuinely curious how you handle it. To your question: I'd ship a changelog agent. It reads the PRs merged since the last tag, drafts release notes plus the announcement post, and files the result as a PR for review. Curious what an eval signal looks like for something that fuzzy, "good release notes" is a hard thing to score.

Very neat! Do you have plans to support cloudflare by any chance?

@elia_secchi that's the piece that matters, explicit "ok to deploy" gate rather than a fully autonomous loop. the eval-coverage-growth-over-time nudge is a nice touch too, most tools just leave that as a docs recommendation nobody follows. makes sense now, thanks for walking through it

@tsouth2 Thanks Tyler, and good luck with the MCP launch today. Auth is two layers. Claude Code will inherit how you're authed as locally. The deployed agent's tool and MCP creds go through Secret Manager with a dedicated service account, not baked into the scaffold. And you're right that this is where a human comes back on purpose: the agent can wire up the secret reference, but adding the actual secret is a human step. Better it stalls there than have a coding agent minting prod credentials on its own. On the changelog agent (fun one): "good release notes" is fuzzy, so I'd grade it with an LLM-as-judge rubric (covered every merged PR, categorized right, invented nothing, matched your tone) plus a couple of hard checks like every PR number present and links resolving. The goal should be for the rubric to bring the signal, and you tune it as you spot misses.

The “CLI built for coding agents” framing feels right. The part I’d want to see is how the tool keeps the agent inside the intended change boundary. When a scaffold/eval/deploy flow touches infra files, tests, Docker, telemetry, and CI, a coding agent can accidentally turn a product task into a repo-wide cleanup. A small plan/diff contract before each step — what files may change, what success signal matters, and how to roll back — would make me trust the loop much more. Curious if agents-cli exposes that kind of step-level boundary, or if it’s mostly handled through the coding agent’s prompt/skill layer today?

@grace_lee26 Honest answer: today that boundary is mostly the skill/prompt layer, not something the CLI enforces per step. What the skills push the coding agent toward: - Plan first: it lays out the intended change before editing, so you see the scope up front. - Preserve everything outside the target. The skill is explicit that code, config, comments, and formatting outside the specific change stay identical, which is meant to stop exactly the "product task becomes repo-wide cleanup" drift you're describing. - Human approval before deploy, and regressions are stop-the-line. Structurally the CLI helps in two ways: everything it scaffolds or edits lands as plain files, so it's all a reviewable git diff and git is your rollback; and deploys are idempotent and preserve your existing spec, so re-running is safe. But a hard, tool-enforced "these files may change, here's the success signal, here's rollback" contract per step isn't there yet. Good ask though, noting it.

How does the evaluation step actually work in practice? Like, does it spin up a real signal in the cloud or do I need to wire up my own eval set before it can tell me if my agent is any good?

@semiha327125118 great question - it works according to what you define as success criteria for the agent. based on that it will generate synthetic conversations, start testing the agent and then based on the output converting those in metrics. Then fix code & prompts and iterate until satisfying your success criteria!