How Hermes uses a secret

I run Hermes every day. This software has changed how I work and how I keep personal things moving.

However... Once you let an agent do real work, it needs keys & passwords.

Then the question shows up fast: how does your agent use passwords, and how do you keep those passwords out of a chat that some AI company might store? Can you even do that?

Spoiler. You can. Sometimes. But not all the time.

Most of us already know not to paste a Stripe key into ChatGPT. Using an agent feels different, because it sits on your computer and needs keys to do anything useful. The mix-up starts when we treat every secret like the same object.

It is not. A secret has a job.

Hermes can use a secret in three ways

When you have a secret, ask which job it is doing.

To run. Hermes needs this to think and talk. That means model keys, chat app integrations, and whatever keeps the agent alive.

To call a tool. A CLI, an MCP server, or a company API needs this. Hermes is asking another program to work. Stripe, GitHub, DigitalOcean.

To sign in as you. A browser needs this to open a website in your name. Username, password, extra locks.

Do not put the key in the chat

A password pasted directly in your chat history can get stored in a vendor log, used as training data, or stolen from a server you do not control.

The better path for a tool key is a proxy or relay. Hermes says “I need the Stripe key,” rather than you providing it. Hermes uses a proxy, which adds the Stripe key out of the context window. The key is used and the model sees the results, but it never reads the key.

For tool keys, the relay that exists today is Treg (I use Treg in my own setup, but this is not sponsored post). You say you need Stripe or DigitalOcean. Treg puts the key on the HTTP request. The model sees the result. It does not see the key.

A password manager is not a relay

You can keep that Stripe key in 1Password or Bitwarden. Keeping it in a password manager and never showing it to the model are two different things.

If your agent is talking to a password manager, that does not mean the password stays hidden. Without the right setup, the agent can retrieve the secret and inject it into the chat. The model then has it, the same as if you pasted it.

Hermes has built-in password manager support. When the agent starts, it can pull keys from 1Password or Bitwarden so you do not have to keep a list of keys on the computer Hermes is running on. That is useful. It does not mean those keys stay out of the model.

What relays actually work

Go back to the three jobs. The relay question is different for each one.

To run. These are the keys Hermes needs to think and talk: the model key, chat app tokens, whatever keeps the agent alive. There is no relay in the middle. You put those on the machine Hermes runs on, either in its own file or pulled in from a password manager at start. The agent has to have them to function.

To call a tool. API calls, MCP, CLI. This is where a relay actually exists. Treg can attach the key on the HTTP request so the model sees the result and not the key. You can self-host Treg, which means the secrets stay on hardware you control. You can also use their hosted version. That is simpler. It also means the keys sit on someone else’s server. You are trusting their security. That is not automatically a terrible option. It is a different risk than a key sitting in the chat.

To sign in as you. Website passwords. There is not a mature relay I would recommend yet for filling a login without the model reading the password. You can still use them. It does not mean you will get hacked. It does mean you should know the risk: if the agent can fetch the password, it can land in the context window.

What to do about website passwords

Until a relay exists for this job, treat website logins as the risky case and shrink the blast.

Give the agent its own logins when the site lets you, not your personal account. Scope those accounts so they can only do what the agent needs. Rotate the passwords regularly. Do not hand it your bank login and the kids’ school portal in the same pile as a Stripe key.

Keep in mind, that using a smaller door does not make the model blind. It means the opening is smaller.

What if you do not want a password manager

You do not have to use a password manager. Hermes can read a simple text file on the computer it runs on. Developers call that a .env file.

In an .env file, each line is a name and a secret, like STRIPE_KEY=sk_live_…. When Hermes starts, it reads the file and has those keys for the session.

That is not sloppy by default. If the agent can use a key, it can expose a key, whether the key lived in 1Password, Bitwarden, or a file next to Hermes. A local password manager does not change that. The agent still has access.

What you should actually do

A .env file is not the end of the world. For a lot of setups it is about as secure as storing the same keys in a password manager, even a Password Manager you host on your own machine (I self-host VaultWarden), because the agent can reach them either way.

If you want API calls to happen without the model reading the key, use a relay like Treg. Or keep those keys in the .env file and be honest that they can show up in the context window.

The thing that actually keeps secrets off a company’s servers is not running the model in the cloud. Yep, it's Local AI.

Local AI means the conversation never leaves the building. That is the real security jump. Very few of us can afford local inference that is anywhere close to frontier models, or even the big open models that need a lot of hardware.

So most of us will keep using a cloud model, keep tool keys on a relay when we can, keep website passwords scoped and separate, and not pretend a password manager is doing the relay’s job.

If writeups like this are useful, the newsletter at daveswift.com is the easy way to get the next one.