Three Buttons and a Half Cent: Making Local AI Actually Useful on the Road

I’ve been running a local AI setup at home for a while now — Hermes Agent on a workstation, talking to a local model through vLLM. None of that is new to me. But this week I had a different problem to solve: could I actually use all of this while I was eleven hours away from home, sitting in a car with nothing but highway ahead?

The idea was simple enough. If I could get a working setup through Telegram, I could turn dead driving time into something productive. Instead of just staring at lane markers, I could research, brainstorm, and actually talk to my AI assistant on the road.

Getting there meant a few steps, and not all of them went the way I expected.

The headache came when I wanted to hook up the X search feature. It’s one of the newer additions to Hermes, and I was excited about it. I don’t use X much myself, but the idea of being able to search it on demand, through my AI assistant, felt like having a useful tool in my back pocket.

Finding where to put the API key, though, was a different story.

I went straight to the config YAML. That’s where all my settings live. That’s where I expected this to go — a nice clean variable I could inject through Docker. But that wasn’t where this particular credential lived. It turned out to be in a dot env file, in a mounted volume, somewhere else entirely. When you’re working on your own infrastructure from a hotel room, that kind of hunt is its own special kind of frustrating.

Once I tracked it down and got it set up though, the whole thing worked beautifully.

Why I Want X Search (Even Though I Don’t Use X)

Here’s the real reason I set this up. A couple days ago, one of my stock holdings took a sudden, sharp drop mid-day. No news articles anywhere. Nothing a regular search engine could find. I knew something had happened, but I couldn’t figure out what.

That’s the gap that X fills. By the time a news outlet writes an article and a search engine indexes it, the window for understanding what’s going on has already closed. But on X, people react immediately. Rumors, analyst takes, official statements — it’s all there in real time.

It’s not necessarily facts, and you have to take it with a grain of salt. But it gives you a sense of what’s driving the movement when nothing else will. That’s the kind of situational awareness I wanted my AI to have access to.

And the cost? I loaded five bucks onto the xAI account. After everything I’ve done, I’ve spent less than ten cents. Each search runs about half a penny. That’s hard to beat.

The Three-Button Dance

Once the research side was working, I wanted to tackle something else: could I actually talk to this thing while I’m driving?

The obvious answer seemed to be a phone line — send a call through Twilio, back to my home lab. But I have real reservations about opening inbound ports to my home network for a telephony service. That’s an attack surface I’m not comfortable with. I’d rather play it safe.

So I went with Telegram. I already had the integration set up. The question was whether it could handle voice.

It can. Sort of.

Here’s how it works: I press the dictation button in Telegram, talk, and it converts my voice to text. That becomes the message I send. Then Hermes responds with a voice message that plays back through my car speakers. The whole loop works.

But it’s not seamless. Right now it’s a three-button dance:

  1. Dictate
  2. Send
  3. Play

My phone is mounted right in front of me, plugged in and visible, so it’s manageable. But it’s not the truly hands-free experience I’m going for. I’m still working on that. I think there’s a better way, but this was the best I could put together before having a long stretch of road to test it on.

The Voice That Didn’t Always Come

Even with voice responses set up, at first the agent wouldn’t always use it. I’d ask for a voice reply and get text instead. I had to remind it, pretty regularly at first. But there’s something encouraging about the fact that it learned. After coaching it for a bit — telling it clearly what I wanted and when — it started picking up the pattern. Now when I need voice, I get voice consistently.

It’s not perfect, but it’s the kind of thing that gets better the more you use it.

Where This Goes

I’m not done with any of this. The three-button dance bugs me. I want something that works without touching the phone at all. And I want the whole system — research, image generation, conversation — running through a local interface I can actually trust.

But sitting here on a long drive, with a working setup that cost me less than ten cents and respects my security concerns, I’d say it’s worth it. The best technology is the kind that gets out of your way.

In my case, that means keeping my eyes on the road ahead.

Hermes Agent on GitHub • xAI API • Telegram Bot API