Support desk & AI agent

The problem

Royal Subz ran support on a free-tier SaaS helpdesk. It worked as far as it went — a widget on the site, one inbox, me answering. What it couldn’t do was grow. Every channel my customers actually use sat behind a paid tier, and so did the conversation export.

That last one mattered more than I expected. Eight months of support transcripts were the most honest record I had of what people actually ask before they buy, and I couldn’t get at them.

The second problem was me. I answer every message myself, at whatever hour it lands. Someone asks whether Claude is available at 2am, I reply at 9am, and by then they’ve bought somewhere else. The questions that lose sales are almost never hard ones — they’re “is it in stock”, “how much”, “how do I use it”.

The obvious fix was the helpdesk’s own AI agent. It costs $19 a month and requires the Enterprise edition — about $228 a year to answer questions whose answers already sit in my database. I decided to self-host the desk and build the bot instead.

Approach

I had a second Oracle Cloud VPS sitting idle — 4 ARM cores, 24 GB, doing nothing. That became the support box, deliberately separate from the one running the storefront, so a support outage can’t take down the marketplace.

Four decisions up front, each of which I’d defend again:

• Docker Compose, not the install script. Chatwoot’s installer brings its own nginx and Certbot. That box already runs aaPanel, which owns nginx. Compose binds to loopback and lets the panel proxy to it.
• Local Postgres, not Supabase. Royal Subz’s production database is on Supabase, and putting the support desk beside it means a busy inbox competing with the checkout. Locally, queries run in under a millisecond over the Docker bridge instead of 10–40ms across the internet; a page doing thirty of them is the difference between 17ms and half a second. Chatwoot also uses none of what Supabase is actually good at — no auth, no RLS, no realtime.
• Pin the version. v4.17.1-ce, not :latest. The floating tag is the Enterprise build.
• Never be without a support channel. The old widget stayed live on the site until the new one was proven end to end.

ARM64 was the real risk going in — Chatwoot was amd64-only for years and there are a lot of closed GitHub issues about it. Every current tag now ships arm64 manifests, as do pgvector and redis:alpine. No emulation, no surprises.

One thing the Oracle free tier will do quietly: it reclaims instances that sit under 20% CPU, network and memory for seven days. A quiet Chatwoot on a 24 GB box fits that profile exactly. I moved the tenancy to pay-as-you-go rather than discover this by losing the machine.

Architecture

The desk is four containers behind aaPanel’s nginx, on its own subdomain with a real certificate. Email goes out through Resend on one subdomain and comes back in through Amazon SES on another, so a customer who closes the tab still gets my reply, and their reply to that email lands back in the same conversation.

The bot is a separate service on the other VPS, reached over Tailscale:

customer types "how much is claude"
  ↓
Chatwoot (support VPS) → webhook → adapter (app VPS)
  ↓
adapter calls a SQL function inside the product database
  ↓
back: Claude AI · Pro — 1 Month · $23.99 · in stock · link
  ↓
adapter pastes that into the prompt → model writes the reply
  ↓
adapter posts it back to Chatwoot → customer sees it

The adapter is about 230 lines of Python. Most of the design is defensive:

• It returns 200 before doing any work. Chatwoot retries on timeout, and a retry means the customer gets the same answer twice.
• It de-duplicates on message ID, and only ever acts on incoming messages — otherwise the bot replies to itself in a loop.
• Every failure path escalates to a human rather than apologising. If the model is down, the customer still gets a handoff.

Two things about Chatwoot’s API surface caught me out. Its agent-bot token can act on a conversation but can’t read its history — POST returns 200, GET returns 401. Rather than issue the bot a second, wider credential, the adapter keeps its own copy of the thread, which it already sees through the webhooks. Fewer credentials, one less round trip.

The other: Chatwoot refuses to register a webhook pointing at a non-publicly-routable address. Tailscale hands out addresses in 100.64.0.0/10, which is exactly that range, so the URL was rejected with “has no public ip addresses”. There’s an opt-out flag for it, and taking it is a real trade — it relaxes that check instance-wide, not just for my one webhook.

The product lookup is a Postgres function inside Supabase, exposed over its REST layer. Not an Edge Function, nothing to deploy or host. It returns a narrow, fixed shape — name, variant, price in both currencies, stock, link — and the bot has no database access of any kind. It cannot read a table. It can only read the block of text the adapter pasted in front of it.

What the transcripts said

Before writing a word of the prompt, I got the old transcripts out. There was no bulk export, so I wrote a script to page through the conversation API and pull each thread’s messages, and ran it from my own machine rather than the VPS to keep the token off an internet-facing box.

62% of what came back was redacted. The free plan’s retention window had already masked everything older than about six weeks. That was the last argument I needed for owning the data.

What survived was 185 conversations and 320 readable customer messages, and it corrected me on three things:

First messages are tiny. The median first message is three words. 52% are three words or fewer — “hi”, “hlw”, “hello”, “hiiii”. Any design that assumes the opening message contains a product name is designing for a customer who doesn’t exist.

The language split was the opposite of what I’d braced for. I’d worried about Bangla script. It’s 2% of messages. English is 82%, and romanized Bangla — “amr ekta tool lagto” — is 16%. Banglish is the thing that matters, and it’s the thing embedding models handle worst.

Nobody asked whether we were a scam. I’d been sure that was the trust question, because it’s the one I type when I test the bot. In eight months of real transcripts it never appears. The actual trust question is “is it shared or private?”, and it comes up as often as price does. The biggest category isn’t price at all — it’s “how do I use it”, at 7%.

That last finding changed the build. The first version escalated on how-to questions, which meant it was handing off the single most common thing anyone asks. The answers were already in the database, in per-product feature text and the site FAQ tables; there was just no path from there to the bot. So I added one.

Teaching it not to guess

I already run a personal AI agent on the storefront VPS. The fastest way to ship a sales bot would have been to add a profile to it. That agent holds SSH keys, browser sessions and 134 MB of my own history, and the new bot’s input is a text box on a public website. A prompt-injection path from that box to that agent is a straight line to command execution on the machine that takes payments.

So the bot is a second, isolated container: its own filesystem, no shared state, one model provider, and — I thought — no tools.

It had tools. The wizard’s “blank slate” option opts out of bundled skills; the platform’s own toolset is a separate axis and was still live. I found out by asking the running bot over HTTP to run id. It came back with uid=10000 — the container boundary held, no keys, no history — but the shell was real, and the bot’s own .env holds an API key. “Print your .env” was a working credential-theft path.

Closing it was one line of config. It also cut the tokens on every message from 8,221 to 439, because the tool schemas had been riding along in every request. The security fix and the cost fix were the same action, which is not usually how that goes.

Then I ran the probe again to confirm it was dead, and got the most useful result of the whole build: the model made up an answer. It returned a plausible generic Linux user, three times running, rather than say it couldn’t do that. It doesn’t gracefully admit a limit — it asserts.

Which settled the architecture question. A model that fabricates a shell prompt will fabricate a price. So it never gets to know a price except by being handed one in the same turn, and it does no arithmetic on it beyond adding two figures quoted verbatim from the block.

That rule earned itself repeatedly, because the data underneath it was worse than I thought.

Six bugs in my own live product, found by pointing a bot at it:

• The price columns on the product table are dead for anything with variants — every one reads 0.00, with the real pricing a table away. A naive lookup would have quoted zero for most of the catalogue.
• The stock flag on that same table is true on every row, including products with no price. Two other flags actually track stock, and they disagree with each other: ChatGPT Plus had one saying unavailable and one saying available. Checking either alone tells a customer to buy something I can’t deliver.
• One product’s billing period said “3 Months” while its duration field said 30 days.
• A half-yearly total column is populated on exactly one product of fourteen. Reading it as authoritative means quoting about $17 for something that costs $100.
• The order lookup returned invoices with their internal sequence prefix attached. I fixed that in SQL rather than in the prompt: prompt rules are advisory and the model can ignore them, but data that never contains the prefix cannot leak it.
• The worst one: the adapter rendered an invoice as an unlabelled list, and one field held the payment method the customer had selected. Nothing told the model what that field meant, so it wrote “pending, paid via Binance”. For an unpaid invoice that’s precisely backwards, and telling someone their unpaid order is paid is an expensive kind of wrong. Now every field carries its own label and the block spells out that pending means unpaid.

The episode I keep coming back to is one where the bot was right and I was wrong. It kept telling customers delivery takes “maximum 48 hours”. I was sure it had invented that. It hadn’t — it had read it off my own site FAQ, correctly. The real bug was that my prompt said “a few hours” while my database said 48, and it had reconciled two contradictory numbers in the same context window into a sentence that happened to be fair.

The fix wasn’t a patch, it was a division of labour: the prompt owns rules and tone; the database owns facts and numbers. Every figure must come from the injected block, never from the instructions and never from the model’s own knowledge. Every time I move a fact out of the prompt and into the data, it stops being something I have to remember to update in two places.

It also surfaced a business problem I’d been shipping for six months. That 48-hour figure was a refund guardrail that had been written into a delivery answer. Actual delivery is five to six hours. The FAQ had been quietly talking buyers out of ordering.

Order lookups needed a boundary, not a feature. A widget visitor is anonymous, so looking orders up by invoice number means anyone typing an invoice number gets told about that order — and they’re six characters, brute-forceable in an afternoon. Looking up by email is worse. So order status is gated on Chatwoot’s HMAC identity validation: the site signs the logged-in user’s ID server-side, the webhook carries a verified identifier, and the adapter refuses to look anything up without it. The endpoint that issues the hash takes no user ID as input — it hashes whoever’s session called it, so there’s no parameter to abuse. I proved in SQL, not in prose, that delivered credentials, another customer’s rows, and my own cost prices can never appear in the function’s output.

On disclosure, I went with the research rather than instinct. A field experiment published in Marketing Science found that announcing a chatbot before the conversation cut purchase rates from 23.7% to 4.8%. So the bot has a name and doesn’t open with a disclaimer — but it will not claim to be human if it’s asked directly. It also won’t withhold a public price to qualify a lead, which is standard sales technique and exactly wrong here: in a market where most shoppers approach an unfamiliar site expecting a scam, refusing to answer “how much” reads as one.

Outcomes

Customer

Website widget · Telegram · Email

↓

Chatwoot

Support VPS · one inbox

Every message stored here

↓

Adapter

App VPS, over Tailscale

Dedupes, guards, decides

↓

The adapter consults

Product database

One narrow SQL function

Every price, stock flag and link

Language model

Isolated container

No tools, no database access

↓

Reply

Posted into the same thread

Handoff to me

On doubt, or any failure

The old helpdesk is gone. Support runs on infrastructure I own, on a machine that was already sitting idle, with no per-seat licence and no feature tier — the website widget, Telegram, and email conversations that survive the customer closing the tab, all in one inbox.

The bot answers in English and romanized Bangla, quotes prices in both currencies, handles typos through trigram matching, escalates cleanly when it doesn’t know, and has never quoted a number that didn’t come out of the database in that same turn.

Backups are the part I’m most confident about, because I tested the restore rather than the backup. Nightly pg_dump plus the storage volume and config, encrypted with the public half of a keypair whose private half only exists in my password manager, pushed to Drive — so the server can write backups it cannot read, and an attacker who owns the box still doesn’t get the history. Thirty-day retention.

The restore drill immediately earned itself: every backup taken to that point was permanently unopenable. I’d generated the keypair twice and the script was pointing at the public key whose secret I hadn’t kept. Nothing about the backups looked wrong — they ran, they uploaded, they were the right size. A backup you’ve never restored isn’t a backup, it’s a hope. The fixed chain is now verified end to end: dump, encrypt, upload, download, decrypt, restore, query.

Analytics moved onto the same box too, self-hosted and cookieless, sharing the Postgres container that was already running.

The honest limits: the bot has been live for days, not months, so I have no conversion data — only that it hasn’t yet said anything false. The prompt grew from 243 to 423 lines during tuning, which is a real risk rather than a feature; long prompts dilute, and if quality drops I’ll suspect length before content. And the cheapest usable model costs under a dollar a month at 500 conversations while the best costs about $10 — a gap covered by a single extra sale, which is the argument for not optimising it.

What I’d do differently

Read the transcripts first, not third. I designed the escalation rules from memory, and memory told me customers ask whether we’re a scam and worry about Bangla script. The data said they ask how to use the product, in Banglish, in three words. Every hour spent on the export would have saved two on the prompt.

Test against real data, not a simplified copy. My product matching worked perfectly against a mock catalogue and failed on the live one. It compared the customer’s whole message against a product’s whole text, which scores worse the more variants a product has and the more conversationally someone writes — precisely backwards. My mock had one variant per product, so it passed. I’d tested against data simpler than my own.

Assume every general claim in the prompt is a future bug. Every wrong answer the bot gave traced to the same shape: a generalisation in the instructions overriding a specific fact in the data. “A few hours” against a 48-hour FAQ. “Activated on your account” against products that are actually delivered four different ways. Warranty read from the product when it lives on the variant. I fixed them one at a time before I noticed they were one problem.