How Sourcedeck is built

Overview

Sourcedeck follows public accounts on several platforms, collects every new photo and video once, and delivers it to where creators work: team channels, folders or their own servers through webhooks. Customers pay monthly, invite their teams, and manage everything from a dashboard.

Three constraints shaped every decision:

• Correctness over cleverness. A creator with a deadline needs each item exactly once, on time. Duplicates and gaps are both failures.
• Other people's limits. Every platform limits how often it can be asked for anything. The system has to slow down gracefully, never hammer.
• Small team, real money. Customers pay and trust us with their accounts, so security can't be a later phase. It has to be cheap to run and hard to misuse.

The stack is deliberately boring: Python 3.12 with asyncio, aiohttp for the dashboard and API, SQLite in WAL mode, systemd services, and Caddy for TLS. Browser-based collection uses Playwright.

Architecture

Work flows left to right. Collectors check each source within its platform's limits and hand new items to the core, which stores each item once by its platform ID. The pacer then delivers each item to every matching destination, respecting each destination's own limits. Every hop is a row in a durable queue, so a crash or restart resumes exactly where it stopped.

Threat model

I used STRIDE to walk through each part of the diagram and ask how it could be misused. Each row below is a control that is built and visible in the product.

Exactly-once payments

Payment processors deliver events at least once. Networks retry, buyers double-click, and the same “invoice paid” event can arrive twice, seconds apart. If the handler isn't careful, that's two upgrades, or two months added to a plan.

The fix is to make the event ID the unit of work and let the database enforce it. The insert and the plan change happen in one transaction, so either both happen or neither does:

async def handle_event(db, event):
    verify_signature(event)                      # forged events stop here
    async with db.transaction():
        inserted = await db.execute(
            "INSERT INTO processed_events (id, type) VALUES (?, ?) "
            "ON CONFLICT (id) DO NOTHING", (event.id, event.type))
        if inserted.rowcount == 0:
            return "ignored"                     # seen before: nothing changes
        await grant_plan(db, event.workspace, event.plan, event.period)
    return "processed"

Outgoing charges carry an idempotency key built from the workspace and billing period, so a retried request can't create a second charge. You can see processed, ignored and rejected events in the operator console.

Sign-in and sessions

• Passkeys first. WebAuthn credentials are bound to our domain, so a look-alike page can't collect them.
• Passwords as a fallback. At least 12 characters, checked against common choices, stored with PBKDF2-SHA256 and a per-user salt.
• Lockout. 5 wrong passwords pause sign-in for that account for 5 minutes and alert the owner. Responses never reveal whether an account exists.
• Sessions that are useless if stolen from the database. The browser gets a random token; the server keeps only its hash.

token = secrets.token_urlsafe(32)                 # 256 bits, sent once as a cookie
store(session_id=sha256(token), user=user.id, expires=now + 30 * DAY)
# cookie: Secure; HttpOnly; SameSite=Lax; Path=/
# on each request: look up sha256(cookie), check expiry and role

Changing a password signs out every other device, and customers can see and end any session from Security.

Rate limits and backoff

Every outgoing call goes through a pacer with two buckets: one per destination and one overall. When a platform answers “slow down”, the worker backs off exponentially with full jitter, so many workers don't all retry at the same moment:

def next_delay(attempt, base=2.0, cap=600):
    return random.uniform(0, min(cap, base * 2 ** attempt))   # full jitter

Nothing is lost while a source backs off; the queue simply waits. Customers see a calm “Retrying” instead of an error, and operators see each platform's remaining budget on the health page.

The self-healing station

Some public pages only load properly in a full browser. Running that browser on a home machine worked far better than in the cloud, but a home machine is exactly where I didn't want an open port.

So the station never listens. It dials out to the core over SSH with its own key, speaks line-delimited JSON over that one connection, and the core sends it lookups as jobs. If the connection drops, it reconnects with backoff. If the browser freezes, it restarts it. If a page needs a human, it says so once instead of failing silently. A heartbeat every few seconds tells the core it's alive; silence for a minute means its work goes back to the queue.

Operator access

The most powerful accounts are the most valuable to attack, so the operator console assumes an operator session could be stolen:

• A fresh passkey check is needed every 12 hours, regardless of the session.
• Every sensitive action asks for a written reason, and links to a ticket where there is one.
• “View as customer” is read-only, ends after 30 minutes, and is visible to the customer.
• Suspending a workspace needs a second operator; nobody can approve their own request.
• Every action is appended to a hash chain, so a quiet edit to cover tracks is detectable.

Built with customers

Most features started as a support message. The clearest example: early users managed everything through chat commands, and the most common request was simply “give me a dashboard”. So I built one, and kept refining it with the people who had asked for it.

The same loop produced caption templates, watermarks, topic routing and team roles. Support conversations go straight to me, every “Was this helpful?” answer is read, and the changelog credits what customers asked for.

Running it

• Each component is its own systemd service on a small VPS, restarted on failure and on deploy.
• Secrets live in root-only environment files on the server, never in the repository.
• Schema changes only ever add columns or tables, so a rollback never meets a missing column.
• Backups run nightly and are kept for 30 days, readable only by the service account.
• A delivery health alarm messages me when delivery falls behind, before customers notice.

What I'd do next

Things I know are worth doing, and haven't done yet:

• Anchor the audit chain outside the server. A hash chain proves edits happened, but someone with full server access could rewrite the whole chain. Publishing the latest hash somewhere append-only closes that gap.
• Re-resolve webhook hosts at send time. Checking the address when it's saved isn't enough against DNS rebinding; the resolved IP should be checked again on every request.
• Lock the station's SSH key to one command. A forced command in authorized_keys, with port forwarding off, would limit what a stolen station key could do.
• Measure uptime and delivery as SLOs with external monitoring, and automate the monthly restore test.
• Encrypt backups with a key kept off the server.

Try it yourself

The dashboard and operator console on this site are interactive replicas of the real ones. They run entirely in your browser with sample data, so you can click everything safely: nothing you do leaves your browser.

• Sign in at /login/, then try a short password five times to see the lockout.
• Add a webhook pointing at http://localhost in Delivery and watch it get refused.
• Create an API key in Settings: it's shown once, then only its hash is kept.
• In the operator console's Security page, verify the audit chain, simulate tampering, and verify again.