---
title: 'Building GroceryTrack: From a Pile of Receipts to Users I Never Expected'
seoTitle: 'Building GroceryTrack, an AI Receipt Scanner'
author: 'Xinjiang Shao'
date: '2026-06-12'
draft: false
fullWidth: true
description:
  'How I went from idea to a live receipt-scanning app in one day, and what 444 receipts, the
  first 100 users, and an unexpected Canadian audience taught me about building in the LLM era.'
tags:
  [
    'Side Project',
    'LLM',
    'OCR',
    'Firebase',
    'Flutter',
    'Product',
    'Indie Hacking',
  ]
coverImage: './images/cover.jpg'
seoDescription: 'How I took a receipt-scanning app from idea to live in one day, and what 444 receipts, the first 100 users and a Canadian audience taught me.'
---

On March 13, 2026, I got annoyed enough at my grocery receipts to do something about it. On March
14, v1 of [GroceryTrack](https://grocerytrack.food/) was live. Three months later, the app has tracked
**444 receipts, 3,012 items, and $41,435.91 in spending across 138 distinct stores** — for its
first hundred-or-so users, including a cluster of Canadians I never planned for.

This post is the story of why I built it, how it's architected, and what real users taught me along
the way.

## The Problem

### Scratching my own itch

I wanted to answer embarrassingly simple questions: How much do I actually spend on groceries each
month? Is Costco really cheaper than the local supermarket for the things I buy? Did egg prices go
up, or does it just feel that way?

Budgeting apps couldn't answer these. They see a single line — `COSTCO $187.43` — and stop there
([bank statements aren't enough](https://grocerytrack.food/blog/why-bank-statements-arent-enough)).
The interesting data lives _inside_ the receipt: the items, quantities, and unit prices. For years,
the only way to get it was manual entry into a spreadsheet, which I tried and abandoned twice.

### Why now: OCR finally works

Receipt digitization isn't a new idea. But traditional OCR pipelines were brittle: every store
formats receipts differently, thermal paper fades, items are abbreviated beyond recognition
(`ORG BNLS SKNLS CKN BRST`, anyone?). Building per-store templates was the old approach, and it
never scaled.

Multimodal LLMs changed the economics overnight. A model that can _read and reason_ about a crumpled
receipt photo — inferring that `ORG BNLS SKNLS CKN BRST` is organic boneless skinless chicken breast
and categorizing it as poultry — turns a months-long parsing project into a well-crafted prompt.
That was the unlock that made a one-day MVP possible.

GroceryTrack sends the receipt photo **directly to Gemini's multimodal API** — no separate OCR step,
no text-extraction pipeline, no per-store templates. One prompt, one image in, structured JSON out:
store name, date, line items with quantities and prices, and item categories. The entire "parsing
engine" that would have taken months to build in 2020 is a model call plus careful prompting and
output validation.

### Who it's for

Four personas kept showing up, in my head and then in the data:

**Avery Chen — the power user.** Urban professional, solo household, quality-first. ~$1,050/month
on food, ~22 trips at a $47 average basket, spread across an astonishing 37 distinct
stores. Wants rich analytics and
quality-for-price comparisons without giving up flexibility. _"Show me where my money goes — I'll
spend on quality, I just want to see it clearly."_

**Sam Rivera — the budget-anchored planner.** Household of 3–4, cost-pressured, plan-driven.
~$650/month, 6 big trips, ~$110 baskets at 2–3 stores. Wants to stay under a set budget and get
warned _before_ overspending, not after. Pain points: end-of-month surprises and quiet price creep
on staples. _"Tell me when I'm about to blow the budget — this week, not next month."_

**Maya Brooks — the wellness optimizer.** Active, label-reading, macro-aware. ~$800/month across
~4 stores. Wants nutritional quality visible alongside spend — supplements and organics balloon
the bill, and no app shows nutrition across receipts. _"I track what I eat as closely as what I
spend — show me the nutrition behind the basket."_

**Leo Martin — the waste-conscious steward.** Sustainability-minded, fridge-inventory thinker.
~$520/month, 2–3 stores. Wants to use everything before it spoils and buy only what gets eaten;
feels real guilt over tossed food. _"Flag what's about to go off so I cook it before I lose it —
wasted food is wasted money."_

Notice these four don't want the same product: Avery wants analytics, Sam wants alerts, Maya wants
nutrition, Leo wants expiry tracking. But they all start from the same raw material — the receipt —
which is why the core loop comes first and everything else grows out of it.

## The Solution

The initial idea was a single, simple loop — turn a paper receipt into structured data, then turn
that data into insights:

```mermaid
graph LR
    A["Capture<br/>snap a receipt photo<br/>(web or mobile)"]
    B["Digitize<br/>Gemini multimodal<br/>unstructured image → structured JSON<br/>store · date · items · prices"]
    C["Visualize<br/>spending trends · category breakdown<br/>store & price comparison"]

    A --> B
    B --> C

    style A fill:#dbeafe,stroke:#1e40af,color:#1e40af
    style B fill:#dbeafe,stroke:#1e40af,color:#1e40af
    style C fill:#dbeafe,stroke:#1e40af,color:#1e40af
```

Everything else in the architecture exists to serve this loop with as little friction as possible.

### Three ways to capture a receipt

Capture is where the loop succeeds or dies, and it now has three entry points — each one born
differently.

**1. Scan it.** The MVP: point your camera at a paper receipt, or upload a screenshot of a digital
one. The surprise here was language. Users from the Middle East showed up with zero marketing
effort and just... used it. I have no way to verify extraction accuracy for languages I can't
read, but I tested Chinese grocery receipts myself, and the extractions were impressive —
multilingual receipt parsing was a feature I never built; it came free with the model.

**2. Snap the groceries (GrocerySnap).** A friend's farmers market trip exposed a gap: market
stalls don't give you itemized receipts, just a total. So GrocerySnap lets you photograph the
groceries themselves — the model identifies the items, estimates prices based on US market rates,
and the user edits anything that's off. Less precise than a receipt, but it captures spending that
would otherwise vanish.

**3. Forward the email.** A group of friends pointed out that a household's receipts scatter
across multiple email addresses. Borrowing from how corporate expense tools work, email forwarding
lets anyone in the household forward digital receipts to a dedicated Resend inbound address, and the
items land in the shared spending view automatically.

Only the first one was my idea. The other two came directly from watching real people try to fit
the app into their actual lives.

### Web first, on a deliberately boring stack

The web app is the core of GroceryTrack, and every layer is managed infrastructure:

| Layer            | Technology              | Purpose                                                  |
| ---------------- | ----------------------- | -------------------------------------------------------- |
| Frontend — Web   | Next.js (React)         | Web application framework; SSR + CSR                     |
| Frontend — Mobile | Flutter                | Mobile receipt capture and on-the-go access              |
| Authentication   | Firebase Authentication | Google OAuth; session management                         |
| Database         | Cloud Firestore         | NoSQL document storage for accounts, receipts, budgets   |
| File Storage     | Google Cloud Storage    | Receipt images and AI-generated images                   |
| AI / ML          | Google Gemini API       | Receipt text extraction, item categorization, name cleaning |
| Payments         | Stripe                  | Subscription billing; PCI-compliant card processing      |
| Email            | Resend                  | Transactional email — inbound receipt forwarding, weekly/monthly summaries |
| Analytics        | Firebase Analytics      | In-app usage event tracking                              |
| Error Monitoring | Sentry                  | Runtime error tracking; session replay                   |
| Infrastructure   | Google Cloud Platform   | Hosting, compute, storage — US region                    |

Nothing in this table is exotic, and that's the point. Sentry paid for itself the first week —
receipt parsing fails in creative ways, and I wanted to see every failure rather than wait for
users to report them. Shipping v1 in a day was only possible because every piece of infrastructure
was managed: the interesting work was the product, not the plumbing.

Here's how the pieces fit together:

```mermaid
graph TD
    subgraph CLIENTS["Clients"]
        IOS["iOS app<br/>(Flutter)"]
        AND["Android app<br/>(Flutter)"]
        WEB["Web app<br/>(Next.js)"]
        AIA["Claude · ChatGPT<br/>(MCP clients)"]
    end

    subgraph API["API layer"]
        RCPT["Receipt API<br/>scan · parse · items"]
        ACCT["Account API<br/>auth · profile"]
        SUB["Subscription API<br/>billing · Stripe webhooks"]
        REF["Referral API"]
        MCP["MCP Server"]
        OAUTH["OAuth APIs<br/>token & consent"]
    end

    subgraph DATA["Data layer"]
        FS[("Firestore<br/>accounts · receipts · budgets")]
        GCS[("Cloud Storage<br/>receipt & AI images")]
        GA["Firebase Analytics"]
        BQ[("BigQuery<br/>analytics")]
    end

    SCHED["Cloud Scheduler<br/>batch jobs · email & push"]
    STRIPE["Stripe"]
    RESEND["Resend<br/>email send & inbound"]
    BI["Claude · ChatGPT · Gemini<br/>(chat-based BI)"]

    IOS --> RCPT
    AND --> RCPT
    WEB --> RCPT
    IOS --> ACCT
    AND --> ACCT
    WEB --> ACCT
    WEB --> SUB
    WEB --> REF
    AIA --> OAUTH
    AIA --> MCP
    STRIPE -->|webhooks| SUB
    RCPT --> FS
    RCPT --> GCS
    ACCT --> FS
    SUB --> FS
    REF --> FS
    MCP --> FS
    OAUTH --> FS
    GA --> BQ
    BQ --> BI
    SCHED --> FS
    SCHED --> BQ
    SCHED -->|summary emails| RESEND
    RESEND -->|forwarded receipts| RCPT

    style IOS fill:#dbeafe,stroke:#1e40af,color:#1e40af
    style AND fill:#dbeafe,stroke:#1e40af,color:#1e40af
    style WEB fill:#dbeafe,stroke:#1e40af,color:#1e40af
    style AIA fill:#dbeafe,stroke:#1e40af,color:#1e40af
    style RCPT fill:#fef3c7,stroke:#92400e,color:#92400e
    style ACCT fill:#fef3c7,stroke:#92400e,color:#92400e
    style SUB fill:#fef3c7,stroke:#92400e,color:#92400e
    style REF fill:#fef3c7,stroke:#92400e,color:#92400e
    style MCP fill:#fef3c7,stroke:#92400e,color:#92400e
    style OAUTH fill:#fef3c7,stroke:#92400e,color:#92400e
    style FS fill:#d1fae5,stroke:#065f46,color:#065f46
    style GCS fill:#d1fae5,stroke:#065f46,color:#065f46
    style GA fill:#d1fae5,stroke:#065f46,color:#065f46
    style BQ fill:#d1fae5,stroke:#065f46,color:#065f46
    style SCHED fill:#fee2e2,stroke:#991b1b,color:#991b1b
    style STRIPE fill:#fee2e2,stroke:#991b1b,color:#991b1b
    style RESEND fill:#fee2e2,stroke:#991b1b,color:#991b1b
    style BI fill:#dbeafe,stroke:#1e40af,color:#1e40af
```

### Going native with Flutter

Capture friction makes or breaks a receipt scanner. The web app works, but "open browser, log in,
upload photo" loses to "open app, point camera." So the native app came next — **Flutter**, one
codebase for both platforms: iOS shipped April 30, Android beta followed on May 10.

For a solo builder, Flutter's trade-off is easy to accept: slightly less platform-native feel in
exchange for not maintaining two codebases. Camera capture, the one flow that truly matters, feels
first-class on both.

### Real-time API vs. batch jobs

Processing splits into two paths:

```mermaid
graph TD
    U["User scans receipt"] -->|"synchronous · latency-sensitive"| API["Real-time API<br/>Gemini parse → Firestore<br/>results in seconds"]
    SCHED["Cloud Scheduler"] -->|"scheduled · fan-out"| BATCH["Batch jobs<br/>weekly & monthly email summaries<br/>push notifications"]
    API --> DB[("Firestore")]
    DB --> BATCH

    style API fill:#dbeafe,stroke:#1e40af,color:#1e40af
    style BATCH fill:#fef3c7,stroke:#92400e,color:#92400e
```

- **Real-time API**: when a user uploads a receipt, parsing happens immediately — they're standing
  in their kitchen waiting to see their items appear. Latency matters here, so this path is
  synchronous and optimized for a fast first result.
- **Batch jobs via Cloud Scheduler**: communication that doesn't need to be instant — **weekly and
  monthly email summaries and push notifications** — runs on a schedule instead, with summary email
  delivery handled by Resend.

The split is about scalability and concurrency. Sending a hundred summary emails today is trivial;
sending 50,000 from a request handler would not be. Cloud Scheduler lets the fan-out work run at
its own pace, in controlled batches, completely isolated from the latency-sensitive scan path. The
interactive API stays lean, and a notification backlog can never slow down someone scanning a
receipt in their kitchen.

One more architectural bet: on April 17, I shipped an
[**MCP server**](https://grocerytrack.food/mcp) for GroceryTrack — so users
(and their AI assistants) can query their own grocery data conversationally. "What did I pay for
eggs in March?" is exactly the kind of question this data exists to answer, and increasingly, people
ask it through an agent rather than a dashboard.

Here's what that looks like in practice. I pointed Claude at my GroceryTrack MCP server and asked
it to design a weekly meal rotation from my actual purchase history — the clams, mackerel, and
yogurt I really buy — while factoring in my health goals (B12, dairy, low sodium) and my swim
schedule. The result was "The Rotation 五道菜": five dishes, one week, in both English and
Chinese, down to batch-prep notes timed around pool sessions. No food diary, no manual export —
the receipts were the input.

- [The Rotation — five dishes, one week (English PDF)](/content/blog/2026-06-12-building-grocerytrack/the-rotation-en.pdf)
- [五道菜 · 一周轮换食谱（中文 PDF）](/content/blog/2026-06-12-building-grocerytrack/the-rotation-zh.pdf)

This is the payoff of structured grocery data: it stops being a spending report and becomes raw
material for whatever question you — or your AI — want to ask next.

The same pattern runs on the operator side. Firebase Analytics events flow into BigQuery, and
instead of building dashboards, I connect Claude, ChatGPT, or Gemini directly to it for
**chat-based BI** — "which capture method do new users try first?" gets answered with a query and
a chart, on demand. Between the MCP server for users and BigQuery-plus-LLM for me, I think this is
a preview of how business intelligence works from now on: no fixed dashboards, just dynamic ones
materialized from whatever question is being asked.

## The Learnings

### Friends are the right first users

The MVP's first users were friends. That's not a growth strategy — it's a correctness strategy.
Friends forgive rough edges but tell you the truth: which receipts failed to parse, which numbers
looked wrong, which screens confused them. Every early parsing bug was found by someone I could
text back within the hour.

Friends are also my qualitative research lab. I can spend an hour on the phone with one of them
just talking through what they like and dislike — not a survey, not an analytics funnel, a real
conversation. Those calls are where the different perspectives surface: the detail a dashboard
would never show, the feature request hiding inside a complaint. GrocerySnap and email forwarding
both started as phone conversations, and the four personas above are distilled from exactly this
kind of listening.

Launching on Product Hunt on March 24 — ten days after v1 — brought the first strangers. Strangers
are different: they don't tell you what's broken, they just leave. The Sentry dashboard became my
proxy for the feedback strangers never send.

### The iOS launch changed the user base

Numbers stayed modest until the iOS app shipped on April 30. Mobile is where receipts happen — you
scan in the kitchen, not at a desk. Today the platform is approaching its **first 100 registered
users**, with a core of weekly active scanners averaging **5.8 scans per month** each.

Small numbers, and I share them deliberately. A handful of weekly active users sounds like nothing
until you realize they're strangers voluntarily photographing their receipts several times a month.
For a three-month-old solo project, retention from people I've never met is the signal I care
about — growth can be bought, habit can't.

### Users tell you about their receipts — literally

The most valuable feedback channel turned out to be users sending me their actual receipts:
"Here's one your parser got wrong." Each one is a test case. The parsing pipeline today is the
accumulation of dozens of these donated edge cases — faded thermal paper, multi-language receipts,
stores that print discounts as negative line items.

My favorite example came from Sabrina, one of my early users: her receipt printed its date as
`26/05/13` — **YY/MM/DD**, meaning May 13, 2026. The model assumed **DD/MM/YY** and quietly filed
the purchase under a completely different date, years off. A human glances at a grocery receipt and
never doubts the date; a model needs to be told that receipts come from the recent past and that
date formats vary by store and region. That one report led to date validation and format-aware
prompting that fixed the issue for every user since.

### Surprise: Canada showed up

The biggest surprise in the data: among the
[138 distinct stores tracked](https://grocerytrack.food/store) (80 grocery stores — the rest are
restaurants, another use I never designed for), **a noticeable cluster originates from Canada**. I never marketed there — the receipts just started showing up. My best
guess is a mix of grocery inflation being a particularly hot topic in Canada and receipt-scanning
habits from cashback apps priming the behavior. Canadian users also brought Canadian receipts —
bilingual French/English layouts, different date formats (see Sabrina's bug above), and store
chains the parser had never seen — which made the product better for everyone.

The lesson: your users decide what your product is for. I built a personal spending tracker; the
data says I'm also building a grocery-inflation lens for Canadians. Listening to that beats any
roadmap I would have written alone.

## The Other Side of the Coin: Three Technical Hypotheses

Beyond being a product, GroceryTrack is also my running experiment on three hypotheses about
AI-built software — each one a claim I wanted to confirm or bust with real evidence. Three months
in, here's where that evidence points.

### Hypothesis 1: AI with LLM is good enough for production

**Verdict: confirmed — but only with guardrails around the model.** Gemini reads receipts better than any
OCR pipeline I could have built, yet "good enough for a demo" and "good enough for production"
are separated by everything wrapped around the model call: schema validation on every response,
sanity checks (a receipt's date must fall in the recent past — see Sabrina's bug), retry logic for
malformed output, and Sentry watching every failure. The model is the engine; production-readiness
is the seatbelt, airbags, and dashboard warning lights. None of the 444 receipts were parsed by a
model alone — they were parsed by a model inside a system that assumes the model will sometimes be
wrong.

### Hypothesis 2: The more work AI does, the better the build economics get

**Verdict: confirmed — at the ~90% tier, the economics invert entirely.** The question behind it:
what does development actually cost when AI does 10%, 50%, or 90% of the work?

GroceryTrack sits at the high end of that spectrum: most of the code was written with AI doing the
heavy lifting and me directing, reviewing, and correcting. The bill for that leverage is concrete.
My Claude invoices tell the story of the build in dollars:

- **March: $231.81** — nearly all of it ($226.06) in a single burst during the first week of
  building, the idea-to-Product-Hunt sprint.
- **April: $207.00** — four steady weekly invoices of $51.75 each, the rhythm of ongoing feature
  work (MCP server, iOS launch).
- **May: ~$210** — the same steady cadence through the Android beta.

On top of that sit two flat subscriptions: **ChatGPT at $20/month and Gemini Pro at $4.99/month**
— call it ~$240/month all-in. Roughly **$725 of AI tooling over three months** to ship a
three-platform product.

"AI does 90%" isn't one tool — it's a toolchain covering the entire product lifecycle, not just
code:

- **Coding (three LLMs in rotation)**: Claude Code as the primary agent, OpenCode running on Gemini
  tokens, and Codex. Different models have different strengths, and a second model is often the
  fastest reviewer of the first one's output.
- **Design**: Claude Design/Cowork, Google AI Studio, Google Pomelli, and Google Stitch for UI
  exploration and assets — work that would otherwise mean hiring a designer or settling for
  programmer art.
- **Marketing**: Claude Cowork for SEO audits and content planning — the part of solo building that
  usually gets skipped entirely.

The way I think about the tiers:

- **AI does ~10%** (autocomplete, lookups): costs almost nothing, saves almost nothing structural.
  You still write the software.
- **AI does ~50%** (pair programming): meaningful speedup on boilerplate and tests, but you remain
  the bottleneck on design and integration.
- **AI does ~90%** (agentic coding): this is where it pays off. ~$240/month is roughly one to two
  hours of a contractor's time — and it bought the majority of a shipped product with web, iOS, and
  Android clients. The cost that doesn't show up on the bill is review: at 90%, your job becomes
  reading code, writing specs, and catching the model's confident mistakes.

A one-day MVP and a three-platform product from a solo builder in three months simply wasn't a
realistic project at the ~10% tier.

### Hypothesis 3: AI applications are expensive to operate

**Verdict: busted — at this scale, operations cost less than most people guess.** Here's the
actual month-by-month Google Cloud bill, straight from the billing export:

| Cost item                                       | Feb 2026 | Mar 2026 | Apr 2026 | May 2026 |
| ----------------------------------------------- | -------- | -------- | -------- | -------- |
| Gemini API                                      | $11.83   | $35.91   | $11.24   | $34.88   |
| Cloud Run + Cloud Run Functions (API & batch)   | $2.37    | $6.09    | $2.96    | $3.28    |
| Secret Manager                                  | $1.08    | $0.61    | $1.02    | $1.26    |
| Artifact Registry + Cloud Build                 | $0.44    | $0.79    | $0.84    | $0.57    |
| Firebase (Hosting, Firestore, Storage)          | $0.05    | $0.05    | $0.06    | $0.09    |
| Cloud Scheduler                                 | $0.00    | $0.00    | $0.00    | $0.10    |
| **Total (GCP)**                                 | **$15.77** | **$43.45** | **$16.12** | **$40.18** |

(February predates the March 13 idea — that's exploratory Gemini usage on the same billing
account. Outside GCP: Sentry runs on the free tier, plus the $99/year Apple Developer Program.)

Two things in this table surprised me:

1. **The serving infrastructure is almost free.** Firestore, Hosting, Storage, Scheduler — the
   entire backbone serving the whole user base rounds to under $5/month. Receipt parsing itself —
   the core AI feature — costs pennies; image _input_ tokens are cheap.
2. **The biggest line item is image _generation_, not parsing.** Most of the Gemini spend
   ($21.78 in March, $34.41 in May) is image-output tokens from Gemini 2.5 Flash Native Image
   Generation — the AI-generated images in the product, not reading receipts. Reading and
   understanding a receipt costs fractions of a cent; generating pixels is where the tokens go.

And the larger asymmetry: development AI (~$240/month across Claude, ChatGPT, and Gemini
subscriptions) costs roughly 5–10× the entire production infrastructure, inference included. For a
small AI application, the expensive model is the one writing the code, not the one running in
production.

## What's Next

The product has already outgrown its original loop. The capture → digitize → visualize core is
still the engine, but user feedback keeps pulling it into new territory — recipe planning from what
you actually buy, shopping list recommendations based on purchase history, food waste tracking, and
fitness and wellness optimization from what you eat:

```mermaid
graph TD
    CORE["Core loop<br/>capture → digitize → visualize"]
    FB["User feedback"]

    CORE --> FB
    FB --> P["Recipe planning<br/>from purchase history"]
    FB --> S["Shopping list<br/>recommendations"]
    FB --> W["Waste tracking<br/>bought vs. used"]
    FB --> F["Fitness & wellness<br/>nutrition optimization"]
    P --> CORE
    S --> CORE
    W --> CORE
    F --> CORE

    style CORE fill:#dbeafe,stroke:#1e40af,color:#1e40af
    style FB fill:#fef3c7,stroke:#92400e,color:#92400e
    style P fill:#d1fae5,stroke:#065f46,color:#065f46
    style S fill:#d1fae5,stroke:#065f46,color:#065f46
    style W fill:#d1fae5,stroke:#065f46,color:#065f46
    style F fill:#d1fae5,stroke:#065f46,color:#065f46
```

None of these were on [my roadmap](https://grocerytrack.food/changelog) in March. All of them came
from users describing what they wished the data could do next — the
[changelog](https://grocerytrack.food/changelog) is the public record of that feedback loop.

The 138 distinct stores in the database hint at the longer-term opportunity: enough structured,
item-level price data to answer questions no single shopper can — like how the price of the same
item differs across stores and over time. That's the direction I'm heading, one scanned receipt at
a time.

If you want to try it, GroceryTrack is live [on the web](https://grocerytrack.food/), on the
[iOS App Store](https://grocerytrack.food/ios-app), and in Android beta. Bring your receipts —
especially the weird ones. Those are my favorite.
