Building subscription detection: what makes a charge recurring
An engineering note on the messy problem of deciding, from a bank feed alone, that a charge will happen again.
Detecting subscriptions from a transaction feed sounds like a solved problem until you look at real data. This is a note on why it is harder than it appears and how our detection is built.
The naive version
Group transactions by merchant, look for a repeating amount at a repeating interval, call it a subscription. This gets you perhaps sixty percent of the way and then fails in ways that are individually small and collectively fatal to trust.
Where it breaks
Merchant strings are not stable. The same service can appear under a legal entity name, a trading name, a payment processor, and a truncated variant with a store number appended. Naive grouping splits one subscription into four unrelated merchants, each of which now looks irregular.
Amounts move. Currency conversion, tax changes, tier changes, proration, and promotional periods all produce a different figure for the same commitment. A strict amount match misses exactly the cases a user most wants flagged.
Intervals wobble. Monthly billing is not thirty days. It is the same calendar day, shifted by weekends, retries, and bank posting delays. A tolerance too tight misses real subscriptions; too loose and a fortnightly coffee habit becomes a subscription.
Regular is not recurring. Commuting, groceries, and a standing coffee purchase all produce beautifully periodic data and none of them are commitments you can cancel. Periodicity alone is not the signal.
How we approach it
Normalise the merchant first. Descriptor strings are cleaned, processor prefixes are stripped, and the remainder is matched against a catalogue of known billing descriptors. Getting the entity right upstream removes most of the downstream noise.
Score, do not classify. Each candidate gets a confidence score built from interval regularity, amount stability within a tolerance band, descriptor family, and catalogue membership. A score is honest about uncertainty in a way a boolean is not.
Set the threshold by cost of error. A false positive costs the user five seconds of dismissal. A false negative costs them a charge they did not expect. The threshold sits accordingly, and low-confidence candidates surface as suggestions rather than facts.
Let the user be the final classifier. Confirming or rejecting a candidate is one tap, and those decisions feed back. Human confirmation is by far the highest-quality label available, and a product that pretends not to need it ends up quietly wrong.
Prefer explaining over asserting. Every detected subscription shows why it was detected — the matched charges, the interval, the variance. Users trust a system that shows its working, and they can correct it precisely when it is wrong.
The part that is still hard
Annual subscriptions are genuinely difficult, because a single prior observation is not a pattern. Here the catalogue does most of the work: a known annual billing descriptor is treated as a subscription on first sight, with an explicit note that the interval is inferred rather than observed.
The other open problem is bundles. When one payment covers several services, the useful unit for the user is the service, and the bank only ever sees the payment. That gap cannot be closed from transaction data alone, which is one reason manual entries sit alongside detected ones as first-class objects rather than a fallback.