Skip to content

Latest commit

 

History

History
340 lines (277 loc) · 20.2 KB

File metadata and controls

340 lines (277 loc) · 20.2 KB

Umber

Umber

Your bank already texts you every transaction.
Umber turns those texts into a spending tracker that never leaves your phone.

Latest release Android 8.0+ No internet permission MIT


Umber is an offline Android expense tracker for India. It reads bank and UPI SMS, categorises each spend with a small model that trains on your own corrections, and shows the totals on a home-screen widget. Kotlin and Jetpack Compose, Room for storage, about 7,800 lines of Kotlin, 148 JVM unit tests.

The default privacy build has no INTERNET permission. That is not a policy promise. The permission is absent from app/src/main/AndroidManifest.xml, the main source set has no network code at all, and Android backups are switched off. Your messages, ledger, model and location cannot leave the device because the process cannot open a socket.

Why

Every expense tracker wants one of three things: your net-banking login, your statements uploaded to someone's server, or transactions typed in by hand. But the bank SMS on your phone already carries the amount, the payee, the account tail and a reference number. It only needs parsing well and staying put.

Umber parses with regex (bank SMS are templated, so regex is more accurate and far easier to debug than a sequence model) and saves machine learning for the one fuzzy problem, merchant to category. The classifier runs on the phone and learns from every tap in the Review tab.

Demo

Quickstart

Install the app

Download the APK from Releases and sideload it (allow installs from unknown sources). Play Store distribution is not possible: READ_SMS is a restricted permission and expense tracking is not on Google's permitted-use list. This is why apps like Walnut were pulled in 2019.

On first launch, grant SMS access (the app has no data source without it) and optionally location. Then go to Settings > Import history > 1 year to fill the ledger and give the classifier something to learn from.

Umber cannot check for updates itself. Point Obtainium at https://github.com/DeepakSilaych/umber to get notified, or use Settings > About > Check for updates, which opens the releases page in your browser (the browser does the networking).

Build from source

Needs JDK 17 and the Android SDK with API 35. The Gradle wrapper (8.9) downloads everything else.

git clone https://github.com/DeepakSilaych/umber.git && cd umber
./gradlew testPrivacyDebugUnitTest      # 140 JVM tests, no emulator needed
./gradlew installPrivacyDebug           # installs com.deepak.umber.debug on a connected device

Release APKs need a keystore configured in local.properties (see Configuration):

./gradlew assemblePrivacyRelease        # what Releases ship
./gradlew assembleCloudRelease          # the optional sync build, see "Two builds"

Verify the permission claim

aapt2 dump permissions app/build/outputs/apk/privacy/release/app-privacy-release.apk | grep INTERNET
# no output

Run the optional sync server (cloud build only)

The privacy build never needs this. The cloud build can push its parsed ledger to a self-hosted FastAPI server with a React dashboard. Both live under server/.

cd server
cp .env.example .env                    # set UMBER_SETUP_KEY and UMBER_DASHBOARD_PASSWORD
docker compose up --build               # Postgres 16 + API + dashboard on http://localhost:8000
curl localhost:8000/healthz             # {"ok":true}

Then paste the same UMBER_SETUP_KEY into Settings > Sync > Setup key on the phone and turn the switch on. A debug cloud build can point at a local server via the endpoint override field.

How it works

                 +-----------------+      +------------------+
  live SMS ----> | SmsReceiver     |      | SmsBackfill      | <---- inbox, 1m / 3m / 1y
                 +--------+--------+      +---------+--------+
                          |                         |
  statement -----> StatementImporter                |     LedgerCsv <----- own CSV export
  (.xlsx/CSV/TSV)         |                         |          |
                          v                         v          v
                 +------------------------------------------------------+
                 | IngestPipeline                  ingest/IngestPipeline.kt |
                 |  1 SenderFilter   drop personal numbers, tag DLT senders  |
                 |  2 SmsParser      regex: amount, direction, merchant, ref |
                 |  3 dedupe         refNo  ->  +/-3 min window  ->  per-day |
                 |  4 Classifier     memory  ->  logistic model  ->  lexicon |
                 |  5 LocationCache  attach last foreground fix + confidence |
                 +----------------------------+-------------------------+
                                              v
                 +------------------------------------------------------+
                 | Room  umber.db              data/db/                  |
                 |  raw_message (verbatim, source of truth)              |
                 |  txn  merchant_memory  model_state  last_location     |
                 +------+---------------------+---------------------+---+
                        |                     |                     |
                        v                     v                     v
                Compose UI (4 tabs)    Glance widget         Review tap -> Classifier.learn
                Home History           + RollupWorker         memory + relabel + SGD step
                Review Settings          every 20 min
  1. ingest/SmsReceiver.kt catches SMS_RECEIVED (priority 999, non-aborting), reassembles multi-part messages, and hands each one to the pipeline. ingest/SmsBackfill.kt does the same for the inbox on demand. Statement and CSV rows enter through io/StatementImporter.kt and io/LedgerCsv.kt as already-structured TxnRecords.
  2. parse/SenderFilter.kt hard-rejects personal mobile numbers before anything is stored. It knows 46 bank and 17 PSP DLT codes (VM-HDFCBK, AD-SBIUPI and so on) but an unknown sender is still parsed if the message carries an account tail or a reference number.
  3. parse/SmsParser.kt is pure regex. It rejects OTPs, promos, future-tense and failed transactions first, then extracts amounts (currency-prefixed, currency-suffixed, and verb-anchored bare numbers like SBI's debited by 120.0), tags balances by the text before the digits, picks direction by earliest keyword after masking credit card, and tries an ordered list of merchant patterns. Every parse records SmsParser.VERSION (currently 2).
  4. The raw text goes into raw_message with a SHA-256 fingerprint (sender, body, day) so re-imports are no-ops. Dedupe then runs in three tiers: a normalised reference number is authoritative, live messages fall back to a 3-minute window on amount, direction and account tail, and date-only statement rows use a same-day occurrence count so nine identical mandate debits stay nine rows.
  5. ml/Classifier.kt checks merchant memory (exact match, confidence 1.0), then the on-device model if it has 25 or more training examples and is at least 62% sure, then the 209-entry seed lexicon in ml/SeedLexicon.kt. Anything weaker is flagged needsReview.
  6. Confirming a category in Review calls UmberRepository.confirmCategory: the row becomes USER, every past row from the same merchant is relabelled unless the user already set it, and Classifier.learn writes merchant memory then takes 5 SGD steps on the new example interleaved with 48 random past confirmations so one burst of edits cannot skew the boundary.
  7. widget/SpendWidget.kt renders rolling 24-hour spend plus this week, last week, this month and last month from UmberRepository.widgetSnapshot(). work/RollupWorker.kt refreshes it every 20 minutes because the 24-hour window moves on its own.

Full detail is in docs/ARCHITECTURE.md.

Features

Feature Where
Live SMS capture plus one-tap import of 1 month, 3 months or 1 year of inbox history ingest/SmsReceiver.kt, ingest/SmsBackfill.kt
Bank statement import as .xlsx, CSV or TSV with no per-bank templates: header row found by keyword scoring, format sniffed from magic bytes io/StatementImporter.kt, io/XlsxReader.kt, io/Csv.kt
Three-layer on-device categorisation into 15 fixed categories ml/Classifier.kt, data/model/Categories.kt
Online learning: each correction updates merchant memory, relabels history and trains the model in place data/repo/UmberRepository.kt, ml/LogisticModel.kt
Retrain from every confirmed row, and re-scan messages an older parser rejected Settings > Retrain, Re-scan; IngestPipeline.reparseRejected
Rebuild the whole ledger from stored raw messages; confirmed categories survive via merchant memory IngestPipeline.rebuildFromRawMessages
Reimbursement netting per counterparty, so a repaid split bill is 0 spend, not spend plus income data/repo/Netting.kt
Spend by account or card, top categories, daily series, merchant and category search data/db/Daos.kt, ui/home, ui/history
Home-screen widget with height-based layouts and a 10-bar sparkline of the last 30 days widget/SpendWidget.kt
Ledger export and import as CSV, amounts as plain decimals that sum in a spreadsheet io/LedgerCsv.kt
Optional foreground-only location tag with age-based confidence; no background location permission location/LocationCache.kt
Settings > Messages skipped: a count of every rejection reason, for finding parser misses RawMessageDao.rejectBreakdown
Optional cloud build: sync the parsed ledger to your own server with a web dashboard app/src/cloud/, server/

Two builds

privacy cloud
Package com.deepak.umber com.deepak.umber.cloud
Name Umber Umber Sync
INTERNET permission absent present (app/src/cloud/AndroidManifest.xml)
Network code in the APK none Retrofit, OkHttp, kotlinx.serialization, scoped with cloudImplementation
Classification on-device only on-device first; the server can add LLM verdicts for merchants the phone left as Other
Sync not possible, RemoteSyncFactory returns null off until you paste a setup key; SyncWorker runs every 45 minutes on a network constraint

privacy is the default and is what Releases ship. Both flavours share every line of parsing, storage and UI. cloud gets a different application id and display name on purpose so the two are distinguishable side by side, and its Settings screen states exactly what it sends.

What crosses the wire in cloud is the parsed ledger only: amount, direction, merchant, category, account tail, reference number, timestamp. Raw SMS text, balances and location stay on the phone (remote/RemoteSync.kt, docs/SYNC.md). Server credentials sit in EncryptedSharedPreferences. When categories disagree, data/repo/CategoryPriority.kt makes sure a USER or MEMORY label on the phone is never overwritten by a dashboard edit or an LLM verdict.

Permissions

What privacy requests, from app/src/main/AndroidManifest.xml:

Permission Why
RECEIVE_SMS, READ_SMS The only source of transaction data.
ACCESS_COARSE_LOCATION, ACCESS_FINE_LOCATION Optional. Foreground only. ACCESS_BACKGROUND_LOCATION is never requested.
WAKE_LOCK, FOREGROUND_SERVICE, RECEIVE_BOOT_COMPLETED, ACCESS_NETWORK_STATE Merged in by the WorkManager library for the widget refresh job. Not in this app's manifest.

ACCESS_NETWORK_STATE only allows observing connectivity. Without INTERNET the process cannot open a socket. android:allowBackup="false" and res/xml/data_extraction_rules.xml also exclude the database, preferences and files from cloud backup and device transfer.

Configuration

The app has no runtime config file. Everything is build-time.

Key Where Default Purpose
umber.keystore local.properties (gitignored) unset Path to the release keystore. Missing is fine; only assemble*Release needs it.
umber.storePassword, umber.keyAlias, umber.keyPassword local.properties alias umber, key password falls back to store password Release signing.
CLOUD_ENABLED BuildConfig, set per flavour false / true Which flavour is running.
SYNC_BASE_URL BuildConfig, cloud only https://finance.deepaksilaych.me Sync server. Debug builds can override it in Settings.

Server (server/app/config.py, env prefix UMBER_, read from server/.env):

Variable Default Purpose
UMBER_DATABASE_URL postgresql+psycopg://umber:umber@localhost:5432/umber Postgres DSN.
UMBER_SETUP_KEY dev-setup-key-change-me Shared secret the phone sends to POST /v1/devices/register.
UMBER_DASHBOARD_PASSWORD dev-password-change-me Single-user dashboard login.
UMBER_JWT_SECRET dev-jwt-secret-change-me Signs the dashboard session cookie (30-day TTL).
UMBER_COOKIE_SECURE true Set false for plain-http local dev.
UMBER_AGENT_API_TOKEN unset Optional static bearer token for API access by an AI agent. Unset disables it.
UMBER_CORS_ORIGINS ["http://localhost:5173"] Vite dev server origin.
UMBER_LLM_GATEWAY_URL, UMBER_LLM_GATEWAY_KEY, UMBER_LLM_MODEL gateway URL set, key unset, openai/gpt-oss-20b OpenAI-compatible endpoint used by POST /v1/classify and POST /v1/insights/generate. The key never ships in the APK.

Design decisions

  • No INTERNET permission instead of a privacy policy. The guarantee is enforced by the OS, and the cloud flavour keeps every network dependency behind cloudImplementation so nothing leaks into the privacy APK (app/build.gradle.kts).
  • Regex parsing, ML only for categories. Bank SMS are templates. When a regex is wrong you can point at the line. Every parser bug found on real data becomes a case in app/src/test/.../parse/RealTemplatesTest.kt.
  • Raw messages are the source of truth. raw_message keeps every SMS verbatim with the parser version that handled it, so a better parser can be replayed over history. There is no destructive migration fallback; MIGRATION_1_2 and MIGRATION_2_3 in AppDatabase.kt are real migrations.
  • A linear model over hashed character n-grams, not a transformer. Merchant strings are one to four tokens. FeatureHasher maps 3 to 5 character n-grams, tokens, amount band, hour bucket, weekday and channel into 16,384 signed buckets. A gradient step costs microseconds, which is what makes "learn from every correction, now" possible with no training pipeline. Known cost: the dense weight matrix is 15 x 16,384 floats, about 1 MB, rewritten on every correction.
  • Prefer a rejection to a guess. The model is not consulted below 25 examples, only accepts above 0.62, and seed-lexicon guesses on debits are always queued for review. A wrong confident label hides; a skipped message shows up in Settings > Messages skipped.
  • Money is integer paise everywhere. Never floats. Money.kt formats with en-IN lakh and crore grouping.
  • No DI framework, no CSV or spreadsheet library. AppContainer is a field on Application because a BroadcastReceiver and a Glance widget only hold a Context. .xlsx is read with ZipInputStream plus SAX, and CSV is a small hand-rolled RFC 4180 reader, because every dependency is dead weight in an APK that never updates itself.

Project layout

app/src/main/java/com/deepak/umber/
  parse/       Pure JVM. SmsParser, SenderFilter, Normalize, DateParse, MessageFingerprint.
  ml/          Pure JVM. Classifier, FeatureHasher, LogisticModel, SeedLexicon.
  io/          Pure JVM. Csv, StatementImporter, XlsxReader, LedgerCsv.
  ingest/      IngestPipeline, SmsReceiver (live), SmsBackfill (history), TxnRecord.
  data/db/     Room database, entities, DAOs, converters.
  data/repo/   UmberRepository, Netting, CategoryPriority.
  location/    LocationCache, foreground-only fixes.
  widget/      Glance widget and WidgetUpdater.
  work/        RollupWorker (widget refresh every 20 min).
  remote/      RemoteSync interface shared by both flavours.
  ui/          Compose screens: Home, History, Review, Settings.
app/src/privacy/   RemoteSyncFactory that returns null.
app/src/cloud/     INTERNET manifest, HttpRemoteSync (Retrofit), SyncWorker, CloudPrefs.
app/src/test/      140 JUnit tests, all plain JVM.
app/src/testCloud/ 8 tests for the wire mapping.
server/            FastAPI + SQLAlchemy + Alembic sync server, Dockerfile, docker-compose.yml.
server/web/        React 19 + Vite + Tailwind dashboard, built into the API image.
docs/              ARCHITECTURE.md, ADDING-A-BANK.md, SYNC.md, logo.

Development

App tests run on the JVM. parse/, ml/, io/ and Netting have no Android imports.

./gradlew testPrivacyDebugUnitTest                                   # shared test set
./gradlew testCloudDebugUnitTest                                     # shared + testCloud
./gradlew testPrivacyDebugUnitTest --tests '*SmsParserTest' --tests '*RealTemplatesTest'

Server tests need a reachable Postgres. server/tests/conftest.py defaults to postgresql+psycopg://umber:umber@localhost:5432/umber_test and creates the tables itself.

cd server
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
pytest                                                               # 143 tests
alembic upgrade head && uvicorn app.main:app --reload                # API alone on :8000

Dashboard, from server/web/: npm ci, then npm run dev (Vite on :5173), npm run lint (oxlint), npm run build. The Dockerfile runs the build and copies dist/ into the API image.

There is no CI workflow in the repo yet.

Limitations

  • SMS only. SourceType.NOTIFICATION exists in the enum for a future NotificationListenerService path (the one Play Store distribution would need), but only the SMS path is implemented.
  • No PDF and no legacy .xls statements. PDF needs a rendering library and is usually password-protected; .xls is a different binary container. XlsxReader detects .xls and asks for a re-save as .xlsx or CSV.
  • Netting matches on the normalised counterparty name. "DEEPAK SILAYCH" and "DEEPAKSILAYCH" are two counterparties and never net. Honorifics are stripped; missing spaces are not handled.
  • Location is coordinates only. Reverse geocoding would need network access.
  • Changing Categories.ALL invalidates the model. The list order is the label encoding, so any edit must bump LABELS_VERSION, which discards stored weights and retrains from confirmed rows.
  • Server statement import does not yet read the account number from a statement preamble, so auto-linking to an account only works for phone-synced rows (docs/SYNC.md, "Known gap").
  • No iOS version. iOS gives apps no access to SMS content.

Contributing

The highest-value contribution is a bank message Umber parses badly. Settings > Messages skipped shows why each message was ignored. Add the redacted message to RealTemplatesTest, watch it fail, then fix the regex; see docs/ADDING-A-BANK.md. Please strip account numbers, references and payee names from anything you paste into an issue.

Licence

MIT, see LICENSE. Built by deepaksilaych.