How it works
Architecture
muddler has three layers:
- Mobile app — camera capture, UI, and result display
- Backend API — OCR processing, parsing, and the knowledge engine
- Knowledge base — ingredients, recipes, templates, and syrup specs
When you scan a menu, the image flows through a pipeline: capture, OCR, parse, match, and enrich.
The OCR pipeline
muddler uses a multi-pass OCR strategy:
- Image preprocessing — contrast enhancement, dark-background detection and inversion, noise reduction
- Multi-pass OCR — runs three page segmentation modes (PSM 11, 4, 6) and picks the highest-confidence result
- Text cleanup — strips OCR artifacts, normalizes whitespace, and splits the output into candidate cocktail blocks
For particularly messy scans (handwriting, low contrast, ornamental fonts), an optional AI enhancement layer cleans up the OCR output before parsing. AI enhancement is opt-in.
Recipe parsing
The parser handles multiple menu formats using pattern recognition:
- Detects cocktail names, ingredient lists, prices, and descriptions
- Separates measured ingredients (
2 oz Gin) from unmeasured ones (Gin, Tonic) - Handles inline, block, comma-separated, and mixed formats in the same menu
Ingredient matching
This is the core of the knowledge engine. Each parsed ingredient string goes through an 8-tier fuzzy matcher against a database of 9,000+ ingredients:
| Tier | Strategy | Example |
|---|---|---|
| 1 | Exact match | “Gin” → Gin |
| 2 | Alias exact match | “Genever” → Gin |
| 3 | Starts-with | “Bourbon whi…” → Bourbon Whiskey |
| 4 | Substring | “…sweet verm…” → Sweet Vermouth |
| 5 | Category match | “house rum” → Rum (category) |
| 6 | Sequence similarity | “Cointrau” (OCR error) → Cointreau |
| 7 | Partial word | “elderflwr” → Elderflower Liqueur |
| 8 | Best guess | Lowest-confidence fallback |
Each ingredient carries metadata: category, subcategory, ABV, flavor profile on five dimensions, common brands, and substitutes. See Ingredient database.
Template classification
muddler classifies cocktails into one of 7 structural families:
- Sour — spirit + citrus + sweetener
- Fizz — sour + effervescence
- Spirit Forward — spirit + small modifier (Negroni, Manhattan)
- Highball — spirit + mixer
- Tiki — multiple rums + tropical ingredients
- Spritz — low-ABV, bubbly
- Collins — spirit + citrus + syrup + soda
Each template supplies standard ratios, technique (shake, stir, or build), glassware, and an expected flavor profile. Templates are how muddler fills in the gaps when the menu is silent.
Measurement inference
Most menus don’t list measurements. When they don’t, muddler infers them:
- Classify the cocktail into a template family based on ingredient composition.
- Apply that template’s standard ratios to the identified ingredients.
- Adjust for ingredient category (bitters get dashes, aromatic rinses get a small ounce fraction, etc.).
The unit system handles oz, ml, dashes, teaspoons, and tablespoons interchangeably.
Flavor profiling
Every cocktail gets a computed flavor profile built from its ingredients. Each ingredient is pre-scored on:
- Sweet (0–10)
- Sour (0–10)
- Bitter (0–10)
- Boozy (0–10)
- Aromatic (0–10)
The cocktail’s overall profile is a weighted average based on ingredient proportions, then visualized in the app as radar charts and bar graphs.