Developer Evaluation
How metre guessing works here
We do not feed raw Tamil into a big neural net in the browser. We turn the poem into structure, pack that into 51 numbers, then small models vote among four metre families. Classical rule checks stay separate — they never secretly rewrite the model score.
Three parts that never mix scores
Like a machine with sensors and a separate safety checklist: the structure engine measures the poem; the models guess the metre; a light classical sketch can raise flags — but the sketch does not change the model’s number.
Structure engine
Text → syllables → words/feet → bonds → 51 numbers. Same poem, same structure for a given parser version.
Small models
Rule-ish hybrid path, dense logistic, and “looks like mean Venpaa” prototypes each cast a share (0–1).
Light classical flags
Optional soft warnings for the model’s top guess. Shown beside ML — never fused into one score.
What you see in the lab tab (plain words)
How mixed? (entropy)
Is the model sure, or torn between metres?
If almost all the weight sits on one family, the number is low (clear). If weight is spread across several families, it is high (unclear). This is about the shape of the guess — not a scholar’s grade.
Lead over #2 (confidence gap)
How far is first place ahead of second?
A big lead means one metre clearly wins among the model votes. A tiny lead means two look almost the same. Still not classical proof.
Model votes
Several small tools voting on the same poem
Each head shows a 0–1 share for its favourite family. Use them to compare tools. On poems from our training anthology, votes can look very strong because the models already saw similar rows — that is normal and honest to say out loud.
Strong signals (pattern features)
Which of the 51 numbers mattered for this poem?
A short list of dense slots that pushed the logistic vote. Useful for debugging — not the full research catalogue.
Classical flags (when present)
Anything odd for the model’s top label?
Soft structural notes only. They do not re-score the model. Research dual-compare tables (offline) can group cases like “model said X, flags raised” — still two truths, one table.
Live browser vs offline training
Offline
- Parse special_type anthology
- Fit heads & freeze metrics
- Write ADOPT reports
Live (browser)
- Your poem → parse in WASM
- Cached heads vote once warmed
- Show numbers in Developer Evaluation
| Where | What happens | Why it matters |
|---|---|---|
| Live (this site) | Parse in WASM → show votes for this poem | Fast feedback while you write |
| Offline training | Fit heads on special_type rows; freeze metrics | Honest scores live in dated reports |
| Stress rows | variation poems | Where models fail — not the main grade |
Easy mistakes
- “0.99 means 99% right” — No; it is share among four options for that tool.
- “Flags mean the metre is proven” — No; light sketch only.
- Training only on hard variation rows — Muddies the main grade.
- Expecting every research method in the UI — Most stay offline; the lab shows the product surface.
Fields and methods we researched (and ship or keep offline). Names match the portfolio: control systems, machine learning, information theory, and data mining — mapped onto this poem parser.
Control systems & dynamical systems
Separation of plant and observer
| Field / idea | In this project | Online UI? | Offline research? |
|---|---|---|---|
| Plant | Parse stages: units → syllables → feet → bonds | Yes (structure) | Yes |
| Sensors | dense[51], linkage histograms | Yes (numbers) | Yes |
| State | 4-way metre belief | Yes (votes) | Yes |
| Observer | Hybrid / logistic / prototype heads | Yes | Yes |
| Feedback | Live re-parse debounce (UX rate limit) | Yes | N/A |
| Reference | Gold parent_metre (special_type) | Indirect | Yes (metrics) |
| Noise | Short poems, variation stress rows | Seen live | Yes (robustness) |
| Separation principle | ML ⟂ classical flags (never fuse scores) | Yes (policy) | Yes |
| Identification | Weight fit, PCA, prototypes after schema pin | Cached heads | Yes |
| Sensitivity | ∂score/∂dense_j, ablation, counterfactual flips | Pattern list (light) | Yes (A10–A11) |
| Stability | Schema + weight pins; ledger fingerprint | Pinned WASM | Yes (S03) |
Machine learning (sklearn-map path)
Small N, engineered features first
| Method | Tier | Role | Online UI? | Offline? |
|---|---|---|---|---|
| Multinomial / hybrid logistic | A | Shipped ranking + dense baseline | Yes | Yes |
| Dense logistic (cached) | A | Soft class shares on z-scored dense | Yes (head) | Yes |
| Class prototypes / k-NN mass | A | “Looks like mean Venpaa” | Yes (head) | Yes |
| Linear SVM | A | Second linear ceiling | No | Yes |
| Calibration (Platt / isotonic) | A | Honest probability talk | Partial | Yes |
| PCA / SVD, LDA | A | Directions in dense space | No | Yes |
| Random Forest / GBDT | B | Importances → distill rules | No | Yes |
| Kernel SVM, GMM, clustering | B | Nonlinear / structure discovery | No | Yes |
| HMM / CRF on Ner–Nirai | B | Sequence / structured labels | No | Yes |
| 1D CNN / small Transformer | C | Only after dense baselines | No | Research |
Information theory & statistics
Honesty about uncertainty
| Field | Meaning here | Online UI? | Offline? |
|---|---|---|---|
| Entropy (bits) | How mixed the 4-way vote is | Yes | Yes |
| Epistemic margin / confidence gap | Top1 − top2 mass | Yes | Yes |
| Soft mass (0–1) | Relative share per head — not calibrated % | Yes | Yes |
| Top-1, MRR, correct@2 | Primary ranking metrics | No | Yes (ADOPT) |
| Bootstrap / LOO / stratified CV | Tiny-N uncertainty | No | Yes |
| ECE / reliability diagrams | Calibration quality | No | Yes |
| Mutual information dense_j ↔ metre | Which slots inform labels | No | Yes (A06) |
| χ² / Fisher on linkage × metre | Discrete bond association | No | Yes |
| Head A/B table | Compare estimators fairly | No | Yes (A13) |
Data mining & knowledge discovery
Human-readable patterns
| Method | Role | Online UI? | Offline? |
|---|---|---|---|
| Association rules (Apriori-style) | Itemsets ⇒ metre | No | Yes (A08) |
| Frequent Ner/Nirai motifs | Sequence vocabulary for later classical | No | Yes (A09) |
| Contrast / emerging patterns | Venpaa vs Aciriyappaa differentiators | No | Yes |
| Error subgroup discovery | Why ML fails | No | Yes |
| Pattern cards (A12 freeze) | Per-metre top dense signals | Freeze date shown | Yes |
| Disagreement mining | ML vs classical dual-compare buckets | Policy only | Yes (D03) |
| Anomaly / isolation ideas | Suspect gold or parse | No | Yes (B) |
Product wire fields you can see
From parse_poem_wasm → metre_ml
| JSON / UI field | Plain meaning | Live? |
|---|---|---|
| metre_type | Best guess after hybrid/heuristic path | Yes |
| metre_entropy_bits | How mixed? (entropy) | Yes |
| metre_epistemic_margin | Lead over #2 | Yes |
| top_k_metre_hypotheses | Ranked families with soft mass | Yes (Structure) |
| parse_features.dense[51] | Numeric summary of structure | Adapted |
| metre_ml.head_votes[] | Per-head favourite + soft mass | Yes |
| metre_ml.pattern_features[] | Top dense slots for this poem | Yes |
| metre_ml.dual_truth.* | ML label ∥ classical flags + policy | Yes |
| metre_ml.a12_freeze_date | Pattern freeze era | Yes if present |
| metre_ml.honesty_label / uncertainty_blurb | Short honesty copy | Yes |
| ml_only_classical_flags (reports) | Dual-compare bucket offline | No (reports) |
Ontology
Poem parts & bonds
Semantics
What each number means
Anthology
special_type gold
ML map
linear first
Control
measure ⟂ flags
51 numbers + multi-head product surface
Fixed schema · special_type fit · dual-truth wire
How we train (and what we do not claim)
- Pin meaning first (SOA S00–S03) — ontology of poem parts, exact meaning of each dense slot, anthology split (special_type gold vs variation stress), ledger fingerprint.
- Freeze a baseline (A00) — dated top-1 / MRR on special_type. That freeze is ADOPT evidence, not the browser UI.
- Fit small heads — dense logistic and prototypes on special_type (cached in WASM for live). Hybrid path can re-rank with shipped weights.
- Discover, then freeze patterns (A05–A12) — before soft classical sketches ship as product.
- Classical stays orthogonal (D) — flags and dual-compare reports never overwrite hybrid scores.
- S
Foundations
meaning pinned
- A0
Baseline freeze
ADOPT grade
- A2
Dual-truth wire
no score fusion
- A3–4
Logistic + prototypes
live heads
- A12
Pattern freeze
before classical
- D
Classical dual path
flags only
Online vs offline surfaces
Online (browser WASM)
parse_poem_wasm → dense + head votes + entropy/margin + dual_truth. Product UI: Live, Structure, Developer Evaluation. Offline crates can be compiled out of product WASM.
Offline (cargo examples / reports)
Freeze metrics, PCA/MI/association, pattern cards, head A/B, dual-compare (e.g. ml_only_classical_flags), mismatch tables. Truth for ADOPT lives under data/training/reports.
Repo documents (source of truth)
- METRE_ML_BEGINNER_GUIDE.md — story, I/O, diagrams
- METRE_ML_METHODS_PORTFOLIO.md — Tier A–D catalogue + ADOPT
- PARSE_FEATURES.md — exact dense[51] layout
- METRE_PREDICTION.md — hybrid / heuristic behaviour
- TRAINING_PROCESS.md — export & Monte Carlo