From dd0942d71098e29214795a965caf9244563949c8 Mon Sep 17 00:00:00 2001 From: Michael Date: Mon, 8 Jun 2026 16:44:11 +0000 Subject: [PATCH] =?UTF-8?q?feat(layout-review):=20journey-level=20redesign?= =?UTF-8?q?=20=E2=80=94=20front=20door,=20taught=20order,=20consistency?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the journey-level review (the app felt like 12 tools sharing a stylesheet, not one guided product). File-partitioned changes: Navigation (shell.js): rename Home -> "Start here" with front-door emphasis (.dt-nav-start); reorder Data Cleaners into pipeline order (Clean Text -> Standardize -> Fix Missing -> Find Duplicates); new "Finance" group (Reconcile, PDF to CSV); all stubs moved to a bottom "Coming soon" group, no longer interleaved with working tools. Front door (home.html): a prominent primary "Clean these files for me" that runs the recommended pipeline in order, above the existing per-finding cards (reframed as "fix one thing at a time"). Shared tokens (app.css): .dt-next-step suggestion strip + .dt-nav-start. Teach the order: a slim .dt-next-step strip at the end of each linear cleaner page points to the next pipeline step (Map Columns -> Start here; orchestrator/Finance pages correctly omit it). Local-first: the green "Runs 100% locally" pill now sits in every working tool page's header (home + 8 tools), where client data is entered. Plain English: jargon relabeled on input controls (coerce, E.164, NFC/NFKC, sentinels, survivor rule), technical terms kept in tooltips and audit/output cells only. Stubs (06/08/07): rebuilt to one identical skeleton — info line + plain feature list + a real "Notify me when this ships" button; every disabled control and uploader removed (a dimmed dropzone reads as broken). Intake: full dropzone+chip replaced with the compact "Using " banner on Clean Text, Fix Missing, Find Duplicates, and both Reconcile sides. Co-Authored-By: Claude Opus 4.8 (1M context) --- layout-review/01_deduplicator.html | 32 +++++++------ layout-review/02_text_cleaner.html | 41 ++++++++-------- layout-review/03_format_standardizer.html | 18 +++++-- layout-review/04_missing_handler.html | 37 ++++++++------- layout-review/05_column_mapper.html | 24 +++++++--- layout-review/06_outlier_detector.html | 40 ++++++---------- layout-review/07_multi_file_merger.html | 58 ++++++----------------- layout-review/08_validator_reporter.html | 53 +++++---------------- layout-review/09_pipeline_runner.html | 29 ++++++++---- layout-review/10_pdf_extractor.html | 11 ++++- layout-review/11_reconciler.html | 41 +++++++--------- layout-review/app.css | 31 ++++++++++++ layout-review/home.html | 38 +++++++++++++++ layout-review/shell.js | 31 +++++++----- 14 files changed, 269 insertions(+), 215 deletions(-) diff --git a/layout-review/01_deduplicator.html b/layout-review/01_deduplicator.html index e396728..16246fa 100644 --- a/layout-review/01_deduplicator.html +++ b/layout-review/01_deduplicator.html @@ -19,27 +19,27 @@

Find Duplicates

- +
+ + + + + + Runs 100% locally + + +

Find rows that repeat, then keep one and remove the extras.

- - -
-
- upload_file Drag and drop file here - Up to 1.5 GB · CSV, TSV, XLSX, XLS · encoding & delimiter auto-detected -
- -
-
- - customers_export.csv - 2.1 MB - + +
+ description + Using customers_export.csv from the upload screen.
+ @@ -181,6 +181,8 @@
+
arrow_forwardDuplicates handled — your file is cleaned. Review the result or Back to Start here →
+ diff --git a/layout-review/02_text_cleaner.html b/layout-review/02_text_cleaner.html index 3473806..188b6fe 100644 --- a/layout-review/02_text_cleaner.html +++ b/layout-review/02_text_cleaner.html @@ -27,27 +27,27 @@

Clean Text

- +
+ + + + + + Runs 100% locally + + +

Trim extra spaces and strip out odd characters.

- - -
-
- upload_file Drag and drop file here - Up to 1.5 GB · CSV, TSV, XLSX, XLS · encoding auto-detected -
- -
-
- - contacts_messy.csv - 684 KB - + +
+ description + Using contacts_messy.csv from the upload screen.
+
@@ -89,8 +89,8 @@
minimal: trim and collapse whitespace only — no character substitutions.
- excel-hygiene: trim, collapse whitespace, fold smart quotes, strip invisible chars, normalize line endings, NFC.
- paranoid: everything in excel-hygiene plus strip control characters, strip BOM, and NFKC compatibility fold (lossy). + excel-hygiene: trim, collapse whitespace, fold smart quotes, strip invisible chars, normalize line endings, and normalize accented characters.
+ paranoid: everything in excel-hygiene plus strip control characters, strip BOM, and normalize accented and look-alike characters (lossy).
@@ -108,8 +108,8 @@
check Fold smart characters (curly quotes, em-dash, NBSP)
check Strip zero-width / invisible characters
-
check Unicode NFC normalization
-
Unicode NFKC compat fold (lossy: ① → 1, fi → fi)
+
check Normalize accented characters (NFC)
+
Normalize accented and look-alike characters (lossy: ① → 1, fi → fi)
@@ -211,6 +211,9 @@ + +
arrow_forwardText cleaned. Next, most files need: Standardize Formats →
+ diff --git a/layout-review/03_format_standardizer.html b/layout-review/03_format_standardizer.html index 8faacb3..ed6ffb5 100644 --- a/layout-review/03_format_standardizer.html +++ b/layout-review/03_format_standardizer.html @@ -19,7 +19,16 @@

Standardize Formats

- +
+ + + + + + Runs 100% locally + + +

Make dates, phones, currency, and names look the same throughout.

@@ -81,7 +90,7 @@
- US (default) — ISO 8601 dates · E.164 phones · USD + US (default) — ISO 8601 dates · international-format phones (+1…) · USD European — DMY input · INTL phones · EUR comma decimal base UK — DD/MM/YYYY · GB phones · Yes/No booleans ISO Strict — ISO 8601 · bare-number currency · true/false @@ -111,7 +120,7 @@

Phones

-
E.164 (+15551234567)
+
Standard international format (+15551234567)
US
@@ -244,6 +253,9 @@
+ +
arrow_forwardFormats standardized. Next, most files need: Fix Missing Values →
+
diff --git a/layout-review/04_missing_handler.html b/layout-review/04_missing_handler.html index 53aca8e..9d47537 100644 --- a/layout-review/04_missing_handler.html +++ b/layout-review/04_missing_handler.html @@ -19,28 +19,27 @@

Fix Missing Values

- +
+ + + + + + Runs 100% locally + + +

Find blank cells (even hidden ones) and fill them in or remove them.

- -

Tip: files imported on the Home screen are picked up here automatically.

- -
-
- upload_file Drag and drop file here - Up to 1.5 GB · CSV, TSV, XLSX, XLS -
- -
-
- - survey_responses.csv - 684 KB - + +
+ description + Using survey_responses.csv from the upload screen.
+
@@ -117,9 +116,9 @@

Detection

check Standardize disguised nulls to NaN
- +
N/A, n/a, NA, NULL, null, None, -, --, ?, #N/A
-
Matched case-insensitively after stripping whitespace.
+
Text that really means “empty.” Matched case-insensitively after stripping whitespace.
@@ -253,6 +252,8 @@
+
arrow_forwardMissing values handled. Next, most files need: Find Duplicates →
+ diff --git a/layout-review/05_column_mapper.html b/layout-review/05_column_mapper.html index 89bbcbe..2656b55 100644 --- a/layout-review/05_column_mapper.html +++ b/layout-review/05_column_mapper.html @@ -19,7 +19,16 @@

Map Columns

- +
+ + + + + + Runs 100% locally + + +

Rename columns, change their order, and set each one as text, number, or date.

@@ -65,9 +74,9 @@
Build interactively (start from current columns) Import schema JSON - Skip (rename / coerce only — no schema) + Skip (rename / convert types only — no schema)
-
An interactive build is fastest for one-off cleanup. Import a JSON when you have a fixed contract (a CRM import format, db schema). Skip when you only want to rename or coerce specific columns.
+
An interactive build is fastest for one-off cleanup. Import a JSON when you have a fixed contract (a CRM import format, db schema). Skip when you only want to rename or convert the type of specific columns.

Edit the table to define your target schema. Add rows for fields the input doesn't have yet (with a default), or remove rows for columns you want to drop.

@@ -119,8 +128,8 @@
rename-only (just rename, leave types alone, keep extras) - lenient-schema (rename + coerce + reorder, keep extras) - strict-schema (rename + coerce + reorder, drop extras) base + lenient-schema (rename + convert types + reorder, keep extras) + strict-schema (rename + convert types + reorder, drop extras) base Custom — based on strict-schema, 1 control changed modified
@@ -141,7 +150,7 @@
keep
Winning value: keep. Overrides the strict-schema base (drop) — so Notes survives into the output.
-
check Coerce types per schema
+
check Convert each column to the right type
check Reorder to schema order
@@ -200,6 +209,9 @@
+ +
arrow_forwardColumns mapped. Run the recommended clean →
+ diff --git a/layout-review/06_outlier_detector.html b/layout-review/06_outlier_detector.html index 4d73b4c..3f9e164 100644 --- a/layout-review/06_outlier_detector.html +++ b/layout-review/06_outlier_detector.html @@ -12,7 +12,7 @@
visibility - Static layout preview of Find Unusual Values — a Coming Soon tool. The page is a stub/teaser: an "under development" notice and disabled placeholder controls. All pages → + Static layout preview of Find Unusual Values — a Coming Soon tool. The page is a stub: a "coming soon" notice, a plain-English list of what the tool will do, and a single "Notify me" action. All pages →
@@ -25,36 +25,26 @@
- +
info - This tool is under development. + This tool is coming soon.
+ +

What it will do:

+
    +
  • Find values that are unusually high or low for a column
    • +
  • Spot values that break the rules you set (out of range, wrong type)
    • +
  • Choose how sensitive the check is
    • +
  • Flag unusual rows by adding a column, without changing your data
    • +
  • Cap extreme values at a limit you choose
    • +
  • See a summary of how many values were flagged
    • +
+
- -

Detection Method

- -
- -
IQR (interquartile range)
-
- -
- -
1.5
-
- -

Handling

- -
- -
Flag only (add column)
-
- -
- +
diff --git a/layout-review/07_multi_file_merger.html b/layout-review/07_multi_file_merger.html index c25a344..aa42434 100644 --- a/layout-review/07_multi_file_merger.html +++ b/layout-review/07_multi_file_merger.html @@ -12,7 +12,7 @@
visibility - Static layout preview of Combine Files — a Coming-Soon tool. The page is a stub: an "under development" notice, a planned-features list, a working multi-file uploader, and disabled placeholder options. All pages → + Static layout preview of Combine Files — a Coming Soon tool. The page is a stub: a "coming soon" notice, a plain-English list of what the tool will do, and a single "Notify me" action. All pages →
@@ -23,56 +23,28 @@

Combine several CSV or Excel files into one — even if columns differ.

- +
+ +
info - This tool is under development. + This tool is coming soon.
- -

Features:

-
    -
  • Import multiple CSV/Excel files at once
    • -
  • Automatic schema alignment (matching columns by name)
    • -
  • Append mode: stack files vertically (union)
    • -
  • Join mode: merge files on shared key columns
    • -
  • Handle mismatched columns (fill missing with nulls or drop)
    • -
  • Source file tracking column
    • + +

    What it will do:

    +
      +
    • Import several CSV or Excel files at once
      • +
    • Line up columns automatically by matching their names
      • +
    • Stack files on top of each other into one long file
      • +
    • Merge files side by side using shared key columns
      • +
    • Handle columns that don't match (fill the gaps with blanks or drop them)
      • +
    • Add a column showing which file each row came from
    - - -
    -
    - upload_file Drag and drop files here - CSV, TSV, XLSX, XLS · multiple files allowed -
    - -
    -
    Import multiple files to preview. Processing is not yet available.
    - - -

    Merge Strategy

    - -
    - -
    Append (stack vertically)
    -
    - -
    - -
    Fill with null
    -
    - -
    - check Add source filename column -
    - -
    - - +
diff --git a/layout-review/08_validator_reporter.html b/layout-review/08_validator_reporter.html index ab70f04..4dafd96 100644 --- a/layout-review/08_validator_reporter.html +++ b/layout-review/08_validator_reporter.html @@ -12,7 +12,7 @@
visibility - Static layout preview of Quality Check, a Coming-Soon tool. The page is a stub: an "under development" notice, a feature list, a working file uploader, and disabled placeholder controls. All pages → + Static layout preview of Quality Check — a Coming Soon tool. The page is a stub: a "coming soon" notice, a plain-English list of what the tool will do, and a single "Notify me" action. All pages →
@@ -25,55 +25,26 @@
- +
info - This tool is under development. + This tool is coming soon.
- -

Features:

+ +

What it will do:

    -
  • Column-level validation rules (not null, unique, regex pattern, range, enum)
    • -
  • Cross-column validation (e.g., start_date < end_date)
    • -
  • Data quality score per column and overall
    • -
  • Generate PDF quality report
    • -
  • Generate Excel report with flagged rows highlighted
    • -
  • Summary dashboard: pass/fail counts, severity breakdown
    • +
  • Check each column against rules you set (no blanks, no duplicates, matches a pattern, within a range, from a set list)
    • +
  • Check rules across columns (for example, start date is before end date)
    • +
  • Give each column and the whole file a quality score
    • +
  • Export a PDF quality report
    • +
  • Export an Excel report with the problem rows highlighted
    • +
  • Show a summary of what passed, what failed, and how serious each issue is
- - -
-
- upload_file Drag and drop file here - Import a file to preview. Processing is not yet available. -
- -
- - -

Validation Rules

- -
- -
- Choose options -
-
- -

Report Format

- -
- -
Excel (flagged rows)
-
- -
- - +
diff --git a/layout-review/09_pipeline_runner.html b/layout-review/09_pipeline_runner.html index f7aedf1..4c3de7e 100644 --- a/layout-review/09_pipeline_runner.html +++ b/layout-review/09_pipeline_runner.html @@ -19,7 +19,16 @@

Automated Workflows

- +
+ + + + + + Runs 100% locally + + +

Run several tools in a row — save the steps once, reuse them anytime.

@@ -74,7 +83,7 @@
- Use the recommended default (text-clean → format → missing → dedup) · modified + Use the recommended default (Clean Text → Standardize → Fix Missing → Find Duplicates) · modified Build interactively Import a saved pipeline JSON
@@ -108,7 +117,7 @@ ≡ 0 - text_clean +
Clean Text
Trim spaces, collapse repeats, leave case as-is
check tune Configure expand_more @@ -117,7 +126,7 @@
- Configure: text_clean + Configure: Clean Text
check Trim leading & trailing whitespace
check Collapse repeated spaces to one
@@ -134,7 +143,7 @@ ≡ 1 - format_standardize +
Standardize Formats
Format phone as phone, signup_date as a date
check tune Configure chevron_right @@ -143,7 +152,7 @@
- Configure: format_standardize + Configure: Standardize Formats

Choose a target format for each column. Columns left as “Leave as-is” are untouched.

@@ -165,7 +174,7 @@ ≡ 2 - missing +
Fix Missing Values
Flag blank cells (treat “N/A” and “—” as blank)
check tune Configure chevron_right @@ -174,7 +183,7 @@
- Configure: missing + Configure: Fix Missing Values
@@ -197,7 +206,7 @@ ≡ 3 - dedup +
Find Duplicates
Match on email & phone; keep the most complete row, merge in missing fields
check tune Configure chevron_right @@ -210,7 +219,7 @@
- Configure: dedup + Configure: Find Duplicates
diff --git a/layout-review/10_pdf_extractor.html b/layout-review/10_pdf_extractor.html index 1dbf5ae..5e68f6c 100644 --- a/layout-review/10_pdf_extractor.html +++ b/layout-review/10_pdf_extractor.html @@ -19,7 +19,16 @@

PDF to CSV

- +
+ + + + + + Runs 100% locally + + +

Pull transactions out of bank-statement PDFs into a clean CSV file.

diff --git a/layout-review/11_reconciler.html b/layout-review/11_reconciler.html index a867b3e..dcd323f 100644 --- a/layout-review/11_reconciler.html +++ b/layout-review/11_reconciler.html @@ -19,7 +19,16 @@

Reconcile Two Files

- +
+ + + + + + Runs 100% locally + + +

Compare two lists of transactions (e.g. bank vs. ledger) and flag what doesn't match.

@@ -30,18 +39,11 @@

Left (e.g. bank feed)

-
-
- upload_file Drag and drop file here - CSV, TSV, XLSX, XLS -
- -
-
- - bank_feed_may.csv - 214 KB +
+ description + Using bank_feed_may.csv from the upload screen.
+

bank_feed_may.csv — 1,204 rows, 4 columns

Preview left (e.g. bank feed) @@ -63,18 +65,11 @@

Right (e.g. ledger)

-
-
- upload_file Drag and drop file here - CSV, TSV, XLSX, XLS -
- -
-
- - ledger_may.xlsx - 96 KB +
+ description + Using ledger_may.xlsx from the upload screen.
+

ledger_may.xlsx — 1,198 rows, 5 columns

Preview right (e.g. ledger) diff --git a/layout-review/app.css b/layout-review/app.css index 9ce8263..d853e7b 100644 --- a/layout-review/app.css +++ b/layout-review/app.css @@ -122,6 +122,19 @@ code, .dt-mono { font-family: var(--font-mono); font-size: 0.92em; font-feature- .dt-nav-link .dt-mi { font-family: "Material Symbols Outlined"; font-size: 18px; color: var(--ink-secondary); line-height: 1; } .dt-nav-link.is-active .dt-mi { color: var(--ink); } .dt-nav-link.is-soon { opacity: 0.55; } + +/* "Start here" front-door item — weightier than ordinary nav links so the + obvious entry point reads at a glance. Accent-fill ground + accent-hover ink, + slightly larger hit area, with bottom margin to part it from the groups below. + Layers on .dt-nav-link, so the .is-active treatment still overrides cleanly. */ +.dt-nav-start { + background: var(--accent-fill); color: var(--accent-hover); font-weight: 600; + padding: 8px 10px; margin-bottom: 12px; +} +.dt-nav-start:hover { background: var(--accent-fill-strong); color: var(--accent-hover); } +.dt-nav-start .dt-mi { color: var(--accent); } +.dt-nav-start.is-active { background: var(--accent-fill-strong); color: var(--accent-hover); } +.dt-nav-start.is-active .dt-mi { color: var(--accent); } .dt-nav-soon-tag { margin-left: auto; font-size: 9px; font-weight: 600; letter-spacing: 0.06em; text-transform: uppercase; color: var(--ink-tertiary); @@ -288,6 +301,24 @@ code, .dt-mono { font-family: var(--font-mono); font-size: 0.92em; font-feature- .dt-alert.error { background: var(--danger-fill); color: var(--danger); } .dt-alert code { background: rgba(0,0,0,0.05); padding: 1px 5px; border-radius: 4px; } +/* Next-step strip — slim single-line "what to do next" suggestion shown at the + end of a tool's results. Subtle accent ground + left accent rule so it nudges + without competing with alerts; the trailing dismiss control is unobtrusive. */ +.dt-next-step { + display: flex; align-items: center; gap: 10px; + background: var(--accent-fill); border-left: 3px solid var(--accent); + border-radius: var(--r-md); padding: 10px 14px; margin: 16px 0; + font-size: 13.5px; line-height: 1.4; color: var(--ink); +} +.dt-next-step .dt-mi { font-family: "Material Symbols Outlined"; font-size: 18px; color: var(--accent); flex-shrink: 0; } +.dt-next-step a { color: var(--accent); font-weight: 500; } +.dt-next-step a:hover { color: var(--accent-hover); } +.dt-next-step-dismiss { + margin-left: auto; background: transparent; border: none; cursor: pointer; + color: var(--ink-tertiary); font-size: 13px; line-height: 1; padding: 2px 4px; +} +.dt-next-step-dismiss:hover { color: var(--ink-secondary); } + /* =========================================================================== Inputs (static representations of Streamlit widgets) =========================================================================== */ diff --git a/layout-review/home.html b/layout-review/home.html index 38557da..6f2d8cb 100644 --- a/layout-review/home.html +++ b/layout-review/home.html @@ -96,6 +96,44 @@
+ +
+
+ + auto_awesome + +
+

Recommended

+

Runs the recommended clean — fix text, standardize formats, fill blanks, remove duplicates — in the right order, then hands you the cleaned file.

+
+ +
+ +
+ 1 · Clean Text + arrow_forward + 2 · Standardize + arrow_forward + 3 · Fix Missing + arrow_forward + 4 · Find Duplicates + Result downloads when finished +
+
+ + +

Or fix issues one at a time

+

Prefer to handle things yourself? Open any finding to jump straight to the right tool.

+
diff --git a/layout-review/shell.js b/layout-review/shell.js index 68aecc9..94c55f7 100644 --- a/layout-review/shell.js +++ b/layout-review/shell.js @@ -3,28 +3,32 @@ src/gui/components/_legacy.py:render_sticky_footer(). Each page sets to mark the active nav item. */ (function () { - // Sections + entries in the same order app.py registers them. + // Front-door entry — rendered standalone above the section groups. + var START = { id: "home", icon: "insert_chart_outlined", name: "Start here", href: "home.html" }; + + // Sections + entries in pipeline / job order. var NAV = [ - { label: "Analysis", items: [ - { id: "home", icon: "insert_chart_outlined", name: "File Analysis", href: "home.html" }, - { id: "11_reconciler", icon: "compare_arrows", name: "Reconcile Two Files", href: "11_reconciler.html" }, - ]}, { label: "Data Cleaners", items: [ - { id: "04_missing_handler", icon: "help_outline", name: "Fix Missing Values", href: "04_missing_handler.html" }, - { id: "06_outlier_detector", icon: "insights", name: "Find Unusual Values", href: "06_outlier_detector.html", soon: true }, { id: "02_text_cleaner", icon: "text_format", name: "Clean Text", href: "02_text_cleaner.html" }, { id: "03_format_standardizer", icon: "format_list_bulleted", name: "Standardize Formats", href: "03_format_standardizer.html" }, + { id: "04_missing_handler", icon: "help_outline", name: "Fix Missing Values", href: "04_missing_handler.html" }, { id: "01_deduplicator", icon: "search", name: "Find Duplicates", href: "01_deduplicator.html" }, - { id: "08_validator_reporter", icon: "check_circle", name: "Quality Check", href: "08_validator_reporter.html", soon: true }, ]}, { label: "Transformations", items: [ { id: "05_column_mapper", icon: "view_column", name: "Map Columns", href: "05_column_mapper.html" }, - { id: "07_multi_file_merger", icon: "account_tree", name: "Combine Files", href: "07_multi_file_merger.html", soon: true }, - { id: "10_pdf_extractor", icon: "picture_as_pdf", name: "PDF to CSV", href: "10_pdf_extractor.html" }, ]}, { label: "Automations", items: [ { id: "09_pipeline_runner", icon: "auto_awesome", name: "Automated Workflows", href: "09_pipeline_runner.html" }, ]}, + { label: "Finance", items: [ + { id: "11_reconciler", icon: "compare_arrows", name: "Reconcile Two Files", href: "11_reconciler.html" }, + { id: "10_pdf_extractor", icon: "picture_as_pdf", name: "PDF to CSV", href: "10_pdf_extractor.html" }, + ]}, + { label: "Coming soon", items: [ + { id: "06_outlier_detector", icon: "insights", name: "Find Unusual Values", href: "06_outlier_detector.html", soon: true }, + { id: "08_validator_reporter", icon: "check_circle", name: "Quality Check", href: "08_validator_reporter.html", soon: true }, + { id: "07_multi_file_merger", icon: "account_tree", name: "Combine Files", href: "07_multi_file_merger.html", soon: true }, + ]}, ]; var active = document.body.getAttribute("data-page") || ""; @@ -41,8 +45,13 @@ '' + '' + '