Issue #1 (the make-or-break UX fix): after the analyzer runs, Home now
leads with a primary "Clean these files for me" CTA that runs the
recommended pipeline (Clean Text -> Standardize -> Fix Missing -> Find
Duplicates, in order) on every imported file and hands back a cleaned
CSV per file — collapsing "which tool, what order" to one click. The
existing per-finding cards remain, reframed as "Or fix issues one at a
time" for users who want manual control.
- Reuses the core API verbatim (recommended_pipeline + run_pipeline);
reader mirrors 9_Pipeline_Runner._read_uploaded so files load the same
way the standalone orchestrator loads them.
- Per-file errors are captured so one bad file doesn't kill the batch;
cleaned CSVs are cached in session_state so downloads survive reruns
and are pruned when a file is removed or re-analyzed.
Verified: the read -> run_pipeline -> CSV data path executes correctly
(compile + a non-Streamlit functional smoke test). The Streamlit UI
scaffolding (button / download_button / progress / session_state)
mirrors the proven runner page but still needs a `streamlit run` check.
Front-door copy is English literals for now; i18n keys are a follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bundles a handful of UX cleanups:
- Findings-card chevron moved to the LEFT side of the head. CSS still
rotates it 90° between collapsed/expanded states.
- Tool-link buttons in findings rows (``Clean Text →`` etc.) are now
left-justified against the icon column with minimal surrounding
whitespace. Action column ratio dropped from 1.8 → 1.4 and the
button switched from ``width="stretch"`` (centered text) to
``width="content"`` (shrinks to fit, left-aligned within column).
- Home-page hero now mirrors the sidebar brand block: 56px ink "D"
chip on the left + "UNALOGIX" eyebrow stacked above "DataTools"
wordmark, then the "Clean. Normalize. Transform." tagline beneath.
New ``.dt-page-brand / -row / -words / -mark / -eyebrow /
-wordmark`` rules in ``_DESIGN_TOKENS_CSS``. Streamlit wraps h1
elements in an emotion-cache div with extra padding; a descendant
flattener (``.dt-page-brand-words *`` margin:0 / padding:0) keeps
the eyebrow + wordmark stack the same height as the chip so they
center-align cleanly.
- Sidebar nav restyled to match the sticky-footer Help/Close buttons
exactly: 13px / 500 / 1.3 line-height, 5×10px padding, 8px gap
between icon and label, transparent background. Active item gets
the same ``rgba(0,0,0,0.04)`` tint as the hover state (no white
pill, no shadow), only the heavier weight + ink text distinguishes
it.
- OS app icon (page_icon) switched from SVG to a Pillow-rendered
``datatools_icon_256.png`` so Windows / macOS taskbar+dock pick
it up reliably (some OS shells fall back to a default icon for
SVG favicons). Rounded-square ink ground with cream "D" centered —
same mark as the sidebar chip + hero chip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements ``Business/DataTools/app_icons.html`` §03 "Letter D (sans)"
as the canonical app mark.
- New ``src/gui/assets/datatools_icon.svg`` — 64×64 SVG, 14px corner
radius, ink ground (#1c1917), cream "D" (#fef4ed) in
Geist 700 / -0.04em tracking. Pure SVG so it renders sharp at
every favicon size; font stack falls back through Geist →
system sans where the webfont isn't installed (favicons can't load
Google Fonts).
- ``_home.py``, ``_Activate.py``, ``99_Close.py``: page_icon now
resolves the SVG path via ``Path(__file__).parent / "assets" /
"datatools_icon.svg"`` instead of the broom 🧹 / 🔑 / 🛑
emojis. Streamlit inlines it as a ``data:image/svg+xml;base64,...``
link tag so the browser tab + OS app-icon for ``python -m src.gui``
matches the sidebar chip.
- Sidebar ``.dt-brand-mark`` tightened to match the spec's "Letter D
(sans)" rendering: ``font-weight: 700`` and
``letter-spacing: -0.04em`` (was 600 / -0.02em). The on-screen
chip is now a scaled-up copy of the OS icon.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Batch of UX tweaks the user asked for in quick succession:
- Sidebar brand block (mockup §brand) — 28px ink chip with a "D"
wordmark plus the "DataTools" text — injected into
``stSidebarHeader`` by a small JS bundled into the iframe-mounted
script that already runs from ``hide_streamlit_chrome``. The
Streamlit ``stLogoSpacer`` is hidden when the brand block is
present so it sits flush at the top of the sidebar.
- Findings cards are now collapsible. Each file's card head carries
``data-dt-collapsed="true"`` on first render; clicking the head
flips the attribute via the new ``_WIRE_COLLAPSIBLE_FINDINGS_JS``
(MutationObserver re-wires after reruns). A CSS rule
``[stElementContainer]:has(.dt-finding-group-head[data-dt-collapsed
="true"]) ~ *`` hides every later sibling of the head's element
container — covers both ``stLayoutWrapper`` (the columns rows in
this Streamlit release) and ``stElementContainer`` so the rule
survives future Streamlit layout renames. A chevron icon
(``chevron_right``) rotates 90° when expanded. The head itself
gets ``cursor: pointer`` + an accent-fill hover.
- Tool-link buttons in finding rows dropped the leading ``Open`` —
now read ``Clean Text →``, ``Standardize Formats →`` etc.
- Finding-row column order: action is now LEFT of the description,
matching user feedback (``[icon] [Tool →] [description + meta]``).
- Head padding bumped to ``16px 22px`` so the filename has visible
breathing room from the card's left edge (previously the mono
filename felt like it was bleeding into the rounded corner).
- Head margin-bottom bumped to 1.5rem for breathing room before the
first finding row when expanded; collapsed state tucks the head
flush against the card bottom with full ``--r-lg`` corner radius
and no visible bottom border.
- Files card row layout: ``✕`` button moved to the LEFT of the
filename (``[✕] [chip + filename] [size]``).
- Sidebar nav rows tightened: link padding 7px → 4px, line-height
1.25, 1px margin-bottom per li, section-header padding-top reduced.
Plus a new ``--gap: 0.25rem`` rule for vertical blocks inside
bordered containers so the Files card and findings card body have
denser inter-row spacing.
- Sidebar Language selector restyled: widget labels render as the
spec's "Eyebrow" row (11.5px / 500 / 0.08em uppercase, tertiary
ink), selectbox combobox gets a paper surface + soft border that
matches the rest of the sidebar chrome.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The disabled "Export report" placeholder is gone — it wasn't tied to
a real feature and was just noise in the action bar. Action bar is
back to two buttons (Run analysis · Clear results) on a 1:1:4
column split. ``upload.export_report`` keys removed from en + es
i18n packs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-row file sizes and the Files-card total-size meta both read as
human-readable units now. Smallest unit is KB even for sub-kilobyte
files (so ``538 B`` → ``0.5 KB``, ``4914 B`` → ``4.8 KB``), steps up
to MB at 1 MiB and GB at 1 GiB. Always one decimal place.
New module-level helper ``_format_size(int) -> str`` in ``_home.py``;
both the section meta (``1 file · 4.8 KB total``) and the per-row
``dt-file-size`` cell call it instead of the previous ad-hoc
``f"{n:,} B"`` formatter. Keeps the display consistent regardless of
file size — and keeps the GUI free of raw byte counts that nobody
needs to read.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mockup §file-add lands as the canonical import affordance:
- Streamlit's ``st.file_uploader`` widget is still mounted (only path
that actually receives browser file events), but parked off-screen
via a new ``[data-testid="stFileUploader"] { position:absolute;
left:-10000px; … pointer-events:none }`` rule. Its hidden
``<input type="file">`` stays reachable to JavaScript.
- The Files card is now always rendered (header + bordered body).
The bottom row of the card is a ``button.dt-file-add`` styled per
mockup §file-add: dashed top border bleeding to the card edges,
surface-hover background, ``+ Add more files`` text in
``--ink-secondary``, accent-fill on hover.
- A small ``<script>`` shipped through ``st.iframe`` wires the
button: ``click → input.click()`` on the off-screen
``stFileUploaderDropzoneInput``. Streamlit's HTML sanitizer
strips inline ``onclick`` from ``unsafe_allow_html`` content, so
the binding has to come from a real script element — same pattern
the sticky footer and Upload→Import rewriter use. A
``MutationObserver`` re-wires the button when Streamlit remounts
it across reruns. The ``dataset.dtWired`` guard prevents double
binding.
Section structure also tightened to match the mockup:
- Section heading is now ``<h2>Files</h2>`` (was ``### Import one
or more files to start``) with the count + total size on the
right of the same flex row. When no files: ``No files imported
yet``. When files exist: ``1 file · 4.8 KB total``.
- Dropped the ``upload.intro_multi`` caption and the
``upload.empty_state`` info banner — the card itself plus the
in-card Add button cover both prompts.
- Empty state now ends after the Files card (no stats / no action
bar / no findings rendered) — matches mockup's single-section
empty view.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the remaining gaps between the live home page and the
``datatools_layout_redesign2.html`` mockup. Four pieces land
together because they all consume the same new CSS scaffold:
1. Page header (§page-header)
``st.title`` + ``st.caption`` + ``st.divider`` collapse into one
flex header: h1 + body subtitle on the left, ``Runs 100% locally``
privacy pill (success-fill + lock SVG) on the right, soft border
below. The "Runs 100% locally" phrase moved out of
``home.caption`` into the new ``home.privacy_pill`` i18n key
(en + es).
2. Files card (§files-card)
The "Imported files" list is now a single bordered card with a
section head (count + KB total on the right, mockup §section-head).
Each row renders a 28px accent-fill chip carrying the inline
document SVG, a mono filename, a right-aligned mono size, and a
compact ``✕`` button. The word-button ``Remove`` is gone —
replaced by an icon-only tertiary button styled via a new CSS
rule that goes transparent → danger-fill on hover (mockup
§file-remove).
3. Action bar (§action-bar)
Three buttons in one row: ``Run analysis`` (primary ink), a new
disabled ``Export report`` (secondary; coming soon, tooltip), and
``Clear results``. New i18n key ``upload.export_report``.
4. Findings — per-file group cards (§finding-group)
``render_findings_panel`` rewritten end-to-end. Output is now:
• A head row (``dt-finding-group-head``) bleeding to the card
edges: worst-severity dot · mono filename · count pills
enumerating non-zero severities (e.g. ``2 info`` blue,
``1 warning`` amber, ``1 error`` rose).
• A flat list of finding rows sorted error → warn → info.
Each row: tinted Material-icon chip + title (description
with optional ``<code>`` column chip) + mono meta line
(rows affected, samples captured) + tertiary
``Open <Tool> →`` action button that ``st.switch_page``s
to the relevant tool.
The previous tool-grouped expander stack is dropped — the new
layout is denser and matches the mockup's single-card-per-file
structure.
``_render_one_finding`` (the old per-finding helper that emitted
markdown lines + sample tables) remains in the file but is no
longer called from the home flow; left in place for any other
surface that still depends on the markdown style.
The "no issues" success state renders a green dot + mono
filename + ``no issues`` success pill in the same card chrome,
so empty-result files visually match the rest of the panel
rather than getting a generic ``st.success`` callout.
CSS additions (``_DESIGN_TOKENS_CSS``):
``.dt-page-header / .dt-page-subtitle / .dt-privacy-pill``
``.dt-files-section-head / .dt-section-meta``
``.dt-file-row / .dt-file-icon-chip / .dt-file-name / .dt-file-size``
``.dt-finding-group-head / .dt-severity-dot{.warn,.info,.error,.success}``
``.dt-group-filename / .dt-group-counts``
``.dt-count-pill{.warn,.info,.error,.success}``
``.dt-finding-row / .dt-finding-icon{.warn,.info,.error}``
``.dt-finding-title / .dt-finding-meta``
Tertiary button rule (transparent → danger-fill on hover) for
the X button and the ``Open Tool →`` row action.
theme.py:
Explicitly loads Material Symbols Outlined alongside Geist —
the severity-chip ligatures (``info`` / ``warning`` / ``error``)
need the font present even when no ``:material/`` token has been
emitted yet on the page. Tightened ``.dt-finding-icon .dt-mui``
selector with ``[data-testid="stMarkdownContainer"]``-scoped
variant so the Material font wins over theme.py's base
``var(--font-sans) !important`` on markdown descendants.
Leading section-heading emojis stripped from i18n
(``upload.heading``) for parity with the mockup's clean ``Files``
/ ``Findings`` h2s.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two pieces of the mockup 2 layout that hadn't landed yet:
1. Sidebar nav icons — emoji glyphs (🧹✂️🔍 …) swapped for
Streamlit's ``:material/<name>:`` syntax, picking the outline
Material Symbol that best matches each mockup SVG:
Home → :material/home:
Fix Missing Values → :material/help_outline:
Find Unusual Vals → :material/insights:
Clean Text → :material/text_format:
Standardize Fmts → :material/format_list_bulleted:
Find Duplicates → :material/search:
Quality Check → :material/check_circle:
Map Columns → :material/view_column:
Combine Files → :material/account_tree:
Auto Workflows → :material/auto_awesome:
Activate → :material/key:
Close → :material/close:
Streamlit injects the icon name as a literal ligature inside a
first-child ``<span>`` of the nav anchor, expected to render
through the Material Symbols font. theme.py's base rule was
forcing Geist on every span under ``stSidebarNav``, turning the
ligatures back into plain text labels — added a structural
exception that targets ``[data-testid="stSidebarNavLink"] >
span:first-child`` (and any descendant), restoring the Material
font family, neutralizing the inherited ``ss01/cv01/cv11``
feature settings, and sizing to 18px.
Also stripped the leading emojis from every page title in the
en/es i18n packs (``home.title``, ``close_page.title``,
``activation.title``, ``tools.*.page_title``) — the icons live
in the sidebar now, the page H1 no longer needs to carry one.
2. Stats overview on home — new ``_render_stats_overview`` in
_home.py emits a 4-card grid above the per-file findings panels:
Files analyzed, Total findings, Warnings (severity ``warn`` ∪
``error``), Info (severity ``info``). Card layout follows the
mockup §stats verbatim — Geist 28px / 600 / -0.03em for the
numeric value (the "Display number" row in spec §4), tiny
uppercase tracked label, paper-surface card with the standard
warm border + faint shadow. The Warnings / Info cards tint the
number with ``--warn`` / ``--info`` when the count is non-zero.
CSS for ``.dt-stats / .dt-stat / .dt-stat-label / .dt-stat-value /
.dt-stat-unit`` added to ``_DESIGN_TOKENS_CSS``; falls to a
2-column grid below 900px viewport, matching the mockup's media
query.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DataTools is local-first — "Upload" reads like "send data somewhere
remote", which contradicts the product positioning. Sweep replaces
the user-visible term throughout the UI:
- ``src/i18n/packs/en.json`` + ``es.json``: all ``upload.*`` strings
(heading, intro, uploader labels, empty state, switch-back, etc.)
and ``gate.default_name``. The ``intro_multi`` "no upload anywhere"
phrasing dropped the verb entirely — now reads "nothing leaves
this computer".
- All 9 tool pages: ``st.file_uploader(label="Upload …")`` →
``"Import …"``; matching ``st.info("Upload a …")`` empty-state
banners; ``help="Upload …"`` strings on disabled uploaders.
- ``9_Pipeline_Runner`` + ``5_Column_Mapper``: radio-option text
``"Upload schema/pipeline JSON"`` → ``"Import …"`` plus the
``.startswith("Upload")`` branch guards that read those values.
- ``_home.py``: "**Uploaded files**" → "**Imported files**".
- ``app_demo.py``: "Uploaded file is …" → "Imported file is …".
Internal identifiers left untouched: function names
(``pickup_or_upload``, ``_StashedUpload``), session-state keys
(``home_upload``, ``home_uploads``, ``home_uploaded_*``,
``merger_file_upload``), audit-log event category (``"upload"``),
Streamlit testid CSS selectors. None of those are visible to the
user.
The file_uploader's dropzone button text is a baked-in React
literal that Streamlit's ``label=`` doesn't reach; rewritten at the
DOM level with a small ``_RENAME_UPLOAD_BUTTON_JS`` snippet shipped
through ``st.iframe`` (same pattern the sticky footer uses to mount
on ``<body>``). A ``MutationObserver`` on the parent document re-
applies the swap when Streamlit remounts the dropzone after file
add/remove or page navigation, throttled via ``requestAnimationFrame``.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
``use_container_width`` is being removed after 2025-12-31. Streamlit
log was flooding the terminal with the deprecation notice on every
rerun. Mechanical sweep:
use_container_width=True → width="stretch"
use_container_width=False → width="content"
51 call sites across 11 page files + ``app_demo.py``. Also renamed
the ``local_download_button`` helper's ``use_container_width`` kwarg
to ``width`` (default ``"stretch"``); it has no external callers
passing the old name, so this is a safe rename.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Diagnostics confirmed the "white bar" the user has been describing is
not a separate element — it's ``[data-testid=stApp]``'s solid white
background (``rgb(255,255,255)``, viewport-locked) showing through the
gap between where page content ends and where the fixed Help/Close
footer overlay begins. ``stApp`` stays put while content scrolls
inside it, which is why the bar "doesn't change when scrolling".
The gap exists because ``render_sticky_footer`` overrides the block
container's ``padding-bottom`` to ``3rem`` (48px) to reserve clear
room for the fixed footer. The footer is only ~32-33px tall (min-
height 32px + 0.25rem top/bottom padding), so ~16px of that reserve
was pure visible white space sitting above the buttons.
Reduce ``padding-bottom`` to ``2rem`` (~32px) — just enough to
prevent content from rendering under the footer overlay, no more.
Eliminates the visible gap without exposing text to clipping.
Also remove the diagnostic banner + click-to-inspect iframe from
the home page now that the bar is identified.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User reported the previous diagnostic was too cluttered to read,
and the white bar showed no outline anyway — meaning the flat
``querySelectorAll('body *')`` walker missed it (likely inside an
iframe's contentDocument, which the script didn't recurse into).
New approach: a single red button "CLAUDE: click here, then click
the white bar" in the top-right. Clicking the button arms an
inspect handler. The next click anywhere on the page reports the
full element stack at that point via ``elementsFromPoint`` AND
recursively descends into any same-origin iframe at the click
location, so iframe contents are no longer invisible.
A black report panel lists every element in the stack with its
tag/id/testid/class, position, z-index, background color, and
bounding rect — TOP element highlighted in red. User clicks the
white bar exactly once and we know what it is.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous diagnostic only outlined fixed/sticky elements; user
confirmed the offending white bar isn't one of those. Cast a much
wider net:
- Outline every element whose visible rect intersects the bottom
200px of the viewport, regardless of position.
- Border style encodes position: solid=fixed, dashed=sticky,
dotted=absolute, thin=static/relative.
- Render a readable list in a top-right panel showing each element's
tag/id/testid/class, position, z-index, height, and background.
- Skip fully transparent + un-positioned elements (those can't
actually overlay anything).
With this, scroll to the bottom and the panel + colored outlines
will identify exactly which element is the white bar — fixed or
not. The user can paste the panel list (or just name the colored
box) so we know what to remove.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User reports: TEST #3 marker sits at the true bottom of the home
page's main content, but when scrolled the test text "goes behind"
an opaque white bar — confirming the bar is fixed/sticky (overlays
scrolling content). Our CSS only declares ONE fixed element near
the bottom (``#datatools-sticky-footer``), which the user already
ruled out. So something else — Streamlit native chrome, a third-
party widget, or a fixed element we haven't enumerated — is
overlaying the content.
Inject a small diagnostic iframe whose JS, running against the
parent document, walks every element on the page and outlines each
``position: fixed`` or ``position: sticky`` node with a distinct
color + a top-left label showing ``tagName#id[data-testid] pos=…
h=…px bg=…``. Re-runs after initial paint, on a couple of delays
(for late-mounting components), and on every scroll.
This is read-only — no DOM mutations beyond outline styles and
labels — so it's safe to ship even if I miss removing it.
The user can now visually identify which colored box is the
offending white bar and report its label.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User reported the previous TEST #2 banner appeared at the *top* of
the main content area instead of the bottom. Root cause: on the home
page, ``render_sticky_footer()`` is called at line 107 — before
``st.title()`` — so anything that function injects in document flow
lands at the top of ``stAppViewBlockContainer``. Other pages call
``render_sticky_footer()`` at the end of their script, so the flow
content lands at the bottom there.
Remove the marker from ``render_sticky_footer`` and add it directly
at the very end of ``_home._home_page()`` — after the findings
panels. If this banner lines up with the offending white strip when
scrolled to the bottom, the strip is something rendered at the tail
of the page (likely an iframe wrapper from ``render_findings_panel``
or the block container's ``padding-bottom``).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported: on the Home page after uploading data files, the Remove
buttons "on the right side" did nothing — the file kept showing up in
the list. That was the file_uploader widget's BUILT-IN ✕ icons (the
ones inside the uploader's chrome, on the right of each file row),
not our custom "Remove" buttons further down — the custom ones have
worked correctly since 84e4665.
Cause: ``_home_page`` deliberately treated the widget as add-only and
never honored widget-side removals. The reasoning, per the prior
comment, was that navigation can remount the widget with value ``[]``
— a render-time sync would then wipe ``home_uploads``. Real, but the
side effect was that the widget's own ✕ appeared to do nothing: the
file vanished from the widget chrome, stayed in ``home_uploads``, and
re-rendered immediately in the custom list below.
Fix: hook the file_uploader's ``on_change`` callback to reconcile
``home_uploads`` against the widget's current value. Streamlit's
``on_change`` fires ONLY on user-initiated value changes; the
remount-induced ``[]`` reset doesn't trigger it, so the stash still
survives navigation. Removals from the callback also drop the file's
findings entry and clear the singular ``home_uploaded_*`` keys when
the active upload was removed — matching the custom-button path.
The custom "Remove" buttons further down keep working unchanged; the
existing AppTest path through ``_home_remove_<sha1>`` still removes
exactly the file clicked. 2220 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The sticky footer was only wired into the 9 tool pages — the home
page (``_home.py``) called ``hide_streamlit_chrome`` but never
``render_sticky_footer``, so the app-level Close+Help bar was
missing whenever the user was on the home page. Add the call.
Also drop the home page's now-redundant trailing
``st.divider() + st.caption(t("chrome.footer"))`` block — same
"blank white bar above the sticky footer" symptom that motivated
removing the per-page version from the tool pages.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported: uploading multiple files on the home page and clicking Run
analysis blew up with
StreamlitDuplicateElementKey: key='_findings_open_02_text_cleaner'
when two uploaded files both had Clean Text findings.
Root cause: ``render_findings_panel`` is invoked once per uploaded
file from ``_home.py``, but the per-tool jump button used a
filename-agnostic key:
key=f"_findings_open_{tool_id}"
Two files both flagging Clean Text → two buttons with identical keys
→ Streamlit rejects the second one.
Fix:
- Add ``key_namespace: str = ""`` to ``render_findings_panel``. The
helper hashes it (sha1 truncated to 8 chars) and appends to every
button key, so different namespaces produce different keys but the
same namespace stays stable across reruns.
- The home page now passes the filename:
``render_findings_panel(findings, header=f"📄 {name}", key_namespace=name)``.
- The single-call site in ``upload_and_analyze_section`` (the legacy
helper, only used outside the new home-page path) keeps the default
empty namespace, which is fine because that path renders findings
for ONE file at a time.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New ``src/audit.py`` module records GUI actions to a per-session
JSONL file under ``~/.datatools/logs/`` (overrideable via
``DATATOOLS_AUDIT_DIR``). The file is human-readable (one JSON
object per line, each with a ``message`` field) AND trivially
machine-parseable — the support flow is "client mails the file,
we read it and explain what went wrong."
Format example::
{"ts":"2026-05-17T05:30:00.123+00:00","level":"info","category":"session",
"session":"a1b2c3d4","message":"Session started",
"platform":"Windows 11","python":"3.14.0","user":"Michael Dombaugh",
"log_file":"C:\\Users\\Michael Dombaugh\\.datatools\\logs\\datatools-...jsonl"}
{"ts":"...","category":"upload","message":"Uploaded customers.csv",
"filename":"customers.csv","bytes":24813}
{"ts":"...","category":"analyze","message":"Analyzed customers.csv (3 findings)",
"filename":"customers.csv","findings":3,"rows":120,"cols":8}
{"ts":"...","category":"tool_run","message":"Clean Text run",
"page":"2_Text_Cleaner"}
{"ts":"...","category":"error","level":"error",
"message":"analyze(weird.csv): EmptyDataError: No columns to parse",
"filename":"weird.csv","outcome":"empty_after_repair"}
Public API:
- ``log_event(category, message, **extra)``
- ``log_session_start()`` — idempotent banner with platform info
- ``log_page_open(slug)`` — emit a ``nav`` event, deduplicated per
Streamlit session so reruns don't spam the log
- ``log_exception(where, exc, **extra)`` — convenience wrapper
- ``audit_log_path()`` / ``audit_log_dir()`` — for the UI
Wired in at:
- ``hide_streamlit_chrome``: stamps session start, mounts a small
"🩺 Diagnostics" expander in the sidebar with the log path and
an "Open log folder" button so the user can grab the file to
attach to a support email.
- Home page: ``upload`` event on every new file, ``upload`` event
on per-file remove, ``analyze`` event with file count when
Run-analysis fires.
- ``_run_analysis_on_upload``: ``analyze`` event with rows / cols /
findings count per file, plus ``error`` events on every caught
exception (empty upload, empty after repair, pandas EmptyDataError,
generic Exception).
- Every Ready tool page (1, 2, 3, 4, 5, 9): ``tool_run`` event
immediately after the primary action stashes its result.
- Every tool page (1-9): ``log_page_open(slug)`` on render — deduped
via session state so we don't get one event per Streamlit rerun.
Safety:
- ``log_event`` wraps every write in try/except. A broken audit
log must NOT crash the GUI.
- Non-JSON-serializable extras are ``str()``-coerced before writing.
- File CONTENTS are never logged. We capture filename, byte count,
and (in the analyzer) a 12-char sha1 fingerprint of the bytes so
the same file re-uploaded gets the same trace.
- License keys, session cookies, etc. are not logged.
- ``DATATOOLS_AUDIT_DIR`` env var lets tests redirect writes into a
tmp dir.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported: the "✕" buttons on the uploaded file list removed files
inconsistently — some clicks took, some didn't.
Two compounding causes:
1. ``key=f"_home_remove_{name}"`` embedded the raw filename in the
Streamlit widget key. Streamlit's widget-identity machinery
normalizes keys differently across reruns when they contain
spaces, dots, brackets, or non-ASCII characters, so a button's
identity could shift between the render where the user clicked
it and the rerun that should have processed the click. The click
was registered, but the post-rerun render produced a new widget
under a new effective key, and the original click was "lost".
2. The handler mutated ``home_uploads`` mid-loop while subsequent
iterations were still creating buttons. ``st.rerun()`` raises
synchronously, but if ANOTHER button's state changed in the same
pass (e.g. a stale click held over from a fast double-tap), the
ordering of state-mutation vs widget-key-update vs rerun could
race.
Fixes:
- Stable widget keys: ``f"_home_remove_{sha1(name)[:10]}"``. The
hash is identifier-safe regardless of spaces / dots / Unicode in
the filename. Verified across "sample with spaces.csv",
"sample.csv", and "日本語.csv" — three sequential Remove clicks
each remove exactly one file with no clicks lost.
- Two-phase capture: the loop collects the target ``to_remove``
filename, finishes rendering every other row at consistent widget
identity, THEN mutates state once and reruns. No more mid-loop
``del`` racing other widgets' click handlers.
- Wider click target: column ratio ``[8, 1]`` (was ``[12, 1]``) and
``use_container_width=True`` on the Remove button so the click
surface fills the entire column. Label changed to "Remove" for
the same reason — "✕" is a thin glyph that compressed the
hit-test region.
2220 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported: clicking "Back to Home" from a tool page returned the user
to an empty home — their previously-uploaded files were gone.
Root cause: Streamlit's ``st.file_uploader`` widget state does not
reliably survive ``st.switch_page``. The widget gets unmounted on
navigation, and its ``UploadedFile`` objects don't always re-attach
on remount. The home page was treating the widget's return value as
the source of truth, so after navigation the list was empty.
Fix: introduce a session-state stash keyed by filename
(``home_uploads: dict[str, {"bytes": bytes, "size": int}]``) and
treat it as the source of truth for everything downstream — the
active-file pickup keys for tool pages, the per-file findings
cache, and the rendered file list. The widget is reduced to its
narrow role of capturing NEW uploads, which we merge into the stash
without ever removing.
Per-file remove: a "✕" button next to each filename drops just that
file (and its findings). The widget's own "✕" is bypassed by our
rendering, since trusting it would let the widget's state diverge
from the stash.
Clear-results button is unchanged: it wipes only the analysis cache,
leaving uploaded files intact (per the user's "persistent until
cleared" requirement — removal is per-file via "✕").
Tool-page compatibility: the singular ``home_uploaded_{name,size,
bytes}`` keys still get populated from the first entry in the stash
on every render, so ``pickup_or_upload`` on a tool page keeps
finding the active upload. When the user removes the active file,
those keys are cleared so the next render repopulates from whatever
file is now first.
``_StashedUpload`` is a small duck type ( ``.name``, ``.size``,
``.getvalue()`` ) so ``_run_analysis_on_upload`` accepts entries
restored from the stash without changes.
2220 tests pass. Smoke-verified via AppTest: pre-stashed
``home_uploads`` renders the file list with per-file remove buttons,
and the persistent state survives a simulated navigation round-trip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two issues, same fix surface.
(1) Reported crash on Back-to-Home:
StreamlitAPIException: Could not find page: app.py.
``st.switch_page("app.py")`` doesn't work under ``st.navigation`` —
the entry script is the nav manager itself and is not a registered
page. The fix needs to pass an ``st.Page`` object whose script
identity matches one registered in the nav.
First-pass attempt (``from src.gui.app import _home_page``) hit a
worse failure: importing ``app.py`` from inside a tool-page render
re-executes the nav setup with the WRONG "main script" context, so
every ``st.Page("pages/N_foo.py", ...)`` call in ``_build_navigation``
fails with "file could not be found".
Extract the home renderer into its own module ``src/gui/_home.py``
which has no top-level Streamlit side effects. Both the nav manager
and the back-link helper import ``_home_page`` from there. The Page
object built at click time has the same callable identity as the one
registered, so ``st.switch_page`` resolves it.
(2) Reported UX: the back button scrolled out of view on long pages.
Add a second ``back_to_home_link(key="_back_to_home_link_bottom")``
call near the footer of every tool page (1-9). The unique key avoids
widget-id collision with the top instance. Coming-Soon stubs get it
unconditionally; Ready tools render it only after a result exists
because the page short-circuits with ``st.stop()`` before then —
when no result is on screen the page is short enough that the top
link is sufficient.
2220 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
