DataTools is local-first — "Upload" reads like "send data somewhere
remote", which contradicts the product positioning. Sweep replaces
the user-visible term throughout the UI:
- ``src/i18n/packs/en.json`` + ``es.json``: all ``upload.*`` strings
(heading, intro, uploader labels, empty state, switch-back, etc.)
and ``gate.default_name``. The ``intro_multi`` "no upload anywhere"
phrasing dropped the verb entirely — now reads "nothing leaves
this computer".
- All 9 tool pages: ``st.file_uploader(label="Upload …")`` →
``"Import …"``; matching ``st.info("Upload a …")`` empty-state
banners; ``help="Upload …"`` strings on disabled uploaders.
- ``9_Pipeline_Runner`` + ``5_Column_Mapper``: radio-option text
``"Upload schema/pipeline JSON"`` → ``"Import …"`` plus the
``.startswith("Upload")`` branch guards that read those values.
- ``_home.py``: "**Uploaded files**" → "**Imported files**".
- ``app_demo.py``: "Uploaded file is …" → "Imported file is …".
Internal identifiers left untouched: function names
(``pickup_or_upload``, ``_StashedUpload``), session-state keys
(``home_upload``, ``home_uploads``, ``home_uploaded_*``,
``merger_file_upload``), audit-log event category (``"upload"``),
Streamlit testid CSS selectors. None of those are visible to the
user.
The file_uploader's dropzone button text is a baked-in React
literal that Streamlit's ``label=`` doesn't reach; rewritten at the
DOM level with a small ``_RENAME_UPLOAD_BUTTON_JS`` snippet shipped
through ``st.iframe`` (same pattern the sticky footer uses to mount
on ``<body>``). A ``MutationObserver`` on the parent document re-
applies the swap when Streamlit remounts the dropzone after file
add/remove or page navigation, throttled via ``requestAnimationFrame``.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
``use_container_width`` is being removed after 2025-12-31. Streamlit
log was flooding the terminal with the deprecation notice on every
rerun. Mechanical sweep:
use_container_width=True → width="stretch"
use_container_width=False → width="content"
51 call sites across 11 page files + ``app_demo.py``. Also renamed
the ``local_download_button`` helper's ``use_container_width`` kwarg
to ``width`` (default ``"stretch"``); it has no external callers
passing the old name, so this is a safe rename.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Diagnostics confirmed the "white bar" the user has been describing is
not a separate element — it's ``[data-testid=stApp]``'s solid white
background (``rgb(255,255,255)``, viewport-locked) showing through the
gap between where page content ends and where the fixed Help/Close
footer overlay begins. ``stApp`` stays put while content scrolls
inside it, which is why the bar "doesn't change when scrolling".
The gap exists because ``render_sticky_footer`` overrides the block
container's ``padding-bottom`` to ``3rem`` (48px) to reserve clear
room for the fixed footer. The footer is only ~32-33px tall (min-
height 32px + 0.25rem top/bottom padding), so ~16px of that reserve
was pure visible white space sitting above the buttons.
Reduce ``padding-bottom`` to ``2rem`` (~32px) — just enough to
prevent content from rendering under the footer overlay, no more.
Eliminates the visible gap without exposing text to clipping.
Also remove the diagnostic banner + click-to-inspect iframe from
the home page now that the bar is identified.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User reported the previous diagnostic was too cluttered to read,
and the white bar showed no outline anyway — meaning the flat
``querySelectorAll('body *')`` walker missed it (likely inside an
iframe's contentDocument, which the script didn't recurse into).
New approach: a single red button "CLAUDE: click here, then click
the white bar" in the top-right. Clicking the button arms an
inspect handler. The next click anywhere on the page reports the
full element stack at that point via ``elementsFromPoint`` AND
recursively descends into any same-origin iframe at the click
location, so iframe contents are no longer invisible.
A black report panel lists every element in the stack with its
tag/id/testid/class, position, z-index, background color, and
bounding rect — TOP element highlighted in red. User clicks the
white bar exactly once and we know what it is.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous diagnostic only outlined fixed/sticky elements; user
confirmed the offending white bar isn't one of those. Cast a much
wider net:
- Outline every element whose visible rect intersects the bottom
200px of the viewport, regardless of position.
- Border style encodes position: solid=fixed, dashed=sticky,
dotted=absolute, thin=static/relative.
- Render a readable list in a top-right panel showing each element's
tag/id/testid/class, position, z-index, height, and background.
- Skip fully transparent + un-positioned elements (those can't
actually overlay anything).
With this, scroll to the bottom and the panel + colored outlines
will identify exactly which element is the white bar — fixed or
not. The user can paste the panel list (or just name the colored
box) so we know what to remove.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User reports: TEST #3 marker sits at the true bottom of the home
page's main content, but when scrolled the test text "goes behind"
an opaque white bar — confirming the bar is fixed/sticky (overlays
scrolling content). Our CSS only declares ONE fixed element near
the bottom (``#datatools-sticky-footer``), which the user already
ruled out. So something else — Streamlit native chrome, a third-
party widget, or a fixed element we haven't enumerated — is
overlaying the content.
Inject a small diagnostic iframe whose JS, running against the
parent document, walks every element on the page and outlines each
``position: fixed`` or ``position: sticky`` node with a distinct
color + a top-left label showing ``tagName#id[data-testid] pos=…
h=…px bg=…``. Re-runs after initial paint, on a couple of delays
(for late-mounting components), and on every scroll.
This is read-only — no DOM mutations beyond outline styles and
labels — so it's safe to ship even if I miss removing it.
The user can now visually identify which colored box is the
offending white bar and report its label.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User reported the previous TEST #2 banner appeared at the *top* of
the main content area instead of the bottom. Root cause: on the home
page, ``render_sticky_footer()`` is called at line 107 — before
``st.title()`` — so anything that function injects in document flow
lands at the top of ``stAppViewBlockContainer``. Other pages call
``render_sticky_footer()`` at the end of their script, so the flow
content lands at the bottom there.
Remove the marker from ``render_sticky_footer`` and add it directly
at the very end of ``_home._home_page()`` — after the findings
panels. If this banner lines up with the offending white strip when
scrolled to the bottom, the strip is something rendered at the tail
of the page (likely an iframe wrapper from ``render_findings_panel``
or the block container's ``padding-bottom``).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported: on the Home page after uploading data files, the Remove
buttons "on the right side" did nothing — the file kept showing up in
the list. That was the file_uploader widget's BUILT-IN ✕ icons (the
ones inside the uploader's chrome, on the right of each file row),
not our custom "Remove" buttons further down — the custom ones have
worked correctly since 84e4665.
Cause: ``_home_page`` deliberately treated the widget as add-only and
never honored widget-side removals. The reasoning, per the prior
comment, was that navigation can remount the widget with value ``[]``
— a render-time sync would then wipe ``home_uploads``. Real, but the
side effect was that the widget's own ✕ appeared to do nothing: the
file vanished from the widget chrome, stayed in ``home_uploads``, and
re-rendered immediately in the custom list below.
Fix: hook the file_uploader's ``on_change`` callback to reconcile
``home_uploads`` against the widget's current value. Streamlit's
``on_change`` fires ONLY on user-initiated value changes; the
remount-induced ``[]`` reset doesn't trigger it, so the stash still
survives navigation. Removals from the callback also drop the file's
findings entry and clear the singular ``home_uploaded_*`` keys when
the active upload was removed — matching the custom-button path.
The custom "Remove" buttons further down keep working unchanged; the
existing AppTest path through ``_home_remove_<sha1>`` still removes
exactly the file clicked. 2220 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The sticky footer was only wired into the 9 tool pages — the home
page (``_home.py``) called ``hide_streamlit_chrome`` but never
``render_sticky_footer``, so the app-level Close+Help bar was
missing whenever the user was on the home page. Add the call.
Also drop the home page's now-redundant trailing
``st.divider() + st.caption(t("chrome.footer"))`` block — same
"blank white bar above the sticky footer" symptom that motivated
removing the per-page version from the tool pages.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported: uploading multiple files on the home page and clicking Run
analysis blew up with
StreamlitDuplicateElementKey: key='_findings_open_02_text_cleaner'
when two uploaded files both had Clean Text findings.
Root cause: ``render_findings_panel`` is invoked once per uploaded
file from ``_home.py``, but the per-tool jump button used a
filename-agnostic key:
key=f"_findings_open_{tool_id}"
Two files both flagging Clean Text → two buttons with identical keys
→ Streamlit rejects the second one.
Fix:
- Add ``key_namespace: str = ""`` to ``render_findings_panel``. The
helper hashes it (sha1 truncated to 8 chars) and appends to every
button key, so different namespaces produce different keys but the
same namespace stays stable across reruns.
- The home page now passes the filename:
``render_findings_panel(findings, header=f"📄 {name}", key_namespace=name)``.
- The single-call site in ``upload_and_analyze_section`` (the legacy
helper, only used outside the new home-page path) keeps the default
empty namespace, which is fine because that path renders findings
for ONE file at a time.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New ``src/audit.py`` module records GUI actions to a per-session
JSONL file under ``~/.datatools/logs/`` (overrideable via
``DATATOOLS_AUDIT_DIR``). The file is human-readable (one JSON
object per line, each with a ``message`` field) AND trivially
machine-parseable — the support flow is "client mails the file,
we read it and explain what went wrong."
Format example::
{"ts":"2026-05-17T05:30:00.123+00:00","level":"info","category":"session",
"session":"a1b2c3d4","message":"Session started",
"platform":"Windows 11","python":"3.14.0","user":"Michael Dombaugh",
"log_file":"C:\\Users\\Michael Dombaugh\\.datatools\\logs\\datatools-...jsonl"}
{"ts":"...","category":"upload","message":"Uploaded customers.csv",
"filename":"customers.csv","bytes":24813}
{"ts":"...","category":"analyze","message":"Analyzed customers.csv (3 findings)",
"filename":"customers.csv","findings":3,"rows":120,"cols":8}
{"ts":"...","category":"tool_run","message":"Clean Text run",
"page":"2_Text_Cleaner"}
{"ts":"...","category":"error","level":"error",
"message":"analyze(weird.csv): EmptyDataError: No columns to parse",
"filename":"weird.csv","outcome":"empty_after_repair"}
Public API:
- ``log_event(category, message, **extra)``
- ``log_session_start()`` — idempotent banner with platform info
- ``log_page_open(slug)`` — emit a ``nav`` event, deduplicated per
Streamlit session so reruns don't spam the log
- ``log_exception(where, exc, **extra)`` — convenience wrapper
- ``audit_log_path()`` / ``audit_log_dir()`` — for the UI
Wired in at:
- ``hide_streamlit_chrome``: stamps session start, mounts a small
"🩺 Diagnostics" expander in the sidebar with the log path and
an "Open log folder" button so the user can grab the file to
attach to a support email.
- Home page: ``upload`` event on every new file, ``upload`` event
on per-file remove, ``analyze`` event with file count when
Run-analysis fires.
- ``_run_analysis_on_upload``: ``analyze`` event with rows / cols /
findings count per file, plus ``error`` events on every caught
exception (empty upload, empty after repair, pandas EmptyDataError,
generic Exception).
- Every Ready tool page (1, 2, 3, 4, 5, 9): ``tool_run`` event
immediately after the primary action stashes its result.
- Every tool page (1-9): ``log_page_open(slug)`` on render — deduped
via session state so we don't get one event per Streamlit rerun.
Safety:
- ``log_event`` wraps every write in try/except. A broken audit
log must NOT crash the GUI.
- Non-JSON-serializable extras are ``str()``-coerced before writing.
- File CONTENTS are never logged. We capture filename, byte count,
and (in the analyzer) a 12-char sha1 fingerprint of the bytes so
the same file re-uploaded gets the same trace.
- License keys, session cookies, etc. are not logged.
- ``DATATOOLS_AUDIT_DIR`` env var lets tests redirect writes into a
tmp dir.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported: the "✕" buttons on the uploaded file list removed files
inconsistently — some clicks took, some didn't.
Two compounding causes:
1. ``key=f"_home_remove_{name}"`` embedded the raw filename in the
Streamlit widget key. Streamlit's widget-identity machinery
normalizes keys differently across reruns when they contain
spaces, dots, brackets, or non-ASCII characters, so a button's
identity could shift between the render where the user clicked
it and the rerun that should have processed the click. The click
was registered, but the post-rerun render produced a new widget
under a new effective key, and the original click was "lost".
2. The handler mutated ``home_uploads`` mid-loop while subsequent
iterations were still creating buttons. ``st.rerun()`` raises
synchronously, but if ANOTHER button's state changed in the same
pass (e.g. a stale click held over from a fast double-tap), the
ordering of state-mutation vs widget-key-update vs rerun could
race.
Fixes:
- Stable widget keys: ``f"_home_remove_{sha1(name)[:10]}"``. The
hash is identifier-safe regardless of spaces / dots / Unicode in
the filename. Verified across "sample with spaces.csv",
"sample.csv", and "日本語.csv" — three sequential Remove clicks
each remove exactly one file with no clicks lost.
- Two-phase capture: the loop collects the target ``to_remove``
filename, finishes rendering every other row at consistent widget
identity, THEN mutates state once and reruns. No more mid-loop
``del`` racing other widgets' click handlers.
- Wider click target: column ratio ``[8, 1]`` (was ``[12, 1]``) and
``use_container_width=True`` on the Remove button so the click
surface fills the entire column. Label changed to "Remove" for
the same reason — "✕" is a thin glyph that compressed the
hit-test region.
2220 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported: clicking "Back to Home" from a tool page returned the user
to an empty home — their previously-uploaded files were gone.
Root cause: Streamlit's ``st.file_uploader`` widget state does not
reliably survive ``st.switch_page``. The widget gets unmounted on
navigation, and its ``UploadedFile`` objects don't always re-attach
on remount. The home page was treating the widget's return value as
the source of truth, so after navigation the list was empty.
Fix: introduce a session-state stash keyed by filename
(``home_uploads: dict[str, {"bytes": bytes, "size": int}]``) and
treat it as the source of truth for everything downstream — the
active-file pickup keys for tool pages, the per-file findings
cache, and the rendered file list. The widget is reduced to its
narrow role of capturing NEW uploads, which we merge into the stash
without ever removing.
Per-file remove: a "✕" button next to each filename drops just that
file (and its findings). The widget's own "✕" is bypassed by our
rendering, since trusting it would let the widget's state diverge
from the stash.
Clear-results button is unchanged: it wipes only the analysis cache,
leaving uploaded files intact (per the user's "persistent until
cleared" requirement — removal is per-file via "✕").
Tool-page compatibility: the singular ``home_uploaded_{name,size,
bytes}`` keys still get populated from the first entry in the stash
on every render, so ``pickup_or_upload`` on a tool page keeps
finding the active upload. When the user removes the active file,
those keys are cleared so the next render repopulates from whatever
file is now first.
``_StashedUpload`` is a small duck type ( ``.name``, ``.size``,
``.getvalue()`` ) so ``_run_analysis_on_upload`` accepts entries
restored from the stash without changes.
2220 tests pass. Smoke-verified via AppTest: pre-stashed
``home_uploads`` renders the file list with per-file remove buttons,
and the persistent state survives a simulated navigation round-trip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two issues, same fix surface.
(1) Reported crash on Back-to-Home:
StreamlitAPIException: Could not find page: app.py.
``st.switch_page("app.py")`` doesn't work under ``st.navigation`` —
the entry script is the nav manager itself and is not a registered
page. The fix needs to pass an ``st.Page`` object whose script
identity matches one registered in the nav.
First-pass attempt (``from src.gui.app import _home_page``) hit a
worse failure: importing ``app.py`` from inside a tool-page render
re-executes the nav setup with the WRONG "main script" context, so
every ``st.Page("pages/N_foo.py", ...)`` call in ``_build_navigation``
fails with "file could not be found".
Extract the home renderer into its own module ``src/gui/_home.py``
which has no top-level Streamlit side effects. Both the nav manager
and the back-link helper import ``_home_page`` from there. The Page
object built at click time has the same callable identity as the one
registered, so ``st.switch_page`` resolves it.
(2) Reported UX: the back button scrolled out of view on long pages.
Add a second ``back_to_home_link(key="_back_to_home_link_bottom")``
call near the footer of every tool page (1-9). The unique key avoids
widget-id collision with the top instance. Coming-Soon stubs get it
unconditionally; Ready tools render it only after a result exists
because the page short-circuits with ``st.stop()`` before then —
when no result is on screen the page is short enough that the top
link is sufficient.
2220 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
