Files

Michael abb720997e docs: tight, scannable rewrite — every item earns its place

Refactors all 10 docs (README, USER-GUIDE, CLI-REFERENCE, REQUIREMENTS,
TECHNICAL, DEVELOPER, BUSINESS, DECISIONS, RECOVERY, docs/README) from
prose-heavy to bullet-heavy + table-heavy. Same information density,
significantly less reading load.

Net: 2600 → 1652 lines (~37% reduction) WHILE adding the new content
that landed since v1.6:

- Format Standardizer (3rd Ready tool)
- 199-row buyer corpus
- src/core/errors.py structured hierarchy + ensure_dataframe /
  ensure_choice / wrap_file_read|write / format_for_user helpers
- src/core/_constants.py shared USPS/state lookup tables
- Cross-tool audit fixes (NaN matching, removed_df schema, validation,
  enum-bounds checks, forward-compat config)
- Per-domain error_policy across format standardizers
- Inconsistent-date-format detector
- Excel header-row auto-detection + write_file delimiter param

Per-doc changes:

- README.md (175 → 71): 9-tool table at top, status column, 3 CLI
  entry points listed, dropped repeated marketing prose.
- docs/README.md (38 → 27): pure index — buyer-facing vs creator-only
  split + version footer.
- USER-GUIDE.md (208 → 118): tool table replaces script descriptions,
  troubleshooting compressed to bullets, gate explanation tightened.
- CLI-REFERENCE.md (451 → 235): collapsed flag tables, removed
  redundant intro text, kept full recipes section.
- REQUIREMENTS.md (146 → 129): 18 numbered sections (was 17), added
  §18 Error Handling, formatting tightened to single-line entries.
- TECHNICAL.md (570 → 350): collapsed §3 build pipeline tables, merged
  redundant §3.5-3.7 OS sections, added §7 (Error handling) +
  §11.3 (Format Standardizer spec) + §11.4-11.7 (analyzer / gate /
  Review page / repair_bytes promoted from §10.2.x sub-numbering).
- DEVELOPER.md (285 → 161): module map table replaces per-file prose,
  extension recipes condensed, new §Errors covers when to use each
  hierarchy class.
- BUSINESS.md (278 → 225): collapsed prose to tables (use cases,
  competitive landscape, costs, risks); honest-status updated.
- DECISIONS.md (269 → 189): scoring rubric + GUI matrix preserved,
  decision log compressed to single-line entries, added v1.6 entries
  (Format Standardizer Ready, errors module).
- RECOVERY.md (180 → 147): rebuild steps as numbered + tabular,
  external dependencies as one table, recovery priorities tightened.

No information removed; redundancy compressed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-01 02:49:29 +00:00

4.9 KiB

Raw Blame History

User Guide

Version: 1.6 · Updated: 2026-05-01

1. Install

You don't need Python — the bundle is self-contained.

OS	File	How
Windows	`BundleName-Setup-1.0.exe`	Double-click installer → desktop shortcut.
macOS	`BundleName-1.0.dmg`	Mount, drag to Applications. Signed + notarized.
Linux	`BundleName-1.0.AppImage`	`chmod +x`, double-click. (`.tar.gz` fallback available.)

Launching opens your default browser to a local page (http://localhost:8501).

How the GUI works

Runs locally on your machine. No internet, no upload.
Browser is just the display surface. Closing it stops the underlying program.
Prefer the terminal? Every tool ships with a CLI too (Section 3).

System requirements

Windows 10/11 (64-bit), macOS 11+, modern Linux (2020+).
Modern browser (Chrome, Edge, Firefox, Safari, last 3 years).
~400-500 MB free disk space.

Full numbered support matrix: REQUIREMENTS.md.

2. What's included

#	Tool	Purpose	Status
01	Deduplicator	Exact + fuzzy match, 5 normalizers, audit	Ready
02	Text Cleaner	Whitespace, smart chars, BOM, line endings, case ops	Ready
03	Format Standardizer	Dates / phones / emails / addresses / names / currencies / booleans	Ready
04	Missing Value Handler	Disguised nulls, imputation, drop-by-threshold	Coming Soon
05	Column Mapper	Rename + enforce schema	Coming Soon
06	Outlier Detector	z-score, IQR, multivariate	Coming Soon
07	Multi-File Merger	Combine multiple files	Coming Soon
08	Validator & Reporter	Rules + PDF/Excel report	Coming Soon
09	Pipeline Runner	One-click multi-tool launcher	Coming Soon

Sample data (samples/): messy_sales.csv, bank_export.xlsx.

3. Usage

3.1 GUI (recommended)

Launch the bundle.
Pick a tool from the sidebar.
Drop your file (or select a sample).
Defaults are pre-filled — click Run to preview.
Click Save Output to write the cleaned file.

Advanced options are tucked in expander panes. The original file is never modified.

3.2 CLI

deduplicator       customers.csv [--apply]
text-cleaner       messy.csv     [--apply]
format-standardize feed.csv      [--apply]

Get help: deduplicator --help. Full reference: CLI-REFERENCE.md.

3.3 Run order (when running tools manually)

If you skip the Pipeline Runner, follow this order:

02 Text Cleaner first — normalizes whitespace + special chars.
03 Format Standardizer — dates, phones, etc. need cleaned text.
04 Missing Value Handler — sentinel codes hide as numbers.
05 Column Mapper — schema before outlier stats.
06 Outlier Detector — needs clean numerics. Stats on data with NaN or -999 are mathematically poisoned.
07 Multi-File Merger, 08 Validator as needed.
01 Deduplicator is order-flexible (normalizes internally for matching).

The Pipeline Runner enforces this automatically.

4. Review & Normalize gate

Every uploaded file is scanned before any tool sees it.

Confidence tiers:

High — round-trip safe. One-click "Auto-fix high-confidence" applies them all.
Medium — usually right, occasional false positives. Preview first.
Low — heuristic. Off by default; opt in per finding.
Error — blocks the gate (empty file, U+FFFD, unrepairable rows).

Encoding override: when the picker reports encoding_uncertain or you spot mojibake (Ã©) or <EFBFBD> chars, choose the right codepage at the top of the page (cp1252 for Western Excel, KOI8-R for older Russian, Big5 for traditional Chinese, …) → Re-analyze.

Advanced output: an ⚙️ expander on the download lets you tune encoding, delimiter, and line terminator. The download filename auto-adjusts (.tsv for tab, .csv otherwise).

5. Output

Every run writes:

Cleaned file next to the input (or wherever you specify).
Audit file (per-cell changes for text/format tools, match groups for dedup).
Timestamped log in logs/.

Original input is never modified.

6. Troubleshooting

GUI won't launch / browser doesn't open — wait 10-15 s; manually visit http://localhost:8501. Port-in-use error → close other instances.
Why does my browser open? — local web app pattern (same as Jupyter, RStudio). Nothing leaves your machine.
Windows SmartScreen — click "More info" → "Run anyway". Standard for non-EV-signed software.
macOS "App is damaged" — re-download (file likely corrupted in transit).
Linux AppImage won't run — chmod +x file.AppImage. Missing FUSE → sudo apt install libfuse2 or use .tar.gz.
Slow on large file — over ~100k rows takes longer; progress bar shows. Multi-million rows → use the CLI directly.
Need help — email the address on your purchase receipt.

7. License

Single-user. See LICENSE.txt.

4.9 KiB Raw Blame History Unescape Escape