Michael
54f92ae47e
Builds 02_text_cleaner.py from stub to working: character-level hygiene for CSV/Excel inputs covering trim, whitespace collapse, smart-character folding, Unicode NFC/NFKC, BOM strip, zero-width strip, control-char strip, line-ending normalization, and per-column case conversion. Three presets (minimal/excel-hygiene/paranoid) keep the buyer surface small. - src/core/text_clean.py: pure helpers + CleanOptions/CleanResult + clean_dataframe with dtype-safe column selection - src/cli_text_clean.py: Typer CLI mirroring the dedup CLI shape (dry-run by default, --apply writes cleaned + changes audit, JSON config save/load) - src/gui/pages/2_Text_Cleaner.py: real Streamlit page with preset picker, advanced toggles, preview, before/after metrics, and three download buttons - tests/test_text_clean.py + test_cli_text_clean.py: 92 new tests covering edge cases E1-E50 from the spec - samples/messy_text.csv: demo dataset surfacing UC1, UC3, UC6, UC10 in 10 rows - test-cases/uc16-uc26 + ec05-ec09: per-use-case and per-edge-case fixtures Docs: TECHNICAL.md §10.2 (full Tier 1/2/3 spec), DECISIONS.md v1.7 entry locking the spec, CLI-REFERENCE.md gains the text cleaner section, README.md gains a top-level Text Cleaner block, USER-GUIDE.md status row 02 promoted Skeleton -> Working. 200/200 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
14 lines
599 B
CSV
14 lines
599 B
CSV
customer_name,email,vendor,memo
|
||
Alice Johnson,alice@example.com,ACME Corp ,Welcome aboard
|
||
Bob Smith,bob@example.com,ACME Corp,Returning customer
|
||
Charlie Brown,charlie@example.com,Globex,Net 30
|
||
Diana Prince,diana@example.com,Globex,VIP
|
||
Edward Norton,ed@example.com,“Best Pet Supplies”,Order#42 - rush
|
||
Frank Castle,frank@example.com,Stark—Industries,"Line 1
|
||
Line 2
|
||
Line 3"
|
||
grace HOPPER ,grace@example.com,Globex,Loves long memos…
|
||
Henry Ford,henry@example.com,Ford Motor,Industrial
|
||
Iris West,iris@example.com,S.T.A.R. Labs,Notewith-bell
|
||
Jane Doe,jane@example.com,Acme,Standard
|