Unicode Normalize
v0.1.0Added Sep 26, 2026Sample dataUnicode cleanup for text pipelines: NFC/NFKC normalization, whitespace folding and removal of invisible characters.
cargo install kura-rs
kura add unicode-normalizeCopies the source file into your project. You own the code from then on.
If another tool already installs a kura command, run cargo install kura-rs --bin kura-rs and use kura-rs add instead.
use crate::parts::unicode_normalize::{normalize, Options};
let clean = normalize(&raw, Options::nfkc().fold_whitespace().strip_invisible());Verification
Sample dataChecked against unicodedata 3.12 (Python) using property-based testing: the same inputs go to both implementations and the outputs are compared.
These numbers are placeholders. They have not been measured yet and must not be read as real results. This part is marked "sample": true in its registry file.
Benchmarks
Sample datakura-rs (Rust) against unicodedata across input sizes. Faster from about 143 inputs, up to 2.2× faster.
These numbers are placeholders. They have not been measured yet and must not be read as real results. This part is marked "sample": true in its registry file.
Show data table
| Input size | kura-rs (Rust) | unicodedata | Speedup |
|---|---|---|---|
| 10 | 0.01 ms | 0 ms | 3.0× slower |
| 100 | 0.02 ms | 0.02 ms | 1.1× slower |
| 1,000 | 0.09 ms | 0.16 ms | 1.8× |
| 10,000 | 0.8 ms | 1.7 ms | 2.1× |
| 100,000 | 7.9 ms | 17 ms | 2.2× |
| 1,000,000 | 78 ms | 172 ms | 2.2× |
Files
- parts/unicode_normalize/mod.rs