uconv: The ICU-Powered Unicode Swiss Army Knife That Makes iconv Look Provincial

2026-09-02

Everyone reaches for iconv when a file arrives in the wrong encoding. Fewer people know that ICU — the Unicode library shipped with essentially every modern OS — comes with its own CLI, uconv, that does everything iconv does and transliteration between scripts, Unicode normalization forms, case folding, and arbitrary composable transform chains. It's usually one apt install icu-devtools away (brew install icu4c on macOS puts it in the keg).

The killer feature is the -x flag, which applies an ICU transform ID — or a whole pipeline of them separated by semicolons. Start with encoding conversion:

# The iconv equivalent — nothing special yet
uconv -f UTF-16BE -t UTF-8 windows-export.txt > clean.txt

Now the part iconv can't do. Transliterate between scripts using the CLDR transliteration tables:

$ echo "Ελληνικά" | uconv -x "Greek-Latin"
Ellēniká

$ echo "Москва" | uconv -x "Cyrillic-Latin"
Moskva

$ echo "北京欢迎你" | uconv -x "Han-Latin"
běi jīng huān yíng nǐ

$ echo "こんにちは" | uconv -x "Hiragana-Latin"
konnichiha

Yes, that's automatic Pinyin generation from Han characters, from a tool that's been on your box since you installed anything that links libicu. Need ASCII-safe filenames from user-supplied names in any script?

$ echo "café — Straße — Ω" | uconv -x "Any-Latin; Latin-ASCII"
cafe -- Strasse -- O

Two transforms, chained. The first normalizes any script to Latin; the second flattens Latin-with-diacritics to plain ASCII. If you want more control, do it manually with normalization plus a Unicode character-class filter:

# Decompose, drop combining marks, recompose
$ echo "café naïve résumé" | uconv -x "NFD; [:Nonspacing Mark:] Remove; NFC"
cafe naive resume

The [:Nonspacing Mark:] is a real Unicode property filter — you can use any of them ([:Punctuation:], [:Digit:], [:Emoji:]). Combine with the built-in case transforms:

$ echo "HELLO — WORLD" | uconv -x "Lower; [:Punctuation:] Remove"
hello  world

# Normalize URLs / cache keys
$ echo "  Café Society!  " | uconv -x "NFKC; Lower; Latin-ASCII" | tr -s ' '
 cafe society!

The normalization forms (NFC, NFD, NFKC, NFKD) matter more than most developers realize — a file named café.txt from macOS Finder (NFD, e + combining acute) will not string-compare equal to café.txt from a Linux terminal (NFC, single precomposed é). uconv -x NFC makes this go away.

You can write transforms inline with the CLDR rule syntax too — useful for one-off munging:

$ echo "hello world" | uconv -x "::Lower; [aeiou] > '*';"
h*ll* w*rld

List every transform your ICU version knows about:

$ uconv -L | head
Accents-Any
Amharic-Latin/BGN
Any-Accents
Any-Publishing
Arabic-Latin
Arabic-Latin/BGN
...

There are hundreds. Fullwidth-Halfwidth flattens those East Asian full-width ASCII characters that show up in copy-pasted spreadsheets. Any-Publishing converts -- to em-dashes and "..." to smart quotes. Any-Hex/Unicode gives you \u00e9-style escapes on demand.

Where iconv tops out at "byte sequence A becomes byte sequence B," uconv gives you the full Unicode processing pipeline as CLI text filter. If you've ever hand-rolled a Python script to strip diacritics or normalize weird input, delete it.

Key Takeaway: uconv is the CLI that ships with ICU and turns Unicode normalization, script transliteration, and encoding conversion into composable text-filter pipelines — everything iconv can't do, already installed on your machine.

All newsletters