[Dxspider-support] What happens when UTF-8 is not used in DXSpider and your OS?

[Kin]

Hi,

If your system, application or file is not encoded in UTF-8, you may
experience serious problems with text display, data handling, and
compatibility, especially when using characters beyond the standard alphabet
(accents, currency symbols, emojis, Cyrillic, Chinese, etc.).

*** Typical Issues When UTF-8 Is Not Used
1. Garbled or Corrupted Characters (mojibake)
   Example:
   “Renée Noël” → “Ren�e No�l” or “Renée Noël”

   This happens when text is read using the wrong encoding — for instance,
interpreting a file written in UTF-8 as ISO-8859-1 or Windows-1252.

2. Errors When Reading or Writing Files
   Applications (e.g. Perl) may throw errors like UnicodeEncodeError or
UnicodeDecodeError.
   Files may open correctly in one editor but appear broken in another.
   CSV, XML, or JSON files might fail to import/export properly if special
characters are used (e.g., “£”, “é”, “ä”).

3. Database Encoding Conflicts
   Files like users.v3j or user_json.
   If your database (e.g. MySQL) or specific columns do not use utf8mb4,
you may encounter:
   Missing or broken characters.
   Truncated strings or silent data corruption.
   Insertion errors with multilingual input.

4. Cross-System Incompatibility
   Web browsers might display "�" or strange symbols if the encoding is
misdeclared.
   APIs might crash or send malformed JSON when receiving multilingual
text.

5. Terminal and Log File Issues
   Terminals not set to UTF-8 may display accented names or non-Latin
characters incorrectly.

*** Summary
Not using UTF-8 in 2025 is risky and unnecessary. It may appear to work in
small projects or single-language systems, but as soon as your data:
- is displayed on the web,
- shared between systems, or
- includes multilingual or special characters...

…it will break.

Using UTF-8 throughout your stack (files, web, databases, logs, APIs) is
the safest and most future-proof approach.

DXSpider is affected by these encodings. Updating your OS is not
complicated, and you will avoid problems in your system and in the data you
send to the rest of the nodes in the Network.

Kin EA3CV

-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 6542 bytes
Desc: not available
URL: <https://mailman.tobit.co.uk/pipermail/dxspider-support/attachments/20250608/002e465e/attachment.bin>