In some installations, telnet is not 8-bit clean by default. In order to be able to send Unicode keystrokes to the remote host, you need to set telnet into "outbinary" mode. There are two ways to do this:
and
$ telnet -L <host>
$ telnet
telnet> set outbinary
telnet> open <host>
You should do telnet from ncst-term.
The communications program C-Kermit http://www.columbia.edu/kermit/ckermit.html , (an interactive tool for connection setup, telnet, file transfer, with support for TCP/IP and serial lines), in versions 7.0 or newer, understands the file and transfer encodings UTF-8 and UCS-2, and understands the terminal encoding UTF-8, and converts between these encodings and many others. Documentation of these features can be found in http://www.columbia.edu/kermit/ckermit2.html#x6.6 .
Netscape 6.01 or later can display HTML documents in UTF-8 encoding. All a document needs is the following line between the <head> and </head> tags:
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
Netscape 6.01 or newer can also display HTML and text files in UCS-2 encoding with byte-order mark.
To setup Netscape so that it displays Hindi characters, follow the steps :
1. Goto, Edit -> Preferences
2. Select category, Appearance -> Fonts
3. Select Language encoding "Unicode"
4. Set Variable-width and Fixed-width fonts to "raghu"
5. Check button "Always use my font settings, overriding web page font"
Also do the following to set character coding scheme to UTF-8
1. Goto, View -> Character Coding
2. Select "Unicode (UTF-8)" from the list
Download Netscape-6 from http://www.netscape.com/computing/download/ or IndiX download page.
Mozilla milestone M16 has much better internationalization than Netscape 4. It can display HTML documents in UTF-8 encoding with support for more languages.
lynx-2.8 has an options screen (key 'O') which permits to set the display character set. When running in an ncst-term or Linux console in UTF-8 mode, set this to "UNICODE UTF-8". Note that for this setting to take effect in the current browser session, you have to confirm on the "Accept Changes" field, and for this setting to take effect in future browser sessions, you have to enable the "Save options to disk" field and then confirm it on the "Accept Changes" field.
Now, again, all a document needs is the following line between the <head> and </head> tags:
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
When you are viewing text files in UTF-8 encoding, you also need to pass the command-line option "-assume_local_charset=UTF-8" (affects only file:/... URLs) or "-assume_charset=UTF-8" (affects all URLs). In lynx-2.8.2 you can alternatively, in the options screen (key 'O'), change the assumed document character set to "utf-8".
There is also an option in the options screen, to set the "preferred document character set". But it has no effect, at least with file:/... URLs and with http://... URLs served by apache-1.3.0.
There is a spacing and line-breaking problem, however. (Look at the Russian section of x-utf8.html, or at utf-8-demo.txt.)
Also, in lynx-2.8.2, configured with --enable-prettysrc, the nice colour scheme does not work correctly any more when the display character set has been set to "UNICODE UTF-8". This is fixed by a simple patch lynx282.diff.
The Lynx developers say: "For any serious use of UTF-8 screen output with lynx, compiling with slang lib and -DSLANG_MBCS_HACK is still recommended."
Latest stable release: ftp://ftp.gnu.org/pub/gnu/lynx/lynx-2.8.2.tar.gz
General home page: http://lynx.browser.org/
Newer development shapshots:
http://lynx.isc.org/current/ ,
ftp://lynx.isc.org/current/
To setup Konqueror so that it displays Hindi characters, follow the steps :
1. Goto, Settings -> Configure Konqueror
2. Select "Konqueror Bowser" from the left pan
3. Goto "Appearance" tab on the right pan
4. Select charset "iso106460-1"
5. Set all fonts to "raghu" for this encoding and also set Default encoding to "utf8"
yudit by Gáspár Sinai http://czyborra.com/yudit/ is a first-class unicode text editor for the X Window System. It supports simultaneous processing of many languages, input methods, conversions for local character standards. It has facilities for entering text in all languages with only an English keyboard, using keyboard configuration maps.
It can be compiled in three versions: Xlib GUI, KDE GUI,or Motif GUI.
Customization is very easy. Typically you will first customize your font. From the font menu I chose "Unicode".
Next, you will customize your input method. The input methods "Straight", "Unicode" and "SGML" are most remarkable. For details about the other built-in input methods, look in /usr/local/share/yudit/data/.
To make a change the default for the next session, edit your $HOME/.yuditrc file.
The general editor functionality is limited to editing, cut&paste and search&replace. No undo.
yudit can display text using a TrueType font. But it doesn't seem to support combining characters.
Open vim under ncst-term. vim (as of version 6.0) has good support for UTF-8: when started in an UTF-8 locale, it assumes UTF-8 encoding for the console and the text files being edited. It supports double-wide (CJK) characters as well and combining characters and therefore fits perfectly into UTF-8 enabled ncst-term.
Installation: Download vim 6.0 from ftp://ftp.vim.org or IndiX download page. After unpacking it, configure with --enable-multibyte, then do "make" and "make install".
gedit is an editor developed using GtkText widget. gedit-0.9.0 does not support FontSet. So you can't edit both English and Hindi text simultaneously. But if you choose proper font then you will be able to use any one language at a time.
With XFree86-4.0.1, xedit is able to edit UTF-8 files if you set the locale accordingly (see above), and add the line "Xedit*international: true" to your $HOME/.Xdefaults file.
As of version 6.1.2, aXe supports only 8-bit locales. If you add the line "Axe*international: true" to your $HOME/.Xdefaults file, it will simply dump core.
mined98 is a small text editor by Michiel Huisjes, Achim Müller and Thomas Wolff. http://www.inf.fu-berlin.de/~wolff/mined.html It lets you edit UTF-8 or 8-bit encoded files, in an UTF-8 or 8-bit ncst-term. It also has powerful capabilities for entering Unicode characters.
mined lets you edit both 8-bit encoded and UTF-8 encoded files. By default it uses an
autodetection heuristic. If you don't want to rely on heuristics, pass the command-line option
-u when editing an UTF-8 file, or +u when editing
an 8-bit encoded file. You can change the interpretation at any time from
within the editor: It displays the encoding ("L:h" for 8-bit, "U:h" for
UTF-8) in the menu line. Click on the first of these characters to change
it.
mined knows about double-width and combining characters and displays them correctly.
mined also has very nice pull-down menus. Alas, the "Home", "End", "Delete" keys do not work.
MIME: RFC 2279 defines UTF-8 as a MIME charset, which can be transported under the 8bit, quoted-printable and base64 encodings. The older MIME UTF-7 proposal (RFC 2152) is considered to be deprecated and should not be used any further.
Mail clients released after January 1, 1999, should be capable of sending and displaying UTF-8 encoded mails, otherwise they are considered deficient. But these mails have to carry the MIME labels
Simply piping an UTF-8 file into "mail" without caring about the MIME labels will not work.
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Mail client implementors should take a look at http://www.imc.org/imc-intl/ and http://www.imc.org/mail-i18n.html .
Now about the individual mail clients (or "mail user agents"):
The situation for an unpatched pine version 4.10 is as follows.
Pine does not do character set conversions. But it allows you to view UTF-8 mails in an UTF-8 text window (Linux console or ncst-term).
Normally, Pine will warn about different character sets each time you view an UTF-8 encoded mail. To get rid of this warning, from Main Menu choose S (setup), then C (config), then change the value of "character-set" to UTF-8. This option will not do anything, except to reduce the warnings, as Pine has no built-in knowledge of UTF-8. Also set "pass-control-characters-as-is" option in "Viewer Preferences" of config menu.
To compose mails in Devanagari script, you must use vim editor inside pine. To do this, change the value of "editor" to vim in config menu and then set "enable-alternate-editor-implicitly" option in "Composer Preferences" of config menu.
To enable 8-bit transfer with UTF-8 encoding at other places in pine, you must set
header values properly. Change the value of "customized-hdrs" and set the following comma separated
values :
Content-Type: text/plain; charset=UTF-8, Content-Transfer-Encoding: 8bit
However, alignment remains broken in many places; replying to a mail does not cause the character set to be converted as appropriate; and the editor, pico, cannot deal with multibyte characters.
kmail (as of KDE 1.0) does not support UTF-8 mails at all.
Netscape's Mail can send and display mails in UTF-8 encoding, but it needs a little bit of manual user intervention.
To send an UTF-8 encoded mail: After opening the "Mail" window,
but before starting to compose the message, select from the menu
"View -> Character Coding -> Unicode (UTF-8)".
Then compose the message and send it.
When you receive an UTF-8 encoded mail, Netscape unfortunately does not display it in UTF-8 right away, and does not even give a visual clue that the mail was encoded in UTF-8. You have to manually select from the menu "View -> Character Coding -> Unicode (UTF-8)".
For displaying UTF-8 mails, Netscape uses different fonts. You can adjust your font settings in the "Edit -> Preferences -> Fonts" dialog; choose the "Unicode" font category.
mutt-1.0, as available from http://www.mutt.org/, contains only rudimentary UTF-8 support. For full UTF-8 support, there are patches by Edmund Grimley Evans at http://www.rano.demon.co.uk/mutt.html .
exmh 2.1.2 with Tk 8.4a1 can recognize and correctly display UTF-8 mails
if you add the following lines to your $HOME/.Xdefaults file.
!
! Exmh
!
exmh.mimeUCharsets: utf-8
exmh.mime_utf-8_registry: iso10646
exmh.mime_utf-8_encoding: 1
exmh.mime_utf-8_plain_families: fixed
exmh.mime_utf-8_fixed_families: fixed
exmh.mime_utf-8_proportional_families: fixed
exmh.mime_utf-8_title_families: fixed
groff 1.16, the GNU implementation
of the traditional Unix text processing system troff/nroff, can output
UTF-8 formatted text. Simply use `groff -Tutf8' instead of
`groff -Tlatin1' or `groff -Tascii'.
The teTeX 0.9 (and newer) distribution contains an Unicode adaptation of TeX, called Omega ( http://www.gutenberg.eu.org/omega/ , ftp://ftp.ens.fr/pub/tex/yannis/omega ). Together with the unicode.tex file contained in utf8-tex-0.1.tar.gz it enables you to use UTF-8 encoded sources as input for TeX. A thousand of Unicode characters are currently supported.
All that changes is that you run `omega' (instead of `tex') or `lambda' (instead of `latex'), and insert the following lines at the head of your source input.
\ocp\TexUTF=inutf8
\InputTranslation currentfile \TexUTF
\input unicode
PostgreSQL 6.4 or newer can be built with the configuration
option --with-mb=UNICODE.
With http://www.flash.net/~marknu/less/less-358.tar.gz you can browse UTF-8 encoded text files in an UTF-8 ncst-term or console. Make sure that the environment variable LESSCHARSET is not set (or is set to utf-8). If you also have a LESSKEY environment variable set, also make sure that the file it points to does not define LESSCHARSET. If necessary, regenerate this file using the `lesskey' command, or unset the LESSKEY environment variable.
lv-4.21 by Tomio Narita http://www.mt.cs.keio.ac.jp/person/narita/lv/ is a file viewer with builtin character set converters. To view UTF-8 files in an UTF-8 console, use "lv -Au8". But it can also be used to view files in other CJK encodings in an UTF-8 console.
There is a small glitch: lv turns off cursor and doesn't turn it on again.
Get the GNU textutils-2.0 and apply the patch textutils-2.0.diff , then configure, add "#define HAVE_MBRTOWC 1", "#define HAVE_FGETWC 1", "#define HAVE_FPUTWC 1" to config.h. In src/Makefile, modify CFLAGS and LDFLAGS so that they include the directories where libutf8 is installed. Then rebuild.
Get the util-linux-2.9y package, configure it, then define ENABLE_WIDECHAR in defines.h, change the "#if 0" to "#if 1" in lib/widechar.h. In text-utils/Makefile, modify CFLAGS and LDFLAGS so that they include the directories where libutf8 is installed. Then rebuild.
figlet 2.2 has an option for UTF-8 input: "figlet -C utf8"
The Li18nux list of commands and utilities that ought to be made interoperable with UTF-8 is as follows. Useful information needs to get added here; I just didn't get around it yet :-)
As of glibc-2.2, regular expressions will only work for 8-bit characters. In an UTF-8 locale, regular expressions that contain non-ASCII characters or that expect to match a single multibyte character with "." will not work. This affects all commands and utilities listed below.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
As of at-3.1.8: The two uses of isalnum in at.c are invalid and should be replaced with a use of quotearg.c or an exclude list of the (fixed) list of shell metacharacters. The two uses of %8s in at.c and atd.c are invalid and should become arbitrary length.
As of sh-utils-2.0i: OK.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
As of fileutils-4.0u: OK.
As of fileutils-4.0u: OK.
As of fileutils-4.0u: OK.
As of sh-utils-2.0i: OK.
As of textutils-2.0e: OK.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
As of fileutils-4.0u: OK.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
As of sh-utils-2.0i: OK.
As of fileutils-4.0u: The conv=lcase, conv=ucase options don't work correctly.
No info available yet.
As of fileutils-4.0u: OK.
As of diffutils-2.7 (1994): diff is not locale aware; the --side-by-side mode therefore doesn't compute column width correctly, not even in ISO-8859-1 locales.
No info available yet.
As of sh-utils-2.0i: OK.
No info available yet.
As of fileutils-4.0u: OK.
As of sh-utils-2.0i: OK.
As of sh-utils-2.0i: OK.
No info available yet.
As of sh-utils-2.0i: The operators "match", "substr", "index", "length" don't work correctly.
As of sh-utils-2.0i: OK.
No info available yet.
No info available yet.
No info available yet.
As of findutils-4.1.5: The "-ok" option is not internationalized; a patch has been submitted to the maintainer. The "-iregex" does not work correctly; this needs a fix in function find/parser.c:insert_regex.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
gzip-1.3 is UTF-8 capable, but it uses only English messages in ASCII charset. Proper internationalization would require: Use gettext. Call setlocale. In function check_ofname (file gzip.c), use the function rpmatch from GNU text/sh/fileutils instead of asking for "y" or "n". The use of strlen in gzip.c:852 is wrong, needs to use the function mbswidth.
No info available yet.
No info available yet.
As of sh-utils-2.0i: OK.
As of sh-utils-2.0i: OK.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No complete info available yet.
No info available yet.
No info available yet.
As of fileutils-4.0u: OK.
No info available yet.
No info available yet.
As of sh-utils-2.0i: OK.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
As of fileutils-4.0y: OK.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
As of fileutils-4.0u: OK.
As of fileutils-4.0u: OK.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
As of fileutils-4.0u: OK.
No info available yet.
No info available yet.
As of sh-utils-2.0i: OK.
No info available yet.
As of sh-utils-2.0i: OK.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
As of sh-utils-2.0i: OK.
No info available yet.
As of sh-utils-2.0i: OK.
No info available yet.
No info available yet.
As of sh-utils-2.0i: OK.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
As of fileutils-4.0u: OK.
As of fileutils-4.0u: OK.
No info available yet.
No info available yet.
No info available yet.
As of sh-utils-2.0i: OK.
No info available yet.
No info available yet.
No info available yet.
As of sh-utils-2.0i: The string "<undef>" should not be translated; this needs a fix in function stty.c:visible.
No info available yet.
As of textutils-2.0e: OK.
No info available yet.
No info available yet.
No info available yet.
As of tar-1.13.17: OK, if user and group names are always ASCII.
No info available yet.
As of sh-utils-2.0i: OK.
No info available yet.
As of sh-utils-2.0i: OK.
No info available yet.
As of fileutils-4.0u: OK.
No info available yet.
No info available yet.
As of sh-utils-2.0i: OK.
No info available yet.
As of sh-utils-2.0i: OK.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
As of sh-utils-2.0i: OK.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
No info available yet.
As of textutils-2.0e: wc cannot count characters; a patch has been submitted to the maintainer.
As of sh-utils-2.0i: OK.
No info available yet.
No info available yet.
As of findutils-4.1.5: The program uses strstr; a patch has been submitted to the maintainer.
No info available yet.
No info available yet.
Owen Taylor is currently developing a library for rendering multilingual text, called pango. http://www.labs.redhat.com/~otaylor/pango/ , http://www.pango.org/ .
