unifuzz(1) [debian man page]

unifuzz(1)						      General Commands Manual							unifuzz(1)

NAME

       unifuzz - Emit strings designed to test Unicode handling

SYNOPSIS

       unifuzz ([option flags])

DESCRIPTION

       unifuzz	emits strings designed to test the ability of programs intended to accept Unicode input to handle unexpected input. These include:
       characters from all Unicode ranges, Private Use characters, surrogates, undefined characters, non-characters,  control  characters,  exotic
       space characters, sequences violating normalization rules, unexpected sequences (e.g. a base character from one range followed by a combin-
       ing character from another range), and long sequences of combining characters.  It can also generate very long  lines,  strings	containing
       embedded nulls, and ill-formed UTF-8.

COMMAND LINE FLAGS

       -b     Restrict the output to the Basic Multilingual Plane (Plane 0).

       -g     Do not emit specific characters.

       -h     Print usage information.

       -l     Emit very long lines.

       -n     Emit string with embedded nulls.

       -q     Be quiet. Omit commentary.

       -r <number>
	      Set the number of random characters to emit.

       -S     Scan ranges - emit a character from each range.

       -s <seed>
	      Set the seed for the random number generator.

       -u     Emit ill-formed UTF-8.

       -v     Print version information.

       The  sequence  of  random  characters is determined by a pseudorandom number generator, so the same sequence can be obtained by setting the
       seed to the same value. If not set on the command line, a seed is chosen based on the time of execution. The seed used is included  in  the
       output  in  a  line  of the form "Seed = NNNNNN" immediately preceding the random character sequence. Note that in order to obtain the same
       sequence it is necessary to keep the same setting for restriction of output to the BMP.

REFERENCES

       Unicode Standard, version 5.0

AUTHOR

       Bill Poser
       billposer@alum.mit.edu

LICENSE

       GNU General Public License

								    April, 2008 							unifuzz(1)

UTF(6)								   Games Manual 							    UTF(6)

NAME

       UTF, Unicode, ASCII, rune - character set and format

DESCRIPTION

       The  Plan 9 character set and representation are based on the Unicode Standard and on the ISO multibyte UTF-8 encoding (Universal Character
       Set Transformation Format, 8 bits wide).  The Unicode Standard represents its characters in 16 bits; UTF-8 represents  such  values  in	an
       8-bit byte stream.  Throughout this manual, UTF-8 is shortened to UTF.

       In Plan 9, a rune is a 16-bit quantity representing a Unicode character.  Internally, programs may store characters as runes.  However, any
       external manifestation of textual information, in files or at the interface  between  programs,	uses  a  machine-independent,  byte-stream
       encoding called UTF.

       UTF  is	designed so the 7-bit ASCII set (values hexadecimal 00 to 7F), appear only as themselves in the encoding.  Runes with values above
       7F appear as sequences of two or more bytes with values only from 80 to FF.

       The UTF encoding of the Unicode Standard is backward compatible with ASCII: programs presented only with ASCII work on Plan 9 even  if  not
       written	to deal with UTF, as do programs that deal with uninterpreted byte streams.  However, programs that perform semantic processing on
       ASCII graphic characters must convert from UTF to runes in order to work properly with non-ASCII input.	See rune(2).

       Letting numbers be binary, a rune x is converted to a multibyte UTF sequence as follows:

       01.   x in [00000000.0bbbbbbb] -> 0bbbbbbb
       10.   x in [00000bbb.bbbbbbbb] -> 110bbbbb, 10bbbbbb
       11.   x in [bbbbbbbb.bbbbbbbb] -> 1110bbbb, 10bbbbbb, 10bbbbbb

       Conversion 01 provides a one-byte sequence that spans the ASCII character set in a compatible way.  Conversions 10 and 11 represent higher-
       valued  characters  as  sequences of two or three bytes with the high bit set.  Plan 9 does not support the 4, 5, and 6 byte sequences pro-
       posed by X-Open.  When there are multiple ways to encode a value, for example rune 0, the shortest encoding is used.

       In the inverse mapping, any sequence except those described above is incorrect and is converted to rune hexadecimal 0080.

FILES

       /lib/unicode
	      table of characters and descriptions, suitable for look(1).

SEE ALSO

       ascii(1), tcs(1), rune(2), keyboard(6), The Unicode Standard.

																	    UTF(6)