NAME
binary - Insert and extract fields from binary strings
SYNOPSIS
bbiinnaarryy ffoorrmmaatt formatString ?arg arg ...? bbiinnaarryy ssccaann string formatString ?varName varName ...?DESCRIPTION
This command provides facilities for manipulating binary data. The
first form, bbiinnaarryy ffoorrmmaatt, creates a binary string from normal Tcl val-
ues. For example, given the values 16 and 22, on a 32 bit architec-
ture, it might produce an 8-byte binary string consisting of two 4-byte
integers, one for each of the numbers. The second form of the command,bbiinnaarryy ssccaann, does the opposite: it extracts data from a binary string
and returns it as ordinary Tcl string values. BBIINNAARRYY FFOORRMMAATTThe bbiinnaarryy ffoorrmmaatt command generates a binary string whose layout is
specified by the formatString and whose contents come from the addi-
tional arguments. The resulting binary value is returned.
The formatString consists of a sequence of zero or more field speci-
fiers separated by zero or more spaces. Each field specifier is a sin-
gle type character followed by an optional numeric count. Most field specifiers consume one argument to obtain the value to be formatted. The type character specifies how the value is to be formatted. The count typically indicates how many items of the specified type aretaken from the value. If present, the count is a non-negative decimal
integer or **, which normally indicates that all of the items in the value are to be used. If the number of arguments does not match the number of fields in the format string that consume arguments, then an error is generated.Here is a small example to clarify the relation between the field spec-
ifiers and the arguments: bbiinnaarryy ffoorrmmaatt dd33dd {{11..00 22..00 33..00 44..00}} 00..11 The first argument is a list of four numbers, but because of the count of 3 for the associated field specifier, only the first three will beused. The second argument is associated with the second field speci-
fier. The resulting binary string contains the four numbers 1.0, 2.0,
3.0 and 0.1.Each type-count pair moves an imaginary cursor through the binary data,
storing bytes at the current position and advancing the cursor to just after the last byte stored. The cursor is initially at position 0 at the beginning of the data. The type may be any one of the following characters: aa Stores a character string of length count in the output string. Every character is taken as modulo 256 (i.e. the low byte of every character is used, and the high byte discarded) so when storing character strings not wholly expressible using the characters\u0000-\u00ff, the eennccooddiinngg ccoonnvveerrttttoo command should be used first
if this truncation is not desired (i.e. if the characters are notpart of the ISO 8859-1 character set.) If arg has fewer than
count bytes, then additional zero bytes are used to pad out the field. If arg is longer than the specified length, the extra characters will be ignored. If count is **, then all of the bytes in arg will be formatted. If count is omitted, then one character will be formatted. For example, bbiinnaarryy ffoorrmmaatt aa77aa**aa aallpphhaa bbrraavvoo cchhaarrlliiee will return a string equivalent to aallpphhaa\\000000\\000000bbrraavvoocc. AA This form is the same as aa except that spaces are used for padding instead of nulls. For example, bbiinnaarryy ffoorrmmaatt AA66AA**AA aallpphhaa bbrraavvoo cchhaarrlliiee will return aallpphhaa bbrraavvoocc.bb Stores a string of count binary digits in low-to-high order within
each byte in the output string. Arg must contain a sequence of 11 and 00 characters. The resulting bytes are emitted in first tolast order with the bits being formatted in low-to-high order
within each byte. If arg has fewer than count digits, then zeros will be used for the remaining bits. If arg has more than the specified number of digits, the extra digits will be ignored. If count is **, then all of the digits in arg will be formatted. If count is omitted, then one digit will be formatted. If the number of bits formatted does not end at a byte boundary, the remaining bits of the last byte will be zeros. For example, bbiinnaarryy ffoorrmmaatt bb55bb** 1111110000 111111000000001111001100 will return a string equivalent to \\xx0077\\xx8877\\xx0055. BB This form is the same as bb except that the bits are stored inhigh-to-low order within each byte. For example,
bbiinnaarryy ffoorrmmaatt BB55BB** 1111110000 111111000000001111001100 will return a string equivalent to \\xxee00\\xxee11\\xxaa00.hh Stores a string of count hexadecimal digits in low-to-high within
each byte in the output string. Arg must contain a sequence of characters in the set ``0123456789abcdefABCDEF''. The resulting bytes are emitted in first to last order with the hex digits beingformatted in low-to-high order within each byte. If arg has fewer
than count digits, then zeros will be used for the remaining dig-
its. If arg has more than the specified number of digits, theextra digits will be ignored. If count is **, then all of the dig-
its in arg will be formatted. If count is omitted, then one digit will be formatted. If the number of digits formatted does not end at a byte boundary, the remaining bits of the last byte will be zeros. For example, bbiinnaarryy ffoorrmmaatt hh33hh** AABB ddeeff will return a string equivalent to \\xxbbaa\\xx0000\\xxeedd\\xx00ff. HH This form is the same as hh except that the digits are stored inhigh-to-low order within each byte. For example,
bbiinnaarryy ffoorrmmaatt HH33HH** aabb DDEEFF will return a string equivalent to \\xxaabb\\xx0000\\xxddee\\xxff00.cc Stores one or more 8-bit integer values in the output string. If
no count is specified, then arg must consist of an integer value; otherwise arg must consist of a list containing at least countinteger elements. The low-order 8 bits of each integer are stored
as a one-byte value at the cursor position. If count is **, then
all of the integers in the list are formatted. If the number ofelements in the list is fewer than count, then an error is gener-
ated. If the number of elements in the list is greater than count, then the extra elements are ignored. For example,bbiinnaarryy ffoorrmmaatt cc33cccc** {{33 -33 112288 11}} 226600 {{22 55}}
will return a string equivalent to \\xx0033\\xxffdd\\xx8800\\xx0044\\xx0022\\xx0055, whereas bbiinnaarryy ffoorrmmaatt cc {{22 55}} will generate an error. ss This form is the same as cc except that it stores one or more16-bit integers in little-endian byte order in the output string.
The low-order 16-bits of each integer are stored as a two-byte
value at the cursor position with the least significant byte stored first. For example,bbiinnaarryy ffoorrmmaatt ss33 {{33 -33 225588 11}}
will return a string equivalent to \\xx0033\\xx0000\\xxffdd\\xxffff\\xx0022\\xx0011. SS This form is the same as ss except that it stores one or more16-bit integers in big-endian byte order in the output string.
For example,bbiinnaarryy ffoorrmmaatt SS33 {{33 -33 225588 11}}
will return a string equivalent to \\xx0000\\xx0033\\xxffff\\xxffdd\\xx0011\\xx0022. ii This form is the same as cc except that it stores one or more32-bit integers in little-endian byte order in the output string.
The low-order 32-bits of each integer are stored as a four-byte
value at the cursor position with the least significant byte stored first. For example,bbiinnaarryy ffoorrmmaatt ii33 {{33 -33 6655553366 11}}
will return a string equivalent to \\xx0033\\xx0000\\xx0000\\xx0000\\xxffdd\\xxffff\\xxffff\\xxffff\\xx0000\\xx0000\\xx0011\\xx0000 II This form is the same as ii except that it stores one or more oneor more 32-bit integers in big-endian byte order in the output
string. For example,bbiinnaarryy ffoorrmmaatt II33 {{33 -33 6655553366 11}}
will return a string equivalent to \\xx0000\\xx0000\\xx0000\\xx0033\\xxffff\\xxffff\\xxffff\\xxffdd\\xx0000\\xx0011\\xx0000\\xx0000 ww This form is the same as cc except that it stores one or more |64-bit integers in little-endian byte order in the output string. |
The low-order 64-bits of each integer are stored as an eight-byte |
value at the cursor position with the least significant byte | stored first. For example, | bbiinnaarryy ffoorrmmaatt ww 77881100117799001166332277771188221166 | will return the string HHeellllooTTccll | WW || This form is the same as ww except that it stores one or more one |or more 64-bit integers in big-endian byte order in the output |
string. For example, | bbiinnaarryy ffoorrmmaatt WWcc 44778855446699662266996600334411334455 111100 | will return the string BBiiggEEnnddiiaann ff This form is the same as cc except that it stores one or more oneor more single-precision floating in the machine's native repre-
sentation in the output string. This representation is not porta-
ble across architectures, so it should not be used to communicate floating point numbers across the network. The size of a floating point number may vary across architectures, so the number of bytes that are generated may vary. If the value overflows the machine's native representation, then the value of FLTMAX as defined by thesystem will be used instead. Because Tcl uses double-precision
floating-point numbers internally, there may be some loss of pre-
cision in the conversion to single-precision. For example, on a
Windows system running on an Intel Pentium processor, bbiinnaarryy ffoorrmmaatt ff22 {{11..66 33..44}} will return a string equivalent to \\xxccdd\\xxcccc\\xxcccc\\xx33ff\\xx99aa\\xx9999\\xx5599\\xx4400. dd This form is the same as ff except that it stores one or more oneor more double-precision floating in the machine's native repre-
sentation in the output string. For example, on a Windows system running on an Intel Pentium processor, bbiinnaarryy ffoorrmmaatt dd11 {{11..66}} will return a string equivalent to \\xx99aa\\xx9999\\xx9999\\xx9999\\xx9999\\xx9999\\xxff99\\xx33ff. xx Stores count null bytes in the output string. If count is not specified, stores one null byte. If count is **, generates an error. This type does not consume an argument. For example, bbiinnaarryy ffoorrmmaatt aa33xxaa33xx22aa33 aabbcc ddeeff gghhii will return a string equivalent to aabbcc\\000000ddeeff\\000000\\000000gghhii. XX Moves the cursor back count bytes in the output string. If countis ** or is larger than the current cursor position, then the cur-
sor is positioned at location 0 so that the next byte stored will be the first byte in the result string. If count is omitted then the cursor is moved back one byte. This type does not consume an argument. For example, bbiinnaarryy ffoorrmmaatt aa33XX**aa33XX22aa33 aabbcc ddeeff gghhii will return ddgghhii. @@ Moves the cursor to the absolute location in the output string specified by count. Position 0 refers to the first byte in the output string. If count refers to a position beyond the last byte stored so far, then null bytes will be placed in the uninitialized locations and the cursor will be placed at the specified location. If count is **, then the cursor is moved to the current end of theoutput string. If count is omitted, then an error will be gener-
ated. This type does not consume an argument. For example, bbiinnaarryy ffoorrmmaatt aa55@@22aa11@@**aa33@@1100aa11 aabbccddee ff gghhii jj will return aabbffddeegghhii\\000000\\000000jj. BBIINNAARRYY SSCCAANNThe bbiinnaarryy ssccaann command parses fields from a binary string, returning
the number of conversions performed. String gives the input to be parsed and formatString indicates how to parse it. Each varName gives the name of a variable; when a field is scanned from string the result is assigned to the corresponding variable. As with bbiinnaarryy ffoorrmmaatt, the formatString consists of a sequence of zero or more field specifiers separated by zero or more spaces. Each field specifier is a single type character followed by an optional numericcount. Most field specifiers consume one argument to obtain the vari-
able into which the scanned values should be placed. The type charac-
ter specifies how the binary data is to be interpreted. The count typ-
ically indicates how many items of the specified type are taken fromthe data. If present, the count is a non-negative decimal integer or
**, which normally indicates that all of the remaining items in the data are to be used. If there are not enough bytes left after the currentcursor position to satisfy the current field specifier, then the corre-
sponding variable is left untouched and bbiinnaarryy ssccaann returns immediately with the number of variables that were set. If there are not enougharguments for all of the fields in the format string that consume argu-
ments, then an error is generated. A similar example as with bbiinnaarryy ffoorrmmaatt should explain the relationbetween field specifiers and arguments in case of the binary scan sub-
command:bbiinnaarryy ssccaann $$bbyytteess ss33ss ffiirrsstt sseeccoonndd
This command (provided the binary string in the variable bytes is long
enough) assigns a list of three integers to the variable first and assigns a single value to the variable second. If bytes contains fewerthan 8 bytes (i.e. four 2-byte integers), no assignment to second will
be made, and if bytes contains fewer than 6 bytes (i.e. three 2-byte
integers), no assignment to first will be made. Hence: ppuuttss [[bbiinnaarryy ssccaann aabbccddeeffgg ss33ss ffiirrsstt sseeccoonndd]]ppuuttss $$ffiirrsstt
ppuuttss $$sseeccoonndd
will print (assuming neither variable is set previously): 11 2255118855 2255669999 2266221133 can't read "second": no such variableIt is iimmppoorrttaanntt to note that the cc, ss, and SS (and ii and II on 64bit sys-
tems) will be scanned into long data size values. In doing this, val-
ues that have their high bit set (0x80 for chars, 0x8000 for shorts, 0x80000000 for ints), will be sign extended. Thus the following will occur: sseett ssiiggnnSShhoorrtt [[bbiinnaarryy ffoorrmmaatt ss11 00xx88000000]]bbiinnaarryy ssccaann $$ssiiggnnSShhoorrtt ss11 vvaall;; # val == 0xFFFF8000
If you want to produce an unsigned value, then you can mask the return value to the desired size. For example, to produce an unsigned short value:sseett vvaall [[eexxpprr {{$$vvaall && 00xxFFFFFFFF}}]];; # val == 0x8000
Each type-count pair moves an imaginary cursor through the binary data,
reading bytes from the current position. The cursor is initially at position 0 at the beginning of the data. The type may be any one of the following characters: aa The data is a character string of length count. If count is **, then all of the remaining bytes in string will be scanned into the variable. If count is omitted, then one character will be scanned. All characters scanned will be interpreted as being inthe range \u0000-\u00ff so the eennccooddiinngg ccoonnvveerrttffrroomm command might
be needed if the string is not an ISO 8859-1 string. For example,
bbiinnaarryy ssccaann aabbccddee\\000000ffgghhii aa66aa1100 vvaarr11 vvaarr22 will return 11 with the string equivalent to aabbccddee\\000000 stored in vvaarr11 and vvaarr22 left unmodified. AA This form is the same as aa, except trailing blanks and nulls arestripped from the scanned value before it is stored in the vari-
able. For example, bbiinnaarryy ssccaann ""aabbcc eeffgghhii \\000000"" AA** vvaarr11 will return 11 with aabbcc eeffgghhii stored in vvaarr11.bb The data is turned into a string of count binary digits in low-to-
high order represented as a sequence of ``1'' and ``0'' charac-
ters. The data bytes are scanned in first to last order with thebits being taken in low-to-high order within each byte. Any extra
bits in the last byte are ignored. If count is **, then all of the remaining bits in ssttrriinngg will be scanned. If count is omitted, then one bit will be scanned. For example, bbiinnaarryy ssccaann \\xx0077\\xx8877\\xx0055 bb55bb** vvaarr11 vvaarr22 will return 22 with 1111110000 stored in vvaarr11 and 11111100000000111100110000000000 stored in vvaarr22.BB This form is the same as bb, except the bits are taken in high-to-
low order within each byte. For example, bbiinnaarryy ssccaann \\xx7700\\xx8877\\xx0055 BB55BB** vvaarr11 vvaarr22 will return 22 with 0011111100 stored in vvaarr11 and 11000000001111110000000000110011 stored in vvaarr22. hh The data is turned into a string of count hexadecimal digits inlow-to-high order represented as a sequence of characters in the
set ``0123456789abcdef''. The data bytes are scanned in first tolast order with the hex digits being taken in low-to-high order
within each byte. Any extra bits in the last byte are ignored. If count is **, then all of the remaining hex digits in ssttrriinngg will be scanned. If count is omitted, then one hex digit will be scanned. For example, bbiinnaarryy ssccaann \\xx0077\\xx8866\\xx0055 hh33hh** vvaarr11 vvaarr22 will return 22 with 770066 stored in vvaarr11 and 5500 stored in vvaarr22.HH This form is the same as hh, except the digits are taken in high-
to-low order within each byte. For example,
bbiinnaarryy ssccaann \\xx0077\\xx8866\\xx0055 HH33HH** vvaarr11 vvaarr22 will return 22 with 007788 stored in vvaarr11 and 0055 stored in vvaarr22.cc The data is turned into count 8-bit signed integers and stored in
the corresponding variable as a list. If count is **, then all ofthe remaining bytes in ssttrriinngg will be scanned. If count is omit-
ted, then one 8-bit integer will be scanned. For example,
bbiinnaarryy ssccaann \\xx0077\\xx8866\\xx0055 cc22cc** vvaarr11 vvaarr22will return 22 with 77 -112222 stored in vvaarr11 and 55 stored in vvaarr22.
Note that the integers returned are signed, but they can be con-
verted to unsigned 8-bit quantities using an expression like:
eexxpprr (( $$nnuumm ++ 00xx110000 )) %% 00xx110000
ss The data is interpreted as count 16-bit signed integers repre-
sented in little-endian byte order. The integers are stored in
the corresponding variable as a list. If count is **, then all ofthe remaining bytes in ssttrriinngg will be scanned. If count is omit-
ted, then one 16-bit integer will be scanned. For example,
bbiinnaarryy ssccaann \\xx0055\\xx0000\\xx0077\\xx0000\\xxff00\\xxffff ss22ss** vvaarr11 vvaarr22will return 22 with 55 77 stored in vvaarr11 and -1166 stored in vvaarr22.
Note that the integers returned are signed, but they can be con-
verted to unsigned 16-bit quantities using an expression like:
eexxpprr (( $$nnuumm ++ 00xx1100000000 )) %% 00xx1100000000
SS This form is the same as ss except that the data is interpreted ascount 16-bit signed integers represented in big-endian byte order.
For example, bbiinnaarryy ssccaann \\xx0000\\xx0055\\xx0000\\xx0077\\xxffff\\xxff00 SS22SS** vvaarr11 vvaarr22will return 22 with 55 77 stored in vvaarr11 and -1166 stored in vvaarr22.
ii The data is interpreted as count 32-bit signed integers repre-
sented in little-endian byte order. The integers are stored in
the corresponding variable as a list. If count is **, then all ofthe remaining bytes in ssttrriinngg will be scanned. If count is omit-
ted, then one 32-bit integer will be scanned. For example,
bbiinnaarryy ssccaann \\xx0055\\xx0000\\xx0000\\xx0000\\xx0077\\xx0000\\xx0000\\xx0000\\xxff00\\xxffff\\xxffff\\xxffff ii22ii** vvaarr11 vvaarr22will return 22 with 55 77 stored in vvaarr11 and -1166 stored in vvaarr22.
Note that the integers returned are signed and cannot be repre-
sented by Tcl as unsigned values. II This form is the same as II except that the data is interpreted ascount 32-bit signed integers represented in big-endian byte order.
For example, bbiinnaarryy ssccaann \\xx0000\\xx0000\\xx0000\\xx0055\\xx0000\\xx0000\\xx0000\\xx0077\\xxffff\\xxffff\\xxffff\\xxff00 II22II** vvaarr11 vvaarr22will return 22 with 55 77 stored in vvaarr11 and -1166 stored in vvaarr22.
ww The data is interpreted as count 64-bit signed integers repre- |
sented in little-endian byte order. The integers are stored in |
the corresponding variable as a list. If count is **, then all of |the remaining bytes in ssttrriinngg will be scanned. If count is omit- |
ted, then one 64-bit integer will be scanned. For example, |
bbiinnaarryy ssccaann \\xx0055\\xx0000\\xx0000\\xx0000\\xx0077\\xx0000\\xx0000\\xx0000\\xxff00\\xxffff\\xxffff\\xxffff wwii** vvaarr11 vvaarr22|will return 22 with 3300006644777711007777 stored in vvaarr11 and -1166 stored in |
vvaarr22. Note that the integers returned are signed and cannot be | represented by Tcl as unsigned values. | WW || This form is the same as ww except that the data is interpreted as |count 64-bit signed integers represented in big-endian byte order. |
For example, | bbiinnaarryy ssccaann \\xx0000\\xx0000\\xx0000\\xx0055\\xx0000\\xx0000\\xx0000\\xx0077\\xxffff\\xxffff\\xxffff\\xxff00 WWII** vvaarr11 vvaarr22|will return 22 with 2211447744883366448877 stored in vvaarr11 and -1166 stored in |
vvaarr22.ff The data is interpreted as count single-precision floating point
numbers in the machine's native representation. The floating point numbers are stored in the corresponding variable as a list. If count is **, then all of the remaining bytes in ssttrriinngg will bescanned. If count is omitted, then one single-precision floating
point number will be scanned. The size of a floating point number may vary across architectures, so the number of bytes that are scanned may vary. If the data does not represent a valid floatingpoint number, the resulting value is undefined and compiler depen-
dent. For example, on a Windows system running on an Intel Pen-
tium processor, bbiinnaarryy ssccaann \\xx33ff\\xxcccc\\xxcccc\\xxccdd ff vvaarr11 will return 11 with 11..66000000000000223388441188557799 stored in vvaarr11. dd This form is the same as ff except that the data is interpreted ascount double-precision floating point numbers in the machine's
native representation. For example, on a Windows system running on an Intel Pentium processor, bbiinnaarryy ssccaann \\xx99aa\\xx9999\\xx9999\\xx9999\\xx9999\\xx9999\\xxff99\\xx33ff dd vvaarr11 will return 11 with 11..66000000000000000000000000000011 stored in vvaarr11. xx Moves the cursor forward count bytes in string. If count is ** or is larger than the number of bytes after the current cursor cursor position, then the cursor is positioned after the last byte in string. If count is omitted, then the cursor is moved forward one byte. Note that this type does not consume an argument. For example, bbiinnaarryy ssccaann \\xx0011\\xx0022\\xx0033\\xx0044 xx22HH** vvaarr11 will return 11 with 00330044 stored in vvaarr11. XX Moves the cursor back count bytes in string. If count is ** or islarger than the current cursor position, then the cursor is posi-
tioned at location 0 so that the next byte scanned will be the first byte in string. If count is omitted then the cursor is moved back one byte. Note that this type does not consume an argument. For example, bbiinnaarryy ssccaann \\xx0011\\xx0022\\xx0033\\xx0044 cc22XXHH** vvaarr11 vvaarr22 will return 22 with 11 22 stored in vvaarr11 and 002200330044 stored in vvaarr22.@@ Moves the cursor to the absolute location in the data string spec-
ified by count. Note that position 0 refers to the first byte in string. If count refers to a position beyond the end of string, then the cursor is positioned after the last byte. If count is omitted, then an error will be generated. For example, bbiinnaarryy ssccaann \\xx0011\\xx0022\\xx0033\\xx0044 cc22@@11HH** vvaarr11 vvaarr22 will return 22 with 11 22 stored in vvaarr11 and 002200330044 stored in vvaarr22. PPLLAATTFFOORRMM IISSSSUUEESS Sometimes it is desirable to format or scan integer values in the native byte order for the machine. Refer to the bbyytteeOOrrddeerr element ofthe ttccllppllaattffoorrmm array to decide which type character to use when for-
matting or scanning integers.SEE ALSO
format(n), scan(n), tclvars(n) KKEEYYWWOORRDDSSbinary, format, scan
Tcl 8.0 binary(n)