v5.0 / 6 of 23 / 01 sep 99 / gvg
* The VEE To/From File object provide a convenient means to create a wide range of different file types. The To/From File/Dataset objects are transaction objects that allow you to dump data to files in binary or formatted text form. Datasets are collections of records; you can use From Dataset to select desired records from a dataset on file using various search criteria. This chapter provides various explanations for performing simple I/O transactions.
* A user had a question on writing and reading records from files. I gave
him the following example:
* A VEE user had a problem with storing VEE data whose solution provided a
neat example of basic principles of VEE file storage.
The user was reading temperatures and storing them in a file in text format.
He organized his data as an array of records, with each record providing the
temperature data for one time sample. However, when he tried to read the
data into Excel, there were curly brackets ("{}") around the rows of data in
the spreadsheet.
This was because that's how VEE defines a record when it's placed in a text
file, otherwise it has no way to determine which record is which. For this
purpose, a cleaner approach is to store the data as a two-dimensional array,
with each time sample stored in a row.
I wrote the following program (see xspshdat.vee for the source) to
experiment:
Note that to ensure that the numeric data is comma-separated, you need to
go into "Edit Properties" and configure the "Data Format" for an "Array
Separator" of "," and an EOL string of "\r\n" (CR-LF). (This
configuration task was performed differently in VEE before rev 3.0 but it
can still be done through the To File's object menu.)
The program generates a file of the form:
* A VEE user wanted to be able to sequentially create ten data files in VEE;
as an example, I decided to create ten data files on my PC with the names
"data0.txt", "data1.txt", ... , "data9.txt" -- each containing ten random
integers (in text format) ranging from -100 to +100.
I wrote the following program (see xmltfile.vee for the source) to do
this:
The output count also drives a second For Count object, which on each count
generates ten random numbers and shoves them into a Collector to create a
ten-element array. The Random Number generator is set to generate values
from -100 to +100.
The file name and data are shoved into a To File object configured as
follows:
* If you transfer VEE binary files between HP-UX and Windows, you run into a
problem: the byte order of the numbers in the files is reversed between
HP-UX (Motorola, more or less) and Windows (Intel) and the data from one
looks like gibberish on another.
It is possible, however, to write a simple VEE program to reverse the bytes
for each number in the file, if all the numbers in the file are of the same
type and you know how long (in bytes) the number type is.
For example, if you have a file full of REALs, each REAL is 8 bytes long.
You could easily read in the file, reverse the bytes in each REAL, and then
write it out again to a second file with just a simple program (see
xbytswap.vee for a developed example):
* A VEE user was trying to use VEE to read a text file into a string array.
The text files contained blank lines, which were ignored when VEE read the
file into the string array, leaving only the text lines in the string array.
The user wanted to be able to have the blank lines entered as null
elements in the array.
According to our lab people, this problem can be avoided using a READ
transaction of the following form (see xblankpc.vee for the source):
Since on HP-UX, lines are ended only with a '\n' and not a "\r\n", this
won't work on that platform as is. There's an even more convoluted way
around it, though: since the "sed" ("stream editor") utility is standard on
HP-UX systems, you can modify the target file by adding the "\r" with "sed",
storing the result in a temporary file, and then reading the temporary file
as if it were a PC file (see xblankux.vee for the source):
[6.1] A VEE CONTAINER EXAMPLE
[6.2] STORING SPREADSHEET DATA
[6.3] CREATING MULTIPLE FILES
[6.4] BYTESWAPPING VEE DATA FILES
[6.5] READING BLANK LINES FROM A FILE
[6.1] A VEE CONTAINER EXAMPLE
+--------------------+ +---------------------+
| Record | | To File |
+--------------------+ +-+-------------------+
| FieldName Value | | | |
| [ A ][ Test ] +-->|A| WRITE CONTAINER A |
| [ B ][ 1.25 ] | | | |
| [ C ][ 2.5 ] | +-+--------+----------+
| [ D ][ 5.0 ] | |
+--------------------+ +----------+----------+
| From File |
+-------------------+-+
| | | +---------------------+
| READ CONTAINER X |X+-->| AlphaNumeric |
| | | +---------------------+
+-------------------+-+ | {"Test",1.25,2.5,5} |
+---------------------+
[6.2] STORING SPREADSHEET DATA
+--------------+
| randomseed() |
+------+-------+ +--------------+
| | Concatenator |
+-----+-----+ +---+--+-------+
| For Count +-------------+--------------------->| A | | |
+-----+-----+ | +----->| B | | Array +---+
| | | +--->| C | | | |
| +-----------+--------+---+ | | +->| D | | | |
| | random( 0, 100 ) | R +--+ | | +---+--+-------+ |
| +-----------+--------+---+ | | |
| | | | |
| +-----------+--------+---+ | | |
| | random( 100, 200 ) | R +----+ | |
| +-----------+--------+---+ | |
| | | |
| +-----------+--------+---+ | |
| | random( 200, 300 ) | R +------+ |
| +--------------------+---+ |
| |
| +----------------------------------------------------------+
| |
| | +--------------------------------------+
| | | To File |
| | +---+----------------------------------+
| | | | To File: [ DATA.TXT ] |
| | | | [*] Clear File At Prerun & Open |
| +->+-----------+ | +----------------------------------+
| | Collector +-->| A | WRITE TEXT "N,","T1,","T2,","T3" |
+---->+-----------+ | | WRITE TEXT A EOL |
| | |
+---+----------------------------------+
Some notes on this program:
N,T #1,T #2,T #3
0,92.2947998046875,128.564453125,228.814697265625
1,35.6129150390625,182.8948974609375,262.469482421875
2,56.574462890625,179.84619140625,298.858642578125
...
24,98.7611083984375,142.401123046875,296.54541015625
-- which, when loaded into a spreadsheet, gives:
N T1 T2 T3
0 92.2947998046875 128.564453125 228.814697265625
1 35.6129150390625 182.8948974609375 262.469482421875
2 56.574462890625 179.84619140625 298.858642578125
...
24 98.7611083984375 142.401123046875 296.54541015625
[6.3] CREATING MULTIPLE FILES
+-----------+ +-->+---------+
| For Count | +---------+ | | To File |
| [ 10 ] +--+------>| Formula +-- file name ----|-->+---------+
+-----------+ | +---------+ |
| |
+-----+-----+ |
| For Count | |
| [ 10 ] +--+ data
+-----+-----+ | |
| | |
| +----+----+ |
| | Random | |
| | Number +----->+-----------+ |
| +---------+ | Collector +--+
| +-->+-----------+
| |
+----------------+
The program is quite simple; the For Count object at the top left generates
the numbers 0 through 9 and drives them into a Formula box that generates the
filenames. This Formula box contains:
"data" + A + ".txt"
-- where "A" is the Formula box's input pin.
+--------------------------------------------------+
| To File |
+-----------+----------------------------------+---+
data ---->| A | To File: [ ] | |
| | [ ] Clear File At PreRun & Open | |
| +----------------------------------+ |
| | WRITE TEXT asInt32(a) INT EOL | |
| | | |
file name ---->| File Name | | |
+-----------+----------------------------------+---+
You add the File Name pin by using the object menu and adding a control pin.
The transaction saves the input data as text. Note how the "asInt32()"
function is used to convert the inputs into integers.
[6.4] BYTESWAPPING VEE DATA FILES
+-------+
| Until |
| Break +------+
+-------+ |
|
+--------------+-------------+
| From File |
+----------------------------+
| From File: [ uxdata.bin ] |
+----------------------------+
| READ BINARY x BYTE ARRAY:8 +--+
| | |
| | |
+----------------------------+ |
|
+-----------------------------+
|
| +---------------------------------------------+
| | Formula |
| +---------------------------------------------+
+-->| [ A[7] A[6] A[5] A[4] A[3] A[2] A[1] A[0] ] +--+
+---------------------------------------------+ |
|
+---------------------------------------------+
|
| +---------------------------------------+
| | To File |
| +---------------------------------------+
| | | To File: [ windata.bin ] |
| | | [x] clear file at prerun and open |
+-->| A +-----------------------------------+
| | WRITE BINARY a BYTE |
| | |
| | |
+---+-----------------------------------+
All this does is read in the source file 8 bytes at a time, then use a
Formula box to write the 8 bytes out to the destination file in byte-reversed
order. It's slow, but it works. If you had 32-bit INTEGER data, of course,
the Formula box would be changed to:
+-------------------------+
| Formula |
+-------------------------+
| [ A[3],A[2],A[1],A[0] ] |
+-------------------------+
[6.5] READING BLANK LINES FROM A FILE
+-------------------------------------------------------------+
| From File |
+----------+----------------------------------------------+---+
| | from file [ ] | |
| +--------------------------------------------+-+ |
| | [READ TEXT x TOKEN EXCLUDE: "\n" ARRAY:*] |^+---+
-->| filename | | | X +---+
| | | +---+ |
| | | | | |
| | |v| | |
+----------+--------------------------------------------+-+---+ |
|
+------------------------------------------------------------------+
|
| +-----------------------------------+
| | Formula |
| +-----------------------------------+
+-->| strFromLen( A, 0, strLen(A) - 1 ) +-->
+-----------------------------------+
Basically you read each line as a token delimited by a newline (a line-feed,
'\n'). Since this leaves the carriage return ('\r') at the end of the
line, you also have to trim off the last character to get rid of it. (You
can't specify the "\r" as part of the token -- "\r\n" instead of just
'\n' -- since that eliminates the blank lines.)
+--------------------------------------------------------+
| Formula |
+----------+---------------------------------------------+
-->| filename | [ "sed \'s/$/\r/\' " + filename + " > .t" ] +--+
+----------+---------------------------------------------+ |
|
+--------------------------------------------------+
|
| +---------+---------------------+
+-->| command | Execute Program (UX |
+---------+-----+---------------+
|
+---------------------------+--------------------------+
| From File |
+---+----------------------------------------------+---+
| | from file [ .t ] | |
| +--------------------------------------------+-+ |
| | [READ TEXT x TOKEN EXCLUDE: "\n" ARRAY:*] |^+---+
| | | | X +---+
| | | +---+ |
| | | | | |
| | |v| | |
+---+--------------------------------------------+-+---+ |
|
+------------------------------------------------------------+
|
| +-----------------------------------+
| | Formula |
| +-----------------------------------+
+-->| strFromLen( A, 0, strLen(A) - 1 ) +-->
+-----------------------------------+
The formula box at top creates the proper command string to be executed so
that a "\r" will be inserted into the temporary file, ".t".