3000 NewsWire: How to Use MPE/iX Byte Stream Files

How to Use MPE/iX Byte Stream Files

January 26, 2015

Back when HP still had a lab for the HP 3000, its engineers helped the community. In those days, system architect and former community liaison Craig Fairchild explained how to use byte stream files on the 3000. Thanks to the memory of the Web, his advice remains long after the lab has gone dark.

These fundamental files are a lot like those used in Windows and Linux and Unix, Fairchild said. HP has engineered "emulation type managers" into MPE/iX, an addition that became important once the 3000 gained an understanding of Posix. In 1994, MPE/XL became MPE/iX when HP added this Unix-style namespace.

Understanding the 3000 at this level can be important to the customer who wants independent support companies to take on uptime responsibility and integration of systems. Fairchild explained the basics of this basic file type.

Byte stream files are the most basic of all file types. They are simply a collection of bytes of data without any structure placed on them by the file system. This is the standard file model that is used in every Unix, Linux and even Windows systems.

MPE's file system has always been a structured file system, which means that the file system maintains a certain organization to the data stored in a file. The MPE file system understands things like logical records, and depending on the file type, performs interesting actions on the data (for example, Circular files, Message files, KSAM files and so on).

Fairchild detailed how HP has given bytestream files the knowledge of "organization of data" for applications.

To bridge the gap between standard byte stream file behavior (only the application knows the organization of data) and traditional MPE file type behavior (the file system knows what data belongs to what records), emulation type managers were created. To an MPE application, a byte stream file looks and behaves like a variable record file, even though the data is stored in a way that would allow any Posix application to also read the same data. (Posix applications also have emulator type managers that allow them to read fixed, variable and spool files in addition to plain byte stream files.)

The way that the byte stream emulator detects record boundaries is through the use of the newline (\n) character, which is used, by convention, to separate data in ASCII text files on Unix-based systems.

The underlying properties of a byte stream file are that each byte is considered its own record. In MPE file system terms, a record is the smallest unit of IO that can be performed on a file. (You can write a partial record fixed length record, but the file system will pad it to a full record.) Since the smallest unit of IO that can be performed on a byte stream file is a single byte, that becomes its MPE record size.

In the MPE file system, the EOF tracks the number of records that are in a file. Since the record size of a byte stream file is one byte, the EOF of a byte stream file is also equal to the number of bytes in the file. This is why one 4-byte variable sized record is equal to 5 byte stream records (4 bytes of data + 1 \n character).

It's also worth noting that any file can be in any directory location and will behave the same way. (Well, almost. CM KSAM files are restricted to the MPE namespace. And of course the special files (that you don't normally see) that make up the file system root, accounts and groups are also restricted: one root, accounts as children of the root, groups as children of accounts. And lockwords aren't allowed outside the MPE namespace. But other than that, the opening sentence is true.)

The general model that we had in architecting the whole Posix addition was that behavior of a file does change regardless of where it is located. This was summed up in the saying, "A file is a file." So there are no such things as "MPE files" and "Posix files". There's just files.

What does change is the way you name that file. Files in the MPE namespace can be named either through the MPE syntax (FILE.GROUP.ACCOUNT), or through the HFS syntax (/ACCOUNT/GROUP/FILE). You can also use symbolic links to create alternate names to the same file. This was summed up as a corollary to the first saying, "But a name is not a name."

Posted on January 26, 2015 at 09:47 PM in Hidden Value, Homesteading | Permalink | Comments (0)