November 24, 2015

The Wide World of Connecting Storage

IO used to be more complex for IT. Sure, the array of choices for disk is vast today. But in the era when 3000s used to think they were lucky to get SCSI plugged into them, configuring disk connections was not simple. HP-IB protocol, built to link HP's instruments, was simple, used for all HP devices, and slow. But it was integrated and seamless compared to the SCSI of single-ended, fast/wide, and Ultra Fast.

Such was the case for one 3000 manager seeking advice from his colleagues. You never think about these things on a 3000 until the hardware breaks. Or backups fail. Or storage media gets rare. Aging hardware is one of several issues that require expertise, even if a 3000 runs the ultimate 7.5 version of MPE/iX. Our manager hunted for his help on the longest-running 3000 classroom in the world, the HP3000-L mailing list.

A single-CPU A-Class was moving away from DDS technology, the DDS-3 that was first launched in the '90s. There are other options for 3000 tape backup. But these options include single-ended, fast/wide, and other cable and termination combinations. DLT technology, introduced more recently but still a 1990s choice, runs with HP 3000s. It helps to get the ends right, though, if DLT is to have a new beginning on an old-school 3000.

"Until now they have done their backup on DDS," a manager talking to the 3000 newsgroup explained. "Lately they had a failure on the DDS drive, and have realized that it is getting difficult to get new tapes. They have decided to move to DLT8000, model C6378A, and have bought two of them. One is supposed to go live on the 3000, and the other to be stored as a spare device."

The DLT is hooked to the Ultra Wide SCSI interface on the A-Class. But ODE/Mapper doesn't recognize the device."

There was an error, and no DLT joy. Soon enough, one veteran consultant said, "You will have trouble connecting a fast wide SCSI device to an ultra-wide SCSI controller." It wasn't a rookie mistake, but the veterans who still prowl 3000-L had a solution and even a link to an inexpensive fix. So it goes, here in the fifth decade of HP 3000 mission-critical service. Answers are everywhere.

This wasn't an inexperienced 3000 pro, it seemed, when reading that he tried to "add the device in IOCONFIG by adding first the path 0/0/1/0.2, and then the device with the command: ad 8;path=0/0/1/0.2.0;ID=dlt;mode=autoreply."

SCSI on the 3000 sure isn't the world of USB, where just 2.0 and 3.0 cover the scope of IO choices. A $59 adapter card connected that DLT to the 3000. The IO challenge also prompted advice even a pro might not know — making a case for having fresher hardware than HP's to run MPE.

There was advice about using Mapper on the 3000 to troubleshoot an IO device from Michalis Melis.

Normally the path and the device should be recognized by running ODE/ Mapper without even loading the operating system. You do not have to go to SYSGEN. If Mapper fails you have a problem before the OS loads.

Craig Lalley made the link between two incompatible kinds of SCSI interfaces.

You are trying to hook up a Fast/Wide SCSI device to an Ultrawide interface. The C6378A can only connect to a HVD Fast Wide SCSI interface (A4800A SCSI card comes to mind). Remember, the A-class does not support Dual-Head cards, so your only option is A4800A. You need either a DLT8000 with a Fast-Wide interface, or you need a cheap A4800A HVD (High Voltage Differential) SCSI card. You can daisy chain devices to the card, but I would only use one tape at a time.

Lalley also tipped his hat to Keven Miller, who supplied the link to that $59 adapter card.

Then Denys Beauchemin, who has been among one of the more prolific contributors to the 3000-L, delivered detailed advice about connecting backup devices. His background reaches back to the first decade of 3000 use, including years spent with Hi-Comp on backup software development.

Fast/Wide SCSI (FWSCSI) is essentially HVD SCSI on SCSI-2 standard. This means that the signal is a differential in the voltage between various wires (HVD is High Voltage Differential) and Ultrawide SCSI is SE (Single Ended) SCSI, on the SCSI-2 standard which makes is wide (16 bits), like the FWSCSI.

So what is needed is a converter to power the signal from the Ultra Wide SCSI interface on your server to the FWSCSI interface on the DLT device. I have a number of those somewhere here, but they were for SE SCSI, not UltraSCSI.  They might work for that, since all they did was provide the powered signal and the cable is the one that converted from wide to narrow.

Another thing to consider is that since HP nicely crippled the A-class, that 3000 system would not be able to keep the DLT8000 streaming. And that device hates not streaming, so much so that it will enter shoeshine mode and perform abysmally. Just a parting gift from HP to the MPE community. You should hear what they're doing to the VMS crowd.

That last comment comes from Beauchemin's current duties as migration manager for the OpenVMS users who are leaving that platform. VMS had a steady Internet community to help Digital users, just as the 3000 has 3000-L. People like Beauchemin, largely working outside the 3000 world, are still providing advice for homesteaders -- even while assisting in migrations. After migration there is much to manage, but simply migrating off Hewlett-Packard's 3000 hardware makes using MPE/iX less complex.

Posted by Ron Seybold at 08:07 PM in Hidden Value, Homesteading, User Reports | Permalink | Comments (0)

Get e-mail notice when the NewsWire blog gets a new entry. Just say "Blog Me" in a message to

November 17, 2015

Putting PDF Into a 3000's Data Flow

PDF logoHP 3000 experts know of a wide array of techniques to create PDF files from the server's data, then move them via FTP to a Windows server. While the simplest answer for getting reports into PDF format and then out via Windows is probably Hillary Software's byRequest, there are other commercial solutions. There's also several bolt-together techniques if you've got very limited budget to homestead.

Bob McGregor reports:

We use txt2pdfPRO by Sanface. We had a job that would run and check a pseudo device for spoolfile output, and if the pri > 0, would run the sf2html process, convert to PDF and then FTP to a Windows server. The process would then delete spoolfiles=0 on the pseudo device the next day. Setup took a bit... but once done, worked well.

Lars Appel, author of the Samba/iX file sharing tool, adds:

I wonder if it might make sense to configure a "dummy" network printer on MPE/iX and have it send spooler output to a little socket listener on the Windows system (similar to the FakeLP example from the 3000-L archive) and then invoke GhostPCL on the Windows side for generating the PDF output.

The "dummy" network printer would let the MPE spooler take care of the PCL conversion and also perform the "file transfer" automagically. The GhostPCL software is probably easier to get (or build / update) on Windows than on MPE (okay, I admit that it did also build on MPE long ago.)

John Pitman employed that concept of Appel's in a combination of an off-the-shelf FTP solution, a freeware PDF converter, along with a good deal of innovative 3000 integration.

Nominate a spooled ldev as always suspended (74 in our case, arbitrary). Users can choose this device as their printer in their menu, and all subsequent reports (until changed to another real printer ldev) will go to this device — and therefore NOT physically print. Some reports that are commonly used to import to Excel have been modified to make headings tightly lined up with the data columns, and only print one page heading, to ease the import process.

Run a job on 3000 that every few minutes scans for spoolfiles for this ldev, then copies them to Posix space specific to 74 (for generality), with the creating user and account in the file name (e.g. mgr_stock_O12345.txt), then delete the original spoolfile.

A scheduled program on the Windows box (every minute) connects via FTP to the 3000. When it finds a spool files as above example, it checks for a Windows destination dir of MGR_STOCK, and copies the file to it as O12345.txt, and deletes the 3000 copy of the file. The account name enables segregation of reports for different applications in our case. If the file is > 1MB (an arbitrary size of your choice), it's zipped. It could as easily be converted to any desired form — for example to PDF via the shareware cutepdf.

It could also readily email the output to a user, given access to a mail server, and a way to develop the email address. Users have a client to access the FTP server and obtain their .txt or .zip files

This has been running for more than 10 years with almost no issues. Occasionally a large file might hang FTP, but canceling and restarting the copy usually fixed it. I have seen report selection errors produce 500 MB TXT files.

You might use several suspended ldevs for different types or groups  of users. We ran this on four 3000s in different locations, each with their own separate windows boxes using BP-FTP server. This means that users in Australia can run a report on the Houston or China system to the local printer 74, pause, connect their FTP client to the relevant FTP server, and download the report without having to print it.

The process also enables soft storage of month end reports, which can be very useful for comparative purposes, auditing, and general historical reference. We now have about eight years of this information stored, with backups and CD copies. Much more compact than paper, and cheaper.

Michael Anderson of consulting firm J3K Solutions added that there is also a open source PDF tool, pdfcreator, a manager can use to set up a network PDF printer. "Some assembly required, and batteries not included," he noted.

Another vote came in for the Advant/X software from Tracy Johnson, the OpenMPE volunteer who manages the Invent3K shared open source server. Johnson notes that this STR Software product "while intended to convert spool files and then e-mail or fax them, can be used short of the transmission process."

Posted by Ron Seybold at 05:44 PM in Hidden Value, Homesteading | Permalink | Comments (1)

November 11, 2015

Protecting a 3000 by Eliminating Its Services

Ron as PrivateHere on this day when we celebrate people who have served in the armed forces, a question emerged about enabling HP 3000 JINETD services. Or disabling them, to make a 3000 more powerful and secure. (Yes, it seems to defy the logic about more services being better, one that we can hear in national defense debates. We didn't have such debates at Signal Corps training for the Second Battalion.) The solution to the 3000 service problem included advice on how to trim back risk as well as performance drains on a 3000.

Grigor Terterian said he was having a Series 979 freeze up, because JINETD was receiving a call "for echo udb." Mark Ranft and Denys Beauchemin said the fastest repair would be to comment out echo in the inetdcnf file. Ranft got specific with an example.

# Internet server configuration database
#echo  stream tcp nowait MANAGER.SYS internal
#echo  dgram  udp nowait MANAGER.SYS internal
#daytime  stream tcp nowait MANAGER.SYS internal
#daytime  dgram  udp nowait MANAGER.SYS internal
#time  stream tcp nowait MANAGER.SYS internal
#time  dgram  udp nowait MANAGER.SYS internal
#discard  stream tcp nowait MANAGER.SYS internal
#discard  dgram  udp nowait MANAGER.SYS internal
#chargen stream tcp nowait MANAGER.SYS internal
#chargen dgram  udp nowait MANAGER.SYS internal
telnet  stream tcp nowait MANAGER.SYS internal
#bootps  dgram  udp wait   MANAGER.SYS /SYS/NET/BOOTPD bootpd
#tftp  dgram  udp wait   NET.SYS   /SYS/NET/TFTPD tftpd
ftp  stream tcp nowait MANAGER.SYS /SYS/ARPA/FTPSRVR ftpsrvr

In the example above, only telnet and ftp services are enabled, Ranft said. This led Art Bahrs, a Certified Security Professional, to add that the services you leave on are the ones that can cause trouble, if you don't need them enabled.

Bahrs, who's also a retired Marine, celebrated his Veterans Day with this advice.

You should never, ever, no time, (did I mention 'never'?) run services you don't use or have a business or production need for.

Two reasons: First is security minded. If you have a service active, it is just another way to be hacked. Second is that an active, running service uses machine power, which is wasteful of electrons if there’s no need for it.

Ranft added his experience with inetd on MPE/iX:

Note that the command: - c

will have inetd re-read the configuration.

Your success with this may vary. I've had lots of trouble with inetd in the far past. They got a lot better with the latest (final, for MPE/iX) set of patches. But on occasion, a scheduled restart ( -k) and re-stream will probably help prevent issues.

I run my inetd with the logging feature.

!job jinetd, manager.sys

This allows one to see the offending IP address in the $STDLIST.

Received call for: telnet tcp
telnet/tcp: Connection from unknown ( at Fri Nov 6 19:56:28 2015
Received call for: echo tcp
echo/tcp: Connection from unknown ( at Wed Nov 11 12:56:45 2015
Received call for: echo udp
echo/udp: Connection from unknown ( at Wed Nov 11 12:57:07 2015
Received call for: echo udp
echo/udp: Connection from unknown ( at Wed Nov 11 12:57:25 2015

As one veteran to others, I honor the services of all on this day, and thank you for your efforts toward our security. Long may it wave.

Posted by Ron Seybold at 09:59 PM in Hidden Value, Homesteading | Permalink | Comments (0)

October 20, 2015

Patching an HP 3000 with Patch/iX, Stage/iX

Editor's Note: Today a system manager asked for directions on using AUTOPAT on an HP 3000. That's very old school tech even for a classic system like the 3000 (TOH to Alan Yeo for digging out that link). More than 10 years ago, our contributor John Burke detailed the use of the more-modern MPE/iX patch tools.

By John Burke

Patch bicycleEven after HP introduced Patch/iX and Stage/iX to MPE/iX, these HP tools were poorly understood and generally under-used. Both are tremendous productivity tools when compared with previous techniques for applying patches to MPE/iX. There's a good reason for using the newer tools. Prior to the introduction of Patch/iX and Stage/iX, system managers did their patching with AUTOPAT, and you had to allow for at least a half-day of downtime. Plus, in relying on tape, you were relying on a notoriously flaky medium where all sorts of things could go wrong and create “the weekend in Hell.”

Patch/iX moves prep time out of downtime, cutting downtime in half because you create the CSLT (or staging area) during production time. Stage/iX reduces downtime to as little as 15-20 minutes by eliminating tape altogether and, furthermore, makes recovery from a bad patches as simple as a reboot.

This article is based upon the Patch Management sessions I presented at three Interex Solutions Symposiums. The complete set of 142 slides (over 100 screen shots) and 20 pages of handouts are downloadable from my website. The complete presentation takes you step-by-step through the application of a PowerPatch using Patch/iX with a CSLT and the application of a downloaded reactive patch using Patch/iX and Stage/iX. Included is an example of using Stage/iX to recover from a bad patch.

Why should you care about Patch Management? Keeping a system running smoothly includes knowing how to efficiently and successfully apply patches to the system. HP supplyed bug fix patches to MPE/iX and its subsystems through 2008, including two PowerPatches a year through 2006. Some patches add functionality to older 3000s. Patch/iX is a tool for managing these patches.

Patch/iX can be used to apply reactive patches, a PowerPatch, or an add-on SUBSYS tape with a PowerPatch. Patch/iX is actually a bundle including the PATCHIX program and a number of auxiliary files that are OS release dependent.

Patch/iX allows you to:

• Qualify all patches in a set of patches;

• Install reactive patches at the same time as a PowerPatch;

• Selectively apply patches from a PowerPatch tape; and,

• Create the CSLT (or staging area for Stage/iX) while users are still on the system; i.e., when it is convenient for you without incurring downtime.

Stage/iX is an OS facility for applying and managing the application of patches. Stage/iX reduces system downtime for applying patches to the length of time required for a reboot and provides an easy and reliable method for backing out patches. Stage/iX includes an interface to Patch/iX that creates the “staging area” and two utilities:

• STAGEMAN, which allows you to manage all aspects of patch staging, including which version of the OS will be used for the next update; and,

• STAGEISL, an ISL utility available from the ISL prompt whenever the system is down. It contains a subset of STAGEMAN functionality that allows you to recover from most problems.

Steps in staging

The set of all operating system files, for example NL.PUB.SYS, etc., are considered the current Base OS. Stage/iX creates and manages staging areas, which are HFS directories that hold versions of files that are different from the Base. More than one staging area can exist at a time. Each staging area contains the difference, or delta, between the Base OS and a patched version of the OS.

When a staging area is activated on the next boot, the files in the staging area are moved into their natural locations while the Base versions of the files are saved in a Stage/iX archive HFS directory. To back out a patch, the reverse takes place and the system is restored to its original state.

Once you are satisfied with the new and patched OS, you can COMMIT the staging area to the Base, deleting the staging area directory and all archived Base files. Note that Stage/iX and Patch/iX allow new patches to be staged and applied in a cumulative fashion. In other words, if you create a new staging area while another staging area is active, the new staging area will contain all the changes between the Base and the active staging area plus all the new changes.

Whether or not you use Patch/iX and Stage/iX, the key to successful OS patching is preparation. Information is the key to preparation. The System Software Maintenance Manual (S2M2) for your particular release of MPE/iX is the bible for all patch management activities. It contains a checklist for each possible update and patch activity along with detailed sections corresponding to checklist items. A hardcopy version and a PDF version on CD usually ship with each major OS release.

The S2M2 for each OS release is also available and downloadable from Information about specific patches is available at the IT Resource Center ( A PowerPatch usually comes with some patch specific documentation – make sure you have it available before you start.

Finally, before you ever sit down at the keyboard, create a Patch Book for the specific patch activity you will be attempting. You can do it with the hardcopy manual and a copy machine, but I prefer to use the PDF version, printing out the two-page checklist and each section that makes up the checklist to create my Patch Book.

How to get, apply patches

Suppose you just got the latest Patch Digest and there is a patch you need to install or you’ve read a thread on the 3000-L mailing list that references a patch you think you should install. Let’s look at how you can get and apply the patch using the ITRC, Patch/iX and Stage/iX. Note that the complete reference manual for both Patch/iX and Stage/iX can be found in the appendices of every S2M2 since, and including, the one for MPE/iX 5.5.

Before proceeding too far, check HPSWINFO.PUB.SYS to ensure the patch has not already been applied. [Note that Patch/iX will tell you if a patch, or even a superceding patch, has already been applied, but it only takes a moment to check HPSWINFO and it could save you some time.] Each patch has an eight character ID. For example, consider TIXMXC3B. The first three characters indicate the subsystem; in this case TIX stands for TurboIMAGE. The next four characters are internal to HP’s patch management mechanism. The final character is the version identifier; in this case “B” indicates the second version of this patch.

First off, identify the proper checklist, in this case Checklist B, and create your Patch Book. Next, review the information about the patches at the ITRC; in particular, look for any patch dependencies.

You need to make sure you have the latest version of Patch/iX installed on your target system. This is critical to your success. All sorts of bad things can happen if you use an old or incomplete version of the Patch/iX bundle. To check the version of Patch/iX, sign on as “MANAGER.SYS,INSTALL” and type in PATCHIX VERSION, The program will respond with something like: Patch/iX Version B.01.09

[Ed. note: getting patches from HP is a free process, but you may have some work in front of you to find a place inside HP support that recognizes the 3000 as a server, and knows how to deliver its patches. If you have a maintenance contract with a good third party — Pivital Solutions, for example — they can help lead you to the right HP cubbyhole.]

Once you have the current Patch/iX and your patches, you are ready to run Patch/iX and create your staging area. There are four steps in any run of Patch/iX:

• “Select Activities,” where you define what type of patching activity you want to perform. You have the choice of adding a PowerPatch, adding reactive patches from tape, adding reactive patches from download or adding SUBSYS products.

• “View Patches” (optional): You can actually view information about all the patches that have been applied previously to your system. Note that this can easily number in the hundreds for a system that is kept reasonably up to date.

• “Qualify Patches,” where Patch/iX does a lot of work to determine which patches of the set you supply it with can and/or should be applied.

• Create the stage, the tape, or both that will be used to actually change the OS.

This is all done while normal production continues and places a minimal load on your system. Once you have created your stage with Patch/iX, you run STAGEMAN to activate your staging area with the SET command. The next time you boot your system (and this can be done remotely from your home at 3 AM Sunday morning if you like), your changes will take effect. Total downtime is the time it takes to do a SHUTDOWN followed by a START NORECOVERY.

What if something goes wrong? If you have problems after successfully rebooting your system and you want to back out your patches, simply run STAGEMAN and use the SET command to make the Base the active stage, reboot your system and you are right back where you started. Suppose you cannot even boot the system successfully after setting the stage? Simply boot to the ISL prompt and use STAGEISL to set the active stage to BASE, reboot and, again, you are back to where you started.

Once you are satisfied with your changes after reasonable testing you can again run STAGEMAN and this time use the COMMIT command to make the active stage the Base and free up the disk space occupied by the old Base.

Of course I have only touched on the key points in the Patch/iX and Stage/iX process. There are a number of details I have left out that are covered in your Patch Book. My goal has been to demonstrate the ease with which you can patch an HP 3000 OS using Patch/iX and Stage/iX. Try it once and you will never go back to applying patches via AUTOPAT — nor will you ever again use a CSLT tape to apply patches if you can possibly help it.

Posted by Ron Seybold at 05:54 PM in Hidden Value, Homesteading | Permalink | Comments (0)

October 07, 2015

Reloading, Redux: How To

Loading cementIt used to be the worst thing that could happen to an HP 3000 was a reload of its data. Adager gained its central place in the 3000 manager's toolbox because it would prevent the need for a reload after database restructuring. Although the worst thing to happen to today's 3000 is the loss of an expert to keep it healthy and sustained, reloads are still a significant event.

Not very long ago, a 3000 manager was looking for a refresher on how to do a system reload. Ernie Newton explained why he needed one and shared what he knew. Advice about adding the DIRECTORY parameter, and using BULDJOB1 to set up the accounting structure, followed when Ernie said

We suffered a double drive failure on our Raid 5 disk array yesterday and I'm thinking that I may have to do a reload. It's been almost 15 years since I've done one. If I recall, I do a load when bringing the system up, then do a restore @.@.@.

Our resident management expert Gilles Schipper provided detailed instruction on doing a reload. We hope it's another 15 years for Ernie until he's got to do this again.

Here are the instructions, assuming your backup includes the ;directory option, as well as the SLT:

1. Boot from alternate path and choose INSTALL (assuming alternate path is your tape drive) 
2. After INSTALL completes, boot from primary path and perform START NORECOVERY. 
3. Use VOLUTIL to add ldev 2 to MPEXL_SYSTEM_VOLUME_SET. 
4. Restore directory from backup (:restore *t;;directory) 
5. openq lp
6. Perform a full restore with the following commands
:file t;dev=7(?)
:restore *t;/;keep;show=offline;olddate;create;partdb;progress=5

Gilles adds, "I would suggest setting permanent and transient space each equal to 100 percent on LDEV 2. The 75 percent default on LDEV 1 is fine as long as you don’t need the space. And if you did, your solution shouldn’t really be trying to squeeze the little extra you’d get by increasing the default maximum limits."

The reason for limiting LDEV to 75 percent is to minimize the otherwise already heavy traffic on LDEV 1, since the system directory must reside there, as well as many other high traffic “system” files.

You won't want to omit the ;CREATE and ;PARTDB options from the restore command. Doing so will certainly get the job done -- but perhaps not to your satisfaction. If any file that exists on your backup was created by a user that no longer exists, that file (or files) will NOT be restored.

Similarly, if you omit the ;PARTDB option, any file that comprises a TurboIMAGE database whose corresponding root file does not exist, will also not be restored.

I suppose it may be a matter of personal preference, but I would rather have all files that existed on my disks prior to disk crash also exist after the post disk-crash RELOAD. I could then easily choose to re-delete the users that created those files -- as well as the files themselves.

Another reason why the ;SHOW=OFFLINE option is used is so that one can quickly see the users that were re-created as the result of the ;CREATE option. Purging the "orphan" datasets would be slightly more difficult, since they don’t so easily stand out on the stdlist.

Finally, it’s critical that a second START NORECOVERY be performed. Otherwise, you cannot successfully start up your network.

Posted by Ron Seybold at 04:10 PM in Hidden Value, Homesteading | Permalink | Comments (0)

September 23, 2015

Where Do Those DBEs Go In MPE?

Where can I get help with storing and restoring an Allbase DBE?

Gilles Schipper replies

SQLUTIL.PUB.SYS should let you access the DBE. IMAGESQL is to a DBE what QUERY/DBUTIL is to TurboIMAGE.

Denys Beauchemin adds

ISQL.PUB.SYS is used to access the data in a DBE. If a TurboIMAGE database is attached to a DBE, there is a DBTC file for each Turbo database and an ATCINFO file with the DBE.

How can I find all the IMAGE databases on a system?

Michael Anderson replies:

IMAGE database root files all have a unique filecode value, -400. IMAGE datasets all have a value of -401. So if you want to find all IMAGE databases on a MPE system I would use the following command:


This will give you all the IMAGE root files and only the IMAGE root files. If you use something like “listf @01.@.@” you get the first dataset in the database, but also any file where the filename ends with “01”, and that may or may not be a database file. Also, the MPE file system allows filecode representations to be alpha (“PRIV”) and Numeric. When you see “PRIV” as a filecode it simply means that the numeric value of the filecode is negative, and again this can include non-database file types.

What is the command to kill a session that refuses to die?

Ernie Newton and Sam Knight reply:

If it is a VT or Telnet session, try NSCONTROL KILLSESS=#Snnnn

What is the maximum sized LUN that MPE 7.5 will recognize?

Guy Paul replies:

You should be able to have up to a 146 GB LDEV. Trial and error over the years has taught me to put no more than 70Gb/FWSCSI on a busy system. If you have Hyper-Volume extension, I would suggest slicing the mechs up a little smaller.

I have a couple of configuration questions about an N-4000. Does it make sense that an A6795A (PCI 4x 2Gb/s Single-port Fibre Channel Adapter) would be installed in PCI slots 1 or 2? The system's docs indicate that PCI slots 1 and 2 are 2x PCI slots; so the fibre channel would be limited to 1Gb/s. Does it make sense to have two Fiber Channel cards for a single VA7110?

Craig Lalley and Guy Paul reply:

The VA7110 does have two fiber connections, and both CAN be used. However, in order to be consistent with HP’s maintenance criteria, you should have one connected to a Command View workstation. Everything in Command View can be done through the serial port on the VA7100, except for logging. It is also a lot easier to define LUN’s graphically (Command View) as opposed to the command line (cryptic) language on the serial port.

Also on the VA7110, one of the fiber channels is “passive” and is for the command view workstation. I would not install a A6795A HBA into slots 1 or 2. Slots 3-12 are the twin-turbo (4x) slots you should use.

Posted by Ron Seybold at 09:02 PM in Hidden Value | Permalink | Comments (0)

September 18, 2015

Passing audits: MPE privileges can be keys

Migrated HP 3000 data can become forgotten while making provisions for an audit. Since some HP 3000s work as mission-critical servers, these active, homesteading systems must weather IT and regulatory audits. The 3000 is capable of passing these audits, even in our era of PCI, HIPAA and Sarbanes-Oxley challenges — all more strenuous than audits of the past.

However, establishing and enforcing a database update procedure is a step onto filling the gap in the security of an MPE/iX system. HP 3000 managers should take a hard look at how their users employ System Manager (SM) privileges. (Privileged Mode, PM, and System Supervisor OP should also be watched. Overall, there can be 21 capabilities to each user.) In their most strict definition, those privileges can expose a database. Hundreds of users can be created at Ecometry sites; even seasonal help gets SM users, according to one consultant's report, users which are seldom deleted after the holiday has passed. One site had a script to create new users, and each had PM capability, automatically.

Privileges are often a neglected aspect of 3000 operations, especially when the system's admin experts have moved on to non-3000 duties, or even to other companies. (Then there's the prospect that nobody knew how to use privileges in the first place.) Some SM users have disturbed the integrity of 3000 databases. It's easy to do accidentally. A creator of a database can also update a 3000 database — a capability that can foul up a manager's ability to pass some audits.

VEAudit from VEsoft, using its LISTUSER @.@ (CAP("SM")) filter, can give you a report of all of the SM users on your HP 3000. You can even ask for the SM users where password="". (Now there's a good list to find: SM users who have no passwords.) There is no MPE command that will do such things, we are reminded by VEsoft co-founder Vladimir Volokh. Even after more than three decades of his business as a 3000 software vendor, he also offers consulting on MPE operations and management, and still travels the US to deliver this. 

If you are worried about arbitrary access via QUERY, you can "disable subsystem access" via DBUTIL. This will, of course, only disable the access on QUERY.

Some less-adept auditors can also demand that a database's password be changed every 90 days. It's quite impossible to do, considering the database password is built into every application program.

So a database's security might be compromised through SM privileges, but it depends on the meaning of "update." This term can be construed to be as restrictive as using DBUPDATE to change an entry. It can also refer to UPDATE access DBOPEN MODE 2. 

To get very specific, an update can mean that the modify date has been changed in the file label of one or more IMAGE-related files. In a very general definition, an SM user can update the database simply by way of a restore from tape. (OP privileges permit this, too.)

Auditors sometimes ask broad questions, the sort of inquiry that fits better with the everyday use of HP 3000s in an enterprise. But for MPE/iX experts, "update" means any kind of modification capability.

So you can answer your auditor's question and say "no, SM privileges don't permit any of our users to update a database in another 3000 account." This answer is true, to the extent that the auditor's concern is about changing data — not just making a minor date change or using DBOPEN MODE 2. For auditors without MPE/iX and IMAGE expertise, well, they might not go so far in their examinations.

As for the SM user's ability to muck up an IMAGE database, it’s a mistake that is not difficult to make. An SM user who obtains a database password can corrupt an IMAGE database just by using the restore command. We’ve heard a story that such a user might explain, "Oops, I thought I was signed onto the test  account."

It's important to make a system fool-proof, because as Vladimir says, "fools are us." 

Posted by Ron Seybold at 04:31 PM in Hidden Value, Homesteading | Permalink | Comments (0)

September 15, 2015

COBOL Tools for Comma Separated Values

CSV exampleIt's been a long time since I wrote in COBOL, and I have an elementary question about creating a comma-delimited file. If I want...

"item one", "item two","","","item three"

How do I create the "," between item one and item two?

And the ","",""," would it be the same?  Just put that inside double quotes?

Something tells me that there is an escape sequence, but the mind is not cooperating. 

Walter Murray, who worked in HP’s Language Labs before moving on to other 3000 work, replies

The STRING statement is helpful. I don't advocate using an apostrophe to delimit nonnumeric literals, preferring to stick with standard COBOL. And yes, QUOTE is a figurative constant guaranteed to give you a quotation mark.

Murray offers a COBOL sample program to illustrate.
000100  IDENTIFICATION DIVISION.                            
000200  PROGRAM-ID. COBTEST.                                
000610  DATA DIVISION.                                      
000620  WORKING-STORAGE SECTION.                            
000630  77  ONE-COMMA    PIC X       VALUE ",".             
000640  01  1ST-ITEM     PIC X(6)    VALUE "ITEM 1".        
000650  01  2ND-ITEM     PIC X(6)    VALUE "ITEM 2".        
000660  01  3RD-ITEM     PIC X(6)    VALUE "ITEM 3".        
000670  01  MY-RECORD    PIC X(72).                         
000700  PROCEDURE DIVISION.                                 
000800  1000-START.                                         
000900      INITIALIZE MY-RECORD                            
001000      STRING  QUOTE 1ST-ITEM QUOTE ONE-COMMA          
001010              QUOTE 2ND-ITEM QUOTE ONE-COMMA          
001011              QUOTE          QUOTE ONE-COMMA          
001012              QUOTE          QUOTE ONE-COMMA          
001013              QUOTE 3RD-ITEM QUOTE                    
001014              DELIMITED SIZE                          
001015              INTO MY-RECORD                          
001016      DISPLAY MY-RECORD                               
001020      STOP RUN.       

And the output...

"ITEM 1","ITEM 2","","","ITEM 3"

Tony Summers adds

It's worth putting a final comma at the end of the line, otherwise some versions of Excel can make the last column very wide.

Ken Roberston notes

One item worth noting, is that right before the STRING statement, you should move spaces to your output buffer.  STRING will not clear the buffer, so if your current string expression turns out to be shorter than your last, whatever was there before will remain - and you will get bad values in your output.



Finally Robert Mills has a macro for COBOL that creates a CSV record (copy and paste to retrieve all the characters in every line of the macro).

*> *************************************************************************
*> %AppendCsv(Field#,CsvRecord#)
*> -------------------------------------------------------------------------
*> Append Field to CsvRecord. If Field contains a comma (,) then Field is
*> quoted (") before being added. Quotes (") within Field will be replaced
*> with single-quotes (').
*> *************************************************************************

01 AppendCsv-macro.
05 AppendCsv-comma-count pic s9(04) comp value zero.
05 AppendCsv-field pic x(512) value spaces.
05 AppendCsv-field-length pic s9(04) comp value zero.
05 AppendCsv-index pic s9(04) comp value zero.
05 AppendCsv-pointer pic s9(04) comp value zero.

$define %AppendCsv=
initialize AppendCsv-macro
move function trim(!1) to AppendCsv-field
move length(function trim(AppendCsv-field)) to AppendCsv-field-length
move length(function trim(!2, trailing)) to AppendCsv-pointer
if AppendCsv-pointer > 1 then
move "," to !2(!3:1)
add 1 to AppendCsv-pointer end-add
varying AppendCsv-index from 1 by 1
until AppendCsv-index > AppendCsv-field-length
if AppendCsv-field(AppendCsv-index:1) = "," then
add 1 to AppendCsv-comma-count end-add
if AppendCsv-field(AppendCsv-index:1) = quote then
move "'" to AppendCsv-field(AppendCsv-index:1)
if AppendCsv-field-length > zero then
if AppendCsv-comma-count > zero then
quote, function trim(AppendCsv-field), quote delimited by size
into !2 with pointer AppendCsv-pointer
function trim(AppendCsv-field) delimited by size
into !2 with pointer AppendCsv-pointer

Posted by Ron Seybold at 03:58 PM in Hidden Value, Homesteading | Permalink | Comments (0)

August 31, 2015

Posix file movements, using FTP and more

I'm attempting to move files in and out of Posix namespace on my HP 3000. The file I've copied becomes bytestream, and has a REC of 1. But I want to transfer that file from Posix down to my PC, I need to maintain it's structure — but what appears to be happening is it's one long record, with no separators. Is there a way (automated) that I can move files in and out of Posix, maybe FCOPY, and be able to keep the structure?

Donna Hofmeister replies

Simply copying/renaming a 'regular' MPE namespace file into HFS-namespace will not change its structural attributes.

If the MPE file was ';rec=-80,,f,ascii' to begin with, it will still be that afterwards.  And it will retain those attributes (cr/lf in particular) following an FTP transfer from your 3000 to [something else less enlightened].

To have a foreign/non-MPE filed take-on MPE fixed-record length attributes during an FTP transfer, simply add something like the following on your transfer line:

[put|get] non_mpe_file_name MPEFILE;rec=-NN,,f,ascii (making all the proper substitutions)

How do I get my HP 3000 to play well with Web-based FTP clients?

HP's James Hofmeister, who led the effort to keep FTP up to date on the 3000, replies

Lots of work went into an implementation of the FTPSRVR to support web access to the 3000. The "SITE POSIX ON" command can be sent by a FTP client and the 3000 FTPSRVR will emit Posix "standard" FTP output and will react like a Posix host (including file naming conventions).

It also is possible as documented to specify "POSIX=ON" mode in the file and achieve this functionality system-wide for all non-3000 client to 3000 FTPSRVR connections; again the FTPSRVR will emit Posix "standard" FTP output and will react like a Posix host (including file naming conventions).

Warning:  Before you specify "POSIX=ON" mode in the file, make sure you read the FTPDOC file closely; as you are warned that MPE file syntax will "no longer" work; The 3000 FTPSRVR is acting in Posix mode.

Posted by Ron Seybold at 06:41 PM in Hidden Value, Homesteading | Permalink | Comments (0)

August 19, 2015

Stripping on the 3000, Carriage-Style

How can I strip out the Carriage Controls from a spool file?

Tony Summers replies:

CarriageWithout dropping into Posix shell, the only other idea that comes to mind would be some third party tool. SPOOLPDF was a program we used from Open Seas to convert spoolfiles into raw PCL. A second program (OPENPDF) subsequently converted the PCL to a PDF document. OPENPDF was simply ported version (to MPE) of a Unix application (pcl2pdf). And we still use pcl2pdf on our HP-UX servers.

Lars Appel pulls a new approach out of his files:

Well, EDITOR.PUB.SYS also can change or replace columns.

/CHANGEQ 1/1 to "" in ALL

This changes column 1 through 1 to "nothing" in all lines (quietly).

If you don't want to strip CCTL but convert it to PCL escape codes, you might try using the network spooler (which typically does this when sending a file to the JetDirect printer). Example programs are in the HP3000-L archives, such as listening for TCP port 9100 to capture such data sent by the network spooler. Look for the FakeLP challenge with examples in Java, Perl, and C.

Tracy Johnson adds a note on a free utility:

I use Beechglen's SF2HTML utility. It's a convenient tool to convert CCTL codes to line and form feeds. Then I run it through an editor to get strip the HTML code. It only generates three codes, "<HTML>", "<BODY>", and "<LISTING>".  Easy enough to convert to blanks or nothings. Then use the Sanface Software program txt2pdf to convert the edited file to the final result.

Beechglen's Doug Werth notes

With enhancements in later versions, SF2HTML can be controlled by several variables, including one to remove the HTML tags from the output. This eliminates the extra step of running the resulting file through an editor before feeding it to txt2pdf.

Dave Powell offers a command file alternative:

Many moons ago I wrote a command file, HP2RTF to convert a cctl file to word-processor-readable rich-text format, converting the carriage control codes to the appropriate number of line-feeds as part of the process. It's happier if you redirect your output to a CCTL disk file, but it can sort-of handle spool files too. It contains its own complete source code, so if it doesn't do exactly what you want you can tweak it.

Posted by Ron Seybold at 08:15 PM in Hidden Value, Homesteading | Permalink | Comments (0)

August 12, 2015

How to Keep Watch on Backup Completions

DAT tapeWe've had a backup hang up on a bad DAT, and we learned about it when the morning jobs couldn't start on the 3000. (We shut everything down we can, back up, then open everything up again.) To find a better way to respond to this, I'm making a procedure to compare the expected backup duration (from a table we've built) to the backup's actual duration so far. The idea is to get an early report if the duration has been exceeded by more than an hour.

I've parsed the JOBLIST output to get what I wanted. But it looks like I'll need help on converting a string variable to a numeric variable as part of this procedure. Does MPE have anything like that?

Francois Desrochers replies with this ![...] construct, an undocumented part of MPE/iX:



would create a string variable, you could do something like:


to create a numeric variable.

We're trying to restore about half a terabyte of data from an old HP 3000 backup set to a non-3000 machine. Is there a sound way to do this, or any way at all?

Stan Sieler replies:

We've done this for a customer who was without an HP 3000. We decided to use MPE/iX on our system to

  1. Restore their data onto our 3000
  2. Package up the files (either with tar, or LZW)
  3. Move them to an external disk drive on a Unix or Linux or Windows machine
  4. Unpack them
  5. Then send that external drive to the customer.

This morning I came in to find our backup job stalled. Abortjob was ineffective, as was abortio. I ended up rebooting the system. While coming up, I got a “defective sector” message with “FILE.GROUP.ACCOUNT has an extent with unreadable data.” The file is now locked, and I need to use FSCHECK to unlock it. How can I determine which drive this extent is on?

Stan Sieler replies

FSCHECK’s DISPLAYEXTENTS command may help. Note that, if I recall correctly, it displays logical unit numbers, not exactly LDEVs.

I'd like to make sure I get complete backups. Should I be using @.@.@ in my TurboSTORE command?

Gilles Schipper replies that backups don't need to be specified with an @.@.@ command to be complete:

People should really be using the forward slash, because it's easy to accidentally omit the Posix file structure if you're not careful constructing your fileset backup. The slash is so much better — a backup specified by HP's TurboStore will replace any @.@.@ operation with "./" Combining @.@.@ with exclusions can lead to omitting files which should have been in a backup.

Posted by Ron Seybold at 05:56 PM in Hidden Value, Homesteading | Permalink | Comments (0)

August 10, 2015

How to Make a 3000 Act Like It Uses DNS

DomainsI have a script that uses FTP to send files to a site which we open by IP address. We've been asked to change to SFTP (port 22) and use the Domain Name Service name instead of an IP address. Does the 3000 support using DNS names?

Allego's Donna Hofmeister replies:

To start, I'm not sure you want to do SFTP on port 22. That's the SSH port. SFTP is meant to use port 115. Have a look at one of Allegro's white papers on how to enable SFTP on MPE

If you are going to use DNS, you must have your 3000 configured for that.  It's easily done.

However, if you've never done anything on your 3000 make it act like a real computer (oh -- that's right, it is a real computer and fully capable of using DNS), this can turn into a can o' worms.

For 'DNS lite' it's probably simplest to:

1. Copy hostsamp. net to

2. Edit to make sure it has loopback   name    <--- where and name are corrected to the system you want to connect to

3. Copy the to

4. Edit to have this line:

hosts : files[SUCCESS=return NOTFOUND=continue]

With this done, the 3000 sorta kinda acts like it's using DNS — because it's looking the the hosts file for how to translate 'name' into ''

Tony Summers provides a caveat:

One warning. The upgrade from FTP to sFTP (or SSH FTP etc) can involve more change to your scripts than you expect.  What we do for FTP (originally on the HP 3000, and now on the HP-UX server) is build a text file with the commands (the sample below, edited)

cat FTPT0070
get /export/002_iccm_extract_1161.csv ICR21161


The file is then presented to the FTP client. On the HP 3000 it was something like....


Then both the output file, FTPS0070, and any JCWs set by the FTP program were inspected to test the success of the FTP session.

cat FTPS0070

Connected to

220 Welcome to FTP service - xxxx.
331 Please specify the password.
230 Login successful.
200 Switching to ASCII mode.
200 PORT command successful. Consider using PASV. 550 Failed to open file.
221 Goodbye.

In particular,  the 3-digit status codes were analysed,  looking for error codes like "550".If you do something similar in your FTP scripts,  then all I can say is welcome to a very different world.

Karsten Brøndum adds:

Here's a completely different approach. 

Depending on your skills in the Java area, there is a nice LPGL package called ftp4j (which requires Java 1.4 or later) that I have used a couple of times. It's a full-featured Java-based FTP client. (By the way, ftp4j will do both SFTP and FTPS). I've found it way easier than to fiddle with files with text files containing commands, especially when it comes to error handling.

Posted by Ron Seybold at 05:04 PM in Hidden Value, Homesteading | Permalink | Comments (0)

July 17, 2015

Do Secure File Transfers from the 3000

I'm trying to use to FTP a file to a SFTP server and it just hangs. Is there a way to do a secure FTP from the HP 3000?

Brian Edminster replies:

The reason that using MPE's FTP client ( fails is because as similar as they sound, FTP and SFTP are very different animals. Fortunately, there is a SFTP client available for the 3000 -- the byproduct of work by Ken Hirsh and others.

It used to be hosted on Ken's account on Invent3K, but when that server was taken out of service, so was Ken's account. As you've no doubt already noticed, it's available from a number of sources (such as Allegro). I'd like to highlight another source:

Edminster goes on to explain he administers that site, as well as puts together the 'pre-packaged' install available there. It's in a single store-to-disc file in Reflection 'WRQ' format, making it easy for the majority of sites to retrieve and use.

I have a customer that's been using SFTP daily as part of their PCI compliance solution for several years. They push and pull data hourly from dozens of Point-of-Sale systems all over the country, and have moved lots of data this way.  

The biggest caveat from that customer's implementation is that if you're moving data over a WAN, SFTP seems to be more sensitive to jitter and latency issues than conventional FTP.  We ended up having to upgrade a couple of their more anemic 'last mile' circuits to accomodate that.  

In all other respects, it’s quite a robust solution, and can be tightly integrated with existing legacy apps. I know; I've done it.

If you have any questions about how to use the pre-packaged install -- or how to get around any limitations you might run into,-- don't hesitate to contact me. I've used this on dozens of systems over the last decade, and have transferred many, many gigabytes of data with it.

Posted by Ron Seybold at 09:02 PM in Hidden Value, Homesteading | Permalink | Comments (0)

July 08, 2015

Freeing HP's Diagnostics Inside the 3000

DiagnosticsWhen HP officially closed its formal HP 3000 support, the vendor left its diagnostics software open for use by anybody who ran a 3000. Throughout the years when HP sold 3000 support, CSTM needed a password that only HP's engineers could supply. But the CSTM diagnostics tools started to run in 2011 without any HP support-supplied password. 

However, managers need a binary patch to free up the diagnostics. Support providers who've taken over for HP know how to enable CSTM. The community has a former Hewlett-Packard engineer to thank, Gary Robillard, for keeping the door to the diagnostics open. Robillard says he is the engineer who last worked on CSTM for MPE/iX when he was a contractor at HP.

A 3000 site must request a patch to get these expert tools working. HP arranged for 3000 sites to get such patches for free at the end of 2010. We tracked the procedure in a NewsWire story, since the HP link on how to get these patches, once on the old division's webpages, has gone dead.

One such patched version of CSTM needs a binary patch. Robillard created this binary patch fix.

Versions of CSTM [patched] with ODINX19A or ODINX25A allow the expert tools with no licensing, but you still have to issue the HLIC command. 

If you install ODINX25A/B/C (6.5, 7.0, 7.5) you won't need to do anything except issue the HLIC command with any password. The HLIC command might say it was not accepted, but the license is activated anyway

When HP halted all its support of the 3000, Robillard said his patch corrects the problem with ODINX19A -- and gives 3000 managers access to these system diagnostics -- for servers running the 6.5, 7.0, or 7.5 versions of CSTM.

If you have installed ODINX19A, you need to do the following:

Logon as MANAGER.SYS. It's safest to create an input file to sompatch.

1. Run

2. Add the following three lines EXACTLY. The sompatch will only work if the instruction at offset 268 matches 86a020c2. The message "Error: Old value does not match" is displayed and no changes are made)

Here are the contents of BINPCHIN file (You will want to copy and paste these): 

~~~~~The 3 lines are below~~~~~~~

; Fix problem in DIAGMOND after 12/19/2010
modify ms_init_manage_sys  + 268,1 86a020c2|08000240 
~~~~The 3 lines are above~~~~~~~~
K binpchin,unn 

• Make sure DIAGMOND is not running (run STMSHUT.DIAG.SYS)
• copy /usr/sbin/stm/uut/bin/sys/diagmond,DIAGMOND 
• copy DIAGMOND,/usr/sbin/stm/uut/bin/sys/diagmond;YES 

After a few minutes, a "SHOWPROC 1;TREE;SYSTEM" should show the DIAGMOND process, and either the mapping processes, or memlogd, diaglogd and maybe cclogd (on A/N Class 3000s only).

Posted by Ron Seybold at 10:13 PM in Hidden Value, Homesteading | Permalink | Comments (0)

June 22, 2015

Fixing Date Problems From The Future

HP 3000 managers have traveled long roads toward the future of their servers, but sometimes the server travels even farther. Into the future, it seems, to apply modification dates to files that couldn't possibly be modified months or years from now.

Back To The FutureThis can cause problems with system maintenance. Craig Lalley experienced some last week. After running the NMVALCK command, he discovered "I have thousands of files with future dates." He was pretty sure there's a way to adjust a date like FRI, SEP 10, 2027, 1:53 AM by using MPEX. (A good bet, since the Vesoft product manages the 3000's files better than MPE/iX itself). But what about other repair options?

There are two, one in the community's freeware resources, and one in its Posix namespace. The freeware comes from Allegro Consultants. FIXFDATE (just do a "find" on the web page to locate the utility's entry) "will sweep through your files and change any creation, modification, access, allocation, or statechange date that is a "future" date to be today."

Another resource comes from within the 3000's Fundamental Operating System. Touch, a common Posix utility, exists on the HP 3000's implementation. 

Touch is chronicled on the Open Group Base Modifications website. While Posix is not as well-loved in the 3000 community as CI commands or freeware, its touch is bristling with options. The basics:

Touch shall change the last data modification timestamps, the last data access timestamps, or both.

The time used can be specified by the -t time option-argument, the corresponding time fields of the file referenced by the -r ref_file option-argument, or the -d date_time option-argument, as specified in the following sections. If none of these are specified, touch shall use the current time.

Changing the dates of a full system's worth of files might someday be the mission for any 3000 owner who's trying to carry MPE beyond January 1, 2028. The world of 3000 Future Time hasn't been explored much, mostly because the community seems confident some solution will be available in 2027. It's still 12 years away, after all.

Allegro's Steve Cooper knows date representation issues have been addressed by the 3000 development community before -- in an era when many companies were still engineering for the system. He can sound sanguine about the issue because his partner Stan Sieler engineered date repairs for the 3000 during Y2K work at the end of the 1990s.

Allegro sold a Y2K utility as part of its 3000 development toolset. "If anyone cares by then, one will need to do remediation similar to what was done for Y2K," Cooper has said. "Each program will need to be inspected for vulnerabilities, then fixed to use an alternate method of date storage and manipulation."

And, yes, as you suspect, this could arise before then. If, for instance, you are manipulating contract expiration dates in your COBOL program, and are using a 2027-sensitive format, then you will not be able to correctly handle any date past the 2027 cut-off.

If you don't mind, though, I'm not going to lose any sleep over this issue for another several years. Remind me again toward the end of the decade, and I'll ask Stan to look into it.  If I ask him now, I will lose a few months of productivity out of him, while he solves next decade's problems.

The issue is not limited to the HP 3000, even though the 2027 date is unique to MPE/iX. Unix has got the same kind of deadline approaching 11 years later. Cooper pointed to a Wikipedia article that explains the "Year 2038 problem" as an analogy to the 3000's.

Posted by Ron Seybold at 07:03 PM in Hidden Value, Homesteading | Permalink | Comments (0)

June 12, 2015

Find hardware specs, move DTCs, and more

Is there a command or way to see the hardware specs of a HP 3000 via MPE or its installed utilities? This machine has no other utilities, like MPEX. I am looking to document the processors, memory, number of hard drives, and size of those drives. 

Jack Connor replies

Depending on MPE/iX version, you can use SYSDIAG for 6.0 and older or CSTM for 6.5 and later. In SYSDIAG, type SYSMAP, then IOMAP, and GENERAL for the IO components, then exit and go to CPUMAP for the CPU info.

In CSTM, type MAP, then SELECT ALL, then INFO, then IL (InfoLog) to get a listing of everything that MPE owns.

I don't work that much with COBOL these days, but I wanted to compile a  program and I got an error message,"size of data segment greater than 1 gig or 64 bytes" How do I get around this?

Steve Cooper replies

That means that the total space you asked for in your Working Storage Section is more than 1 GB.  Now, there are ways to work around that, but my guess is that you don't need to work around that.  My guess is there is a typo or some other unintended problem, where you are asking for way more storage than you intended. Check your OCCURS clauses and PICs to make sure you mean what they say.

We have to move a DTC into our network. Along the way there are Procurve switches and a Cisco router or two. I know that somehow the switches and routers must be configured so as to allow multicasting on addresses 09-00-09-xx-xx-xx to be forwarded and not filtered, but our Procurve administrators aren't quite sure they know how exactly how to do this. What is Procurve-ese for configuring what's necessary to allow remote DTC operation across our network?

Jeff Kell replied

You don't have to do anything at layer-2.  The layer-3 will have to bridge the traffic.  If the layer-3's are Cisco, you can specify the traffic to bridge, in which case you just want the [08|09]-00-09-x-x-x traffic.

The 09-00-09 is multicast used for discovery, but beyond that, you're going to get some directed layer-2 08-00-09 as well.  You will need to include your HP's DTSLINK NIC MAC prefix as well if it is a later model that does not have the old HP 08-00-09 prefix.

If you own the whole infrastructure path, what we used to do is run L2 trunks between sites, and propagate a common vlan for the DTC/HP traffic, while routing everything else.

Or if the Ciscos are real routers rather than Catalyst-style switches, you can run a L2TP tunnel end-to-end to propagate the HP vlan across the routed hops.

I need to rebuild an environment from one HP 3000 system to another. Trouble is, we want to have groups from the same account end up on different user volumes. Is there a way to do this using BULDACCT? 

Keven Miller replies

BULDACCT was made for processing complete accounts. Do BULDACCT  CHC%VSACCT=MEDADV_1. Then edit BULDJOB1 for the other group, changing MEDADV_1 to _2

Mark Ranft adds 

What I usually do is use BULDACCT to move the entire accounting structure. Then I surgically PURGEGROUP and NEWGROUP (with the appropriate HOMEVS= and ONVS= options, plus CAP= and ACCESS= etc) to duplicate the special groups.

I'm having trouble locating TurboIMAGE limits. What's the max number of items in a dataset?

Paul Edwards replies

From HP’s website: 

The objective of this TurboIMAGE enhancement is to raise the limits on number of items, sets and master paths in a TurboIMAGE database to these values: 

Number of items: 1200
Number of sets per database: 240
Number of paths for master : 64

All new databases created will have the benefit of increased limits. The new limits will not be applicable for old databases created under previous versions of TurboIMAGE. However, these databases will continue to function with the old limits and applications will be compatible.

Posted by Ron Seybold at 06:17 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 27, 2015

Make old PCs do a console's work

HP PortableGot a wheezing PC someplace in your IT shop? Believe it or not, even the creakiest of desktops can still serve your HP 3000: as a console, a la the HP700/92 variety. This is the kind of PC where, as one veteran puts it,"the keyboards have turned to glue."

...Trying to type a coherent instruction (or even worse, trying to talk someone through that task remotely) where random keys require the application of a sledgehammer to make them respond, at which point they auto repeatttttttttttttttt.

It's enough to give a veteran manager a pain in the posterior, but hey -- some HP 3000s (of the 900 Series) demand a physical console as part of their configuration. Can't you just hook up such an antique PC straight to the 3000's special console port and let it work as a console? Yes, you can.

Overheard among chatter between 3000 vets:

You can connect a PC via its serial port to the console port on the HP3000, and then run a terminal emulator via the serial connection, leaving it logged on as the console. That way, using free remote control software (VNC Free) on the PC, you could even have control of the physical console (as opposed to just taking the virtual console) so that things like Control-A/B would work.

You'll need a little physical cabling help to make this work. Even though those desktop PCs are old, most of them have not had serial ports in many years. Think about it. It's all USB by now. So you buy a USB-to-serial converter. You'll need a copy of an HP 3000 terminal emulator on the PC configured to connect via the serial port.

Just make sure the PC stays up and the emulator's window is open and connected. You don't need the console buffer filling up.

Posted by Ron Seybold at 04:31 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 01, 2015

Message Files Editing, DLT Autoloading Tips

What tools can I use to 'edit' a message file without destroying the file? I learned the hard way that if you’re using FCOPY to copy from a message file, it destroys the records read from the message file. Can you "COPY" a message file without destroying the records that are copied? 

François Desrochers says

You can use non-destructive reads by specifying the COPY option on a file equation as in:


There's also an FOPEN Aoptions (bit 3:1) to enable the option.

As for editing a message file, the only safe way I know is to extract all the records into a regular flat file, edit it and load it back into the message file.

If it's easy to identify which records to remove, you can use FCOPY.


Extract records with "Y" in column 1. MSG1 is empty at the end of the task. To use non-destructive reads, do: 


MSG1 still has all the original records. 

Use FCOPY's EXCLUDE option to extract all records except the ones with a "Y" in column 1 as in:


I have a HP Q1567A DDS5 auto loader installed on a 987. I installed and configured it on the SE port on the multifunction IO card. I can store files to it fine, but I can't get it to advance each day to the next tape. How do I do this?

Ryan Melander replies

Does it auto-eject the drive when the backup finishes? The reason I ask is I have the same configuration on my account with a DLT7000 autoloader 8/Slot. I have it set up to auto eject after each backup, moving the tape back to its original position. It then auto inserts and mounts the next tape for the next backup time.

Jack Connor notes

I generally set the DLT autoloaders for no circular, stacker mode. This goes through all the tapes in the unit and then stops.

Jim Hawkins says

DEVCTRL.MPEXL.TELESUP should work to set up the eject option. Put it inside your job to be sure that it is set correctly every time.

Gilles Schipper adds,



where the "7" is the LDEV number of the tape drive.

Posted by Ron Seybold at 04:02 PM in Hidden Value, Homesteading | Permalink | Comments (0)

April 27, 2015

Sendmail, Group Purges, and ACD Removal

Is there a proper or "right" way to shut down sendmail?

Donna Hofmeister replies

• Use the Posix kill signal from SERVER.SENDMAIL or any user with SM capability. (The following can be easily turned into a job!)

kill $(head -n 1 /etc/mail/

• Only use :ABORTJOB as a last resort! (This is true for all of the Posix things that got ported to MPE)

If you don't need to run a mail server (e.g. sendmail) on your 3000, you shouldn't. In most cases, using a mail client will be "just the ticket." Point the client at your in-house (SMTP) mail server and enjoy.

How can I easily purge all the files in a group without destroying the group structure?

If GRPNAME is the name of the group then either:

1. chgroup GRPNAME and

purgegroup GRPNAME 

Or, purge @.GRPNAME 

[Ed. note: Vladimir Volokh notes this last command does not purge databases from a group, although it purges everything else. You must be an SM user to purge everything in one account from another account. Of course, MPEX's %purge will purge everything, and will report the list of what is to be purged. %purge(ISPRIV) for a selection of databases only, for example.]

How can I convert an SL to an XL?

Jeff Kell and Gavin Scott reply:

You can OCTCOMP an SL, which will make the code in it run in mostly-Native Mode (though using several times the memory) and with exactly the same limitations as the original CM code. OCTCOMP just adds a pre-translated version of the CM code to the end of the file that will be invoked when you run the program or SL on an MPE/iX system.

We have problems accessing the files of one group because someone has added ACD's on all the files within that group. Is there a way to remove them all easily?

Mark Ranft replies

Using MPEX, you can issue the command


Keven Miller adds

And if you don't have MPEX, you can use a command file like the one below.



This command file takes 2 parameters. 1. a  fileset (wildcards allowed) 2. an MPE command with a ! exclamation as place holder for the filename It does a LISTF ,6 of your fileset into a MSG file. Then it reads through the MSG file and processes each file with your command.

Posted by Ron Seybold at 11:18 AM in Hidden Value, Homesteading | Permalink | Comments (0)

April 24, 2015

Solutions for Keeping Passwords Fresh

Our management wants our 3000 users to be forced to change their password on a regular basis. Also, certain rules must be applied to the new password. We don’t have VEsoft’s Security/3000, although we do have MPEX. I therefore have two options. 1. Write something myself, or 2. See if there is anything in the Contributed Software Library that will do the job, or can be modified to supply the required solution.

Homegrown and bundled solutions follow. Jeff Vance offered this:

There is a pseudo random password generator available among the Jazz files which knows MPE’s password rules. See RANDNAME. There are also UDCs which force a password to be supplied when using NEWUSER, NEWACCT and NEWGROUP CI commands. These required passwords can be random or user entered with a minimal length enforced. 

Then he added as an afterthought, a strategy to program your own password system:

I haven’t thought about it much, but it seems you could have a password file (maybe a CIRcular file?) for each user on the system. This file would have their last N passwords, and the modified date of the file would be the date their password was most recently changed.

A logon UDC could detect if the password file for that user exists. If not create it and require a new password right then.  If the password file exists then get it’s modified date and compare that to today’s date. If greater than X days then in a loop prompt for a new password. Validate the entered password against previous N passwords and your other rules. Maybe run a dictionary checking program to make sure the password is not common, etc.

Update the user-specific password file with their new password, and then logon the user.

From the support community, Donna Hofmeister weighed in with this advice:

The solution that your management demands is going to cost more for you to develop, implement, or maintain than it’ll take for you to get Security/3000. If you have no choice other than to develop a product, then I’d certainly model it after what VEsoft has already done. That is:

Based on a system-wide UDC, examine all sessions (it is just sessions, yes? By the way, a DSLOGON from inside a job is still a session.) against a ‘database’ (By the way, just how secure is this database?  A real database needs passwords... Who’s going to maintain that? A flat file could be lock-worded, but that’s not a slam-dunk answer.) a database which is looking for the ‘age’ of the password (By the way, are you going to provide an advance warning period?). 

If it is time to change the password, get the ‘new’ password from the user... but writing the rules is a pain, and keeping track of reused passwords is just annoying. Auditors in the states love when you can say the password is one-way encrypted. Dunno what your management is saying for encrypting an MPE password.

Then came a solution rolled up by Paul Christidis

Some years ago I had developed a set of command files that could be used  to require users to have passwords. Later on, mostly as an exercise, I  enhanced the process to age passwords and to automatically assign ‘random’  passwords as they expire.  The random passwords are comprised of  alternating consonants and vowels, they can have a minimum and maximum  length and optionally a random digit can be inserted.

The entire ‘process’ is comprised of a system batch job (should be running  always), a command file that is invoked by a log-on UDC and communicates  with the batch job, a ‘control’ command file that starts and stops the  batch job, a command file to determine the password age and a command file  to generate the random password. Below are the comments from the batch job. They explain some of the details.

!# Author:  Paul H. Christidis
!# Remarks:   This job 'listens' at a message file for any requests to
!#    determine if a user has a password.  Once that determination is
!#    made it passes back to the session an indicator to that effect.
!#    A command of STOP causes the job to terminate.
!#      The request comes via the execution of a command file or a System
!#    wide UDC and it is comprised by the file name where the reply should
!#    be placed and the user's name and account.
!#      This job does NOT return the user's password, it only writes in 
!#    message file specified by the client the command:
!#          setvar user_password true/false
!#      The client then executes the command and tests the setting of the
!#    variable 'user_password' to decide what action to take.
!#   The job logic was changed to assign a new password after 30 days.  A
!# file in the posix space is built using the user's name.  Then at each
!# logon a command file is used to determine the file's age, using its 
!# date, and when it is older than 30 days another command file is used to
!# generate a random password.  Said password is sent back to the session
!# and the user is informed about his new password.
!#   If the 'job/session' name is not to be used in creating the posix 
!# that will be used to age the password, then the value of the CI 
!# 'pw_UseSess' should be set to 'FALSE'.
!# DATE:    06/10/2004   [Added alternative aging values functionality]
!#   Alternative aging limits are kept in an ASCII file "pswrdage" 
!# of 'setvar' commands, for each MPE account, MPE user or Session name. 
!# should adhere to the following format:
!#       SETVAR SYS_MANAGER_XTIDIS_pwage   45
!#       SETVAR SYS_MANAGER_pwage          40
!#       SETVAR SYS_pwage                  35
!#  The above have the following implications:
!#    The user's "xtidis,manager.sys" password expires in 45 days
!#    The user's "manager.sys" password expires in 40 days
!#    Any other user of the "sys" account has their password expire in 35 
!#    While the 'default' 30 days applies to every other user on the 
!# NOTE 1: Suffix of '_pwage' is required.
!# NOTE 2: A negative or zero setting equals to NO password aging.
!# NOTE 3: The order MUST be 'ActName_UserName_JobName_pwage'.
!# Date:    06/11/2004   [Added code to 'force' a password change]
!#   When the CI variable "ForcePwChange" is set to TRUE in the session 
!# executes the command file, the 'passed' code is changed and the batch 
!# forces the password change (Unless it was already changed on the same 
!# <---------------------------------------------------->

Dave Powell coded up some of the fine print nicely in his contribution:

It ought to be possible to do everything for free, using just MPE.  Editing the new password is the ugly part, but if you randomly assign it that issue goes away. The next issue is that the :password command seems to like to be purely interactive, as in:

echo oldpass >> tempf
echo newpass >> tempf
echo newpass >> tempf
:password < tempf
Command not allowed in noninteractive environment. (CIERR 2500)

That leaves the altuser cmd, which needs AM cap.  If you don't want a background job (like Paul's suggestion), you can have the command file (called by  your logon UDC) use the echo command to build job with AM cap, which it then streams, kind of like (untested, but I have working examples of cmd-files building other jobs):

ECHO !!TELL !!STREAMED_BY Your new password is !NEW_PASS >> TF
ECHO Please note your new password when it appears

If I haven't screwed up the fine print too badly, this code in the middle of the password cmd-file runs the job that changes your password, then waits for the job to tell you what the new password is.  The single exclamations before HPUSER & NEW_PASS mean that values that are variables to the session and command-file become hard-wired values for the job.

Before all this your cmd-file checks the date, gets a random password, etc., as posted by others.  After it the cmd-file writes (or just builds) a file that serves as a timestamp.  But I am not too comfy with putting the passwords into a plain-text file, so I might skip that part (remember, the user needs both read and write access to it).  Put the command file in a group that users have xeq access to, but not read/write access.

Posted by Ron Seybold at 08:25 PM in Hidden Value, Homesteading | Permalink | Comments (0)

April 13, 2015

How MPE Talks to Its Network Neighbors

Our networking team reports they're going to refresh the hardware on our IP gateways. Our Telecom manager says they will 

  • Change the physical gateway, because the hardware is being replaced
  • Not change the IP address and gateway address
  • Change the MAC address of the gateway (because of different gateway hardware)

Network NeighborhoodWhat do I need to do on our MPE boxes to ensure that they will see the new hardware? Does MPE cache the MAC address of neighbor gateways anywhere? I was thinking I needed to restart networking services, but I wasn't sure if anything more will be needed.

Jack Connor replies

If you're taking it off the air for the network changes, I'd go ahead and close the network down until the work has completed and then reopen it. MPE will be looking for the IPs as it opens up. I know you can see the MAC addresses in NETTOOL, but I don't think they're of any import other than informational and for DTC traffic.

Donna Hofmeister adds

Halt the network (even the system if possible -- because it's almost the same thing) while the larger network work is being done. When the new gear is in place and seems stable, "wake up" the 3000 and watch what happens.

When you halt the network (presuming you're not taking the box down) be sure to halt/quiesce network-dependent things (like jobs/listeners) just prior. I'd suggest doing an 'openq' on your network printers as well (keep the input side of printing open, but not the output side).

Jeff Kell notes

If you have access to your routers/switches, you can also attack this from the other end. 

Cisco switches/routers (layer-3) have an ARP table that holds the MAC addresses of the hosts on the subnet. They are subject to a timeout. However, they will generate a "gratuitous unicast ARP" request to any host in the table 30 seconds before it is scheduled to expire. If the host is up, it will respond, and reset the learning timer. It will ALSO push the MAC address and IP of the gateway (the router that just performed the unicast ARP) into the host.

Unfortunately the default ARP timeout on a Cisco device is 4 hours.

You may tweak this as desired on a per-interface basis via the "arp timeout xxx" configuration directive.

If you are doing network monitoring by any SNMP tools, they work best when the mac-address tables (CAMs/TCAMs/etc) are loaded with the host values, and the ARP tables are populated with the current hosts. The mac-address table timeout default is only 300 seconds (I'd suggest moving that up, depending on how volatile your connections may be; we use 600 seconds).  This will cause the "gratuitous ARP" to occur at 530 seconds, and if the host answers, it perfectly repopulates the tables for your management tools to read.

Howard Hoxie adds (with a prompt from Gilles Schipper)

MPE caches the MAC for the gateway, and that there is a command that updates with an "INTERNET=@" or "INTERNET=ALL" clause.

(The command is NETCONTROL net=netname; update=all, where netname is probably LAN or LAN1)

Posted by Ron Seybold at 06:08 PM in Hidden Value, Homesteading | Permalink | Comments (0)

April 03, 2015

That final 3000 IO upgrade is still in use

Pass ThroughMore than five years after HP rolled out the ultimate release of MPE/iX, the vendor finished its work on an SCSI Pass-Through driver for the HP 3000. It was an one of the last HP-designed MPE enhancements. Independent support companies have the tech resources to create customized patches for their customers. The HP driver still makes it possible to connect and configure SCSI storage devices which HP has not certified for 3000 use.

Full instructions on how to use the software are on the ManualShelf free website. It's a tool for permitting an application to address SCSI devices without the use of the MPE/iX file system or high-level IO interfaces. But the software itself was built, lab-tested, then placed on the HP software improvements leash: It was only available to the HP support customer who was willing to take SPT, as HP called it, as a beta test version.

Patches MPENX01A, MPENX03A and MPENX04A were beta patches required to make the SPT work on MPE/iX 7.5. HP still makes these patches available to any 3000 customers at no charge. Two years ago, Allegro's Donna Hofmeister said "the magic incantation when dealing with the Response Center folks is to use transfer code 798. That’ll get you to an MPE person." 

Consultants and companies which provide support have many of these patches in their resource bins. The entire patch collection is just 1.27GB, small enough to fit onto a giveaway thumb drive.

HP always described the SPT as a tool with many caveats. While the software has a wide scope of help for the customer who wants to add the latest SCSI disk to the 3000 in the future, HP has printed plenty of cautions such as these in its Communicator article:

Sending SPT commands to a device in use by MPE or other applications may result in data loss, data corruption and/or System Aborts. We do not recommend sending SPT commands to Disks with MPE/iX Volumes present nor should one access tapes devices which are used for normal back-up or data logging purposes. Rule of thumb: Don’t do SPT to tape or disk media you cannot recover at a time you don’t wish to cause a system outage.

In 2007, SPT joined the six dozen or so HP developed enhancements and fixes stalled in beta test status. Customers still purchasing HP support were expected to test enough to get SPT into the full community as a general released patch, but few wanted to use beta-grade software in production.

Posted by Ron Seybold at 06:40 PM in Hidden Value, Homesteading | Permalink | Comments (0)

April 01, 2015

File transfer tips flow to move databases

System managers in the 3000 community still want to know how to use FTP to ensure a safe backup of 3000 data. Of special interest is the KSAM XL database, but most managers don't know that FAK files are HP's special Keyed Sequential Access Method database files. What appears to be program files are moved over, but database files get left behind. There's a trick to getting such files over to a Windows server.

One rule of 3000 operations is that database files act differently than all others in transfers. So FTPing them to a Windows 2003 Server won't be a successful way to ensure a safe data recovery. Third party tools can help, but if a customer is stuck on an aging HP system running MPE/IX, it's probably going to have only the budget for the included HP STORE for file backups and transfers.

Donna Hofmeister, who's spent a career helping 3000 users via the community's newsgroup, suggests starting with creating a file called mystd to store the 3000 files to disk -- then transferring that Store To Disk file.

 "First try this little experiment," Hofmeister begins in her advice.

    file mystd;dev=disc

If it works, you just stored/archived all the files that begin with 'a' in the .pub group of the .sys account into a file called 'mystd' (my store-to-disc).  You can expand the number of files being stored into your STD file by modifying your store command to: -- all files in .pub.sys
    @.@.sys   -- all files in .sys
    @.@.acct  -- all files in .acct (for example)
    @.@.@     -- all files on the system (and it's actually better to say '/' instead of '@.@.@')

Keep in mind that as you increase the scope of what you're storing, so does the size of your STD file. In other words, to store the whole system you need 50 percent or more free space , which you probably don't have. So, break what you're storing into chunks (do one account at a time) and things should go smoothly.

If STD doesn't work, you might be able to get tar to work. The same space precautions apply.  One advantage of using tar is you should be able to verify the tar file on the destination system -- something you can't do with STORE without a 3000 in the mix.

3K Associates' Chris Bartram, who operates the 3000 information treasure-house, added explicit advice about FTP and the 3000's files.

If you package all the MPE files up in either a store-to-disk (aka std) or a tar "wrapper" (disc file) you can transport that file around at will -- as a BINARY file -- don't try to transfer it as ASCII or CR/LF translations will trash it.

Once you get it back on the 3000, a simple file equation directing your source (tape) to the new (std) file name (and add the ;DEV=DISC to the file equation) will allow you to restore the files onto the 3000 preserving all the MPE specific file attributes they started with. Tar will work similarly for almost all MPE files, but can't handle database/PRIV files and probably not MSG (message) files and a few other very MPE-specific files.

tar works the same way on MPE as on *nix boxes. But is much more "familiar" if you run it from the Posix shell (sh.hpbin.sys), though that's not necessary. Treat the tar file just like you would on a *nix box. 

For store-to-disc files, you use the same MPE syntax for storing files as you do normally; the only difference is that the output device is file-equated to ;dev=disc. As mentioned, be aware of the disc space required to store another copy of your backed-up files online. 

Likewise, when you restore instead of pointing to tape, you point to the disc file -- and don't forget to add ;dev=disc to that file equation as well. If the store-to-disc files are going to be very large (several gigabytes) you can use some additional syntax to break them into chunks - but hopefully you needn't worry about that for now. 

Treat a store-to-disc file just like a tar file. Record size and most other attributes aren't so critical, but if you move it around do NOT let FTP transfer it in ASCII mode or it will corrupt the file.

As for examples; I back up my primary HP 3000 to a disc file then transfer (FTP) it to a Linux server. Here's the gist of my JCL:

!STORE / - /BACKUP/ ;*T &
! ;SHOW=SHORT,dates,path;progress=5;directory;tree;&

open ftpserver
user userid password
del hp3000-full
put FULLB.PUB.BACKUP hp3000-full

Posted by Ron Seybold at 05:59 PM in Hidden Value | Permalink | Comments (0)

March 26, 2015

Checkup Tips to Diagnose Creeping Crud

When an HP 3000 of the ultimate generation developed trouble for Tom Hula, he turned to the 3000 newsgroup for advice. He'd gotten his system back up and serving its still-crucial application to users. But even after a restart, with the server looking better, things just didn't seem right to him. 

I am concerned, since I don't know what the problem was. It almost reminded me of something I used to call the Creeping Crud, where people started freezing up all over the place, while some people were still able to work. The only thing was a reboot. But in this case, it seemed worse. Only a few people on our 3000 now, but we still depend on it for a high-profile application. What should I check?

CrudThe most revealing advice came from Craig Lalley, who told Hula he'd try a Control-B into the 3000's system log. The steps after the Control-B command are SL (for System Log) and E (for Errors only.) Typing CO puts the 3000 back in console mode. Hula's system had lost its date and time on one error, and the Alert Levels showed a software failure along with lost boot functionality.

But amid the specifics of eliminating the Creeping Crud (it may have been a dead battery) came sound advice on how to prepare for a total failure and where to look for answers to 3000 hardware problems. The good news on the battery is that it's not in a Series 9x7. Advice from five years ago on battery replacement pointed to a hobbyist-grade workbench repair. More modern systems like Hula's A400 at least have newer batteries.

Using a DSTAT ALL was suggested, as well as checking the status of available storage with DISCFREE. Mark Ranft said "I would make sure there's a good full backup. (Just in case you need it for recovery.)  If you don't have one, doing one may help identify a disk issue. I would check the system logs especially for disk errors. I would check for network errors, using linkcontrol @,all." He shared his own recovery experience.

I had a system acting strangely this past weekend. It was basically hung but allowed new logons. I could not abort anyone. When I got to the point where I tried to stop the network, I got a system abort 1458 from Subsystem 102. I didn't bother to take a dump. I completed the boot and everything was better.

Chuck Trites reminded Hula to create a current CSLT tape, and "run BULDJOB to create the BULDJOB1 and 2 files — in case you need to recreate the accounting structure and UDCs — and store them to tape."

Hula's own check list included the following:

During the reset, the 3000 got up to the date and a little past and seemed frozen. I pulled the plug and restarted again. It took 2-3 times as long as normal and at first, the red fault light was on (I never saw that on before). After it got a bit into the restart, the fault light turned off by itself. The only attention message I got about the whole thing was a message with everything unknown on the 3000.

When the computer came all the way up, it still seemed very sluggish. I scheduled the nightly update and backup and went home to look at it more in the morning. I logged on from home and the backup seemed to be running okay.

This morning I tried resetting the GSP and checked the connections to the console terminal. I also found out that someone else had a hard time getting on the 3000 towards the end of the day. Very sluggish. But this morning, everything seems back to normal.

Hewlett-Packard's hardware builds have been extraordinary, but a server that's been churning out critical data for more than 12 years (A-Class boxes production stopped in 2003) can develop crud. Something as simple as replacing a dead battery might be the answer to the woes. Advice for the crud also came from Gilles Schipper, Jack Connor and the others mentioned. What they've got in common is working in a support practice, or at least a consulting business that includes 3000 sites.

Self-maintenance is common in a community like the 3000's. It's also a good practice to have a support vendor, one who knows the system as well as the volunteers posting to the newsgroup.

Posted by Ron Seybold at 09:20 PM in Hidden Value, Homesteading, User Reports | Permalink | Comments (0)

March 17, 2015

Tips to Reinstall Posix, DLT/LTO Tape Drives

What is the patch that installs Posix? I seem to have a corrupt version of Posix.

Donna Garverick of Allegro replies:

These are your instructions for MPE/iX 5.5 and 6.0.

Load the 5.5 or 6.0 FOS tape on a tape drive. For this example, tape drive on ldev# 7 is used. Log on as MANAGER.SYS


Please note:

  • HP36431 is the master product number of the Posix 2 Shell.
  • I0036431.USL.SYS is the installation file.
  • When launched, the job I0036431 should run for less than 5 minutes. When it is done, the Posix environment is re-installed.

[Gilles Schipper notes the process for 7.5 is the same, working from the MPE/iX 7.5 FOS tape.]

I have access to a Tandberg Data Ultrium LTO 3 tape drive. It has a SCSI Ultra160 interface. Would I have any luck hooking one up to an N-Class?

Chad Lester of MPE Support Group replies:

It's worth trying. You might have issues with the dual-port SCSI cards. Also, make sure the firmware is the latest on the single SCSI U160 card.

We recently upgraded our customers to the hot-swappable LTO drives designed for the TA-5300 Array. The array is $350 with an SCSI Cable. Two Q1540A LTO 3s are $1,350, for a grand total of $1,700. That includes phone support from us for installation. 

I have a DLT4000 that will connect to an HP 3000 on path 32.2.0. How do I set the 12 dip switches on the back of the DLT for this path?

Mark Ranft of Pro3K replies:

Off of Google, I’ve found this:

For HP 7980S emulation, 

     1,2 - OFF

     3,4,5 - ON

     6,7,8,9 - OFF

     10,11,12 - SCSI ID (suggest all OFF)

So 10 is the 4's place
And 11 is the 2's place
And 12 is the 1's place

If 10, 11 and 12 are off (down), the SCSI ID will be zero.

Gilles Schipper of GSA adds:

I believe there are two different versions of DLT4000. One has a single-ended SCSI interface and the other has a FW-SCSI interface. I think the one Mark described is for the former.

Posted by Ron Seybold at 05:18 PM in Hidden Value, Homesteading | Permalink | Comments (0)

February 16, 2015

Classic MPE tips: Tar, kills, and job advice

How do I use the tar utility to put data onto tape on an HP 3000?

1) Create a tape node

:MKNOD “/dev/tape c 0 7”

2) Enter posix shell

:SH -L

3) Mount a blank tape and enter the tar command

shell/ix>tar -cvf /dev/tape /ACCOUNT/GROUP/FILENAME

How can I determine the validity of an SLT tape?


What is the command to abort a hung session? I tried ABORTJOB #s3456. I seem to remember there is a command that will do more.

You can use =SHUTDOWN. But seriously, there is a chance that if it is a network connection, NSCONTROL KILLSESS=#S3456 will work. If it is a serial DTC connection, ABORTIO on the LDEV should work. Finally, depending upon what level of the OS you are on, look into the ABORTPROC command. This might help as a last resort.

I recently had a perfect application for the NEWJOBQ feature. We have two groups of users. One group submits jobs that take about 30 seconds each. Typical jobs for the other group take about 5 minutes each. So I thought I’d give the second group of users their own job queue.


When I submit a long job into the ALTJOBQ queue, and a quick job into the default job queue, the second job goes into the WAIT state. Why?

Your NEWJOBQ statement is correct, but your second statement didn’t do what you thought. To put a limit of one on the HPSYSJQ job queue, your statement should read 

By saying :LIMIT 1, you are changing the total job limit on the system to one. Since the total limit is one, and the long job in ALTJOBQ is still running, the second job waits even though he is the only job in his queue.

What does HPSWINFO.PUB.SYS show? All software or only installed software? How do I find out what HP software is installed?

Generally speaking, HPSWINFO.PUB.SYS is a record of system software level and patching activity. If you want information on HP software installed then you want to run psswinvp.

How can I sync the time on my 3000 with my Windows network? The PC side does regular, automated sync to NIST.

First, ensure your timezone is absolutely correct (:setclock/:showclock) and you have a system logon UDC to setvar TZ to the correct timezone.

Install NTP and use the ‘ntpdate’ function to sync your clock to the PC servers. Do this in a batch job that issues the ntpdate command, and then :STREAMs itself;IN=xx to periodically perform the synchronization.

Posted by Ron Seybold at 11:15 PM in Hidden Value, Homesteading | Permalink | Comments (0)

February 10, 2015

Multiple Parallel Tapes on 3000 Backups

Editor's note: When I saw a request this week for a copy of HP patch MPEMX85A (a patch to STORE that enables Store-To-Disk) for older MPE/iX releases, it brought a storage procedure request to mind.

I'm dealing with some MPE storage processes and need assistance. You would think after storing files on tapes after 10-plus years, we would have found a better way to do this. We use TurboStore with four tape drives and need to find a way to validate the backup. Vstore appears to only have the ability to use one tape drive. Currently I have some empty files scattered through the system and use a separate job to delete them, remount the tapes and restore, trying to access all four drives. 

When using vestore:

vstore [vstorefile] [;filesetlist]

It seems that vstorefile is looking for a file equation similar to:

File t; dev=tape
vstore *t;@.@.@; show

This is why it appears that I can't use more than one tape drive, unless they are in serial, while we want to use four drives in parallel. What method or software should I be using?

Mark Ranft of Pro3K replies:

We always found that DLT 8000 tapes worked well in parallel. When the backup got so big that it wouldn't fit on two DLT 8000 tapes, we split the backup, putting the databases on two tapes in parallel and everything else on a third tape. Keep in mind, we didn't have a backup strategy. We had a recovery strategy and backups were a part of that. We found, for us, organizing backups in this manner allowed us to speed recovery — which was far more important than anything else.

You can achieve good times doing Store-to-Disk backups. But then what? Do you back up the STD to tape and send it offsite? FTP it somewhere? The recovery times on getting this back are too slow.

Tracy Johnson adds

I think you can use VSTORE to read multiple tape drives in parallel or series using the ;RESTORESET parameter.

So you make four file equations.

Drop the beginning file single backreference to a equation (like we learned in olden times), and put the four new ones with the ;RESTORESET= parameter instead. It is one of those things that fooled me first time I saw it, and it took about 10 minutes getting used to seeing it.

The parenthesis around the file equations are placed differently:


 ;RESTORESET = (*tape1,*tape2,*tape3,*tape4)



But if the tapes were not also created in parallel, it may not help in the latter case.

Ray Legault adds

I use three DLT8000's and run a Vstore every week.

! setvar _drive "(*p1),(*p2),(*p3)"
!vstore ;@.@.@;restoreset=!_drive;show;progress=5;nodecompress

Posted by Ron Seybold at 09:35 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 26, 2015

How to Use MPE/iX Byte Stream Files

Back when HP still had a lab for the HP 3000, its engineers helped the community. In those days, system architect and former community liaison Craig Fairchild explained how to use byte stream files on the 3000. Thanks to the memory of the Web, his advice remains long after the lab has gone dark.

Mountain-streamThese fundamental files are a lot like those used in Windows and Linux and Unix, Fairchild said. HP has engineered "emulation type managers" into MPE/iX, an addition that became important once the 3000 gained an understanding of Posix. In 1994, MPE/XL became MPE/iX when HP added this Unix-style namespace.

Understanding the 3000 at this level can be important to the customer who wants independent support companies to take on uptime responsibility and integration of systems. Fairchild explained the basics of this basic file type.

Byte stream files are the most basic of all file types. They are simply a collection of bytes of data without any structure placed on them by the file system. This is the standard file model that is used in every Unix, Linux and even Windows systems.
MPE's file system has always been a structured file system, which means that the file system maintains a certain organization to the data stored in a file. The MPE file system understands things like logical records, and depending on the file type, performs interesting actions on the data (for example, Circular files, Message files, KSAM files and so on).

Fairchild detailed how HP has given bytestream files the knowledge of "organization of data" for applications.

To bridge the gap between standard byte stream file behavior (only the application knows the organization of data) and traditional MPE file type behavior (the file system knows what data belongs to what records), emulation type managers were created. To an MPE application, a byte stream file looks and behaves like a variable record file, even though the data is stored in a way that would allow any Posix application to also read the same data. (Posix applications also have emulator type managers that allow them to read fixed, variable and spool files in addition to plain byte stream files.)

The way that the byte stream emulator detects record boundaries is through the use of the newline (\n) character, which is used, by convention, to separate data in ASCII text files on Unix-based systems.

The underlying properties of a byte stream file are that each byte is considered its own record. In MPE file system terms, a record is the smallest unit of IO that can be performed on a file. (You can write a partial record fixed length record, but the file system will pad it to a full record.) Since the smallest unit of IO that can be performed on a byte stream file is a single byte, that becomes its MPE record size.
In the MPE file system, the EOF tracks the number of records that are in a file. Since the record size of a byte stream file is one byte, the EOF of a byte stream file is also equal to the number of bytes in the file. This is why one 4-byte variable sized record is equal to 5 byte stream records (4 bytes of data + 1 \n character).

It's also worth noting that any file can be in any directory location and will behave the same way. (Well, almost. CM KSAM files are restricted to the MPE namespace. And of course the special files (that you don't normally see) that make up the file system root, accounts and groups are also restricted: one root, accounts as children of the root, groups as children of accounts. And lockwords aren't allowed outside the MPE namespace. But other than that, the opening sentence is true.) 

The general model that we had in architecting the whole Posix addition was that behavior of a file does change regardless of where it is located. This was summed up in the saying, "A file is a file." So there are no such things as "MPE files" and "Posix files". There's just files.

What does change is the way you name that file. Files in the MPE namespace can be named either through the MPE syntax (FILE.GROUP.ACCOUNT), or through the HFS syntax (/ACCOUNT/GROUP/FILE). You can also use symbolic links to create alternate names to the same file. This was summed up as a corollary to the first saying, "But a name is not a name."

Posted by Ron Seybold at 09:47 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 06, 2015

Essential Steps for Volume Reloads

When a 3000 drive goes dead, especially after a power outage, it often has to be reloaded. For example, when an LDEV2 has to be replaced. For a cheat sheet on reloading a volume, we turned to our Homesteading Editor Gilles Schipper.

By Gilles Schipper

Assuming your backup includes the ;directory option, as well as the SLT:

1. Boot from alternate path and choose INSTALL (assuming alternate path is your tape drive) 
2. After INSTALL completes, boot from primary path and perform START NORECOVERY. 
3. Use VOLUTIL to add ldev 2 to MPEXL_SYSTEM_VOLUME_SET. 
4. Restore directory from backup (:restore *t;;directory) 
5. openq lp
6. Perform a full restore with the following commands
:file t;dev=7(?)
:restore *t;/;keep;show=offline;olddate;create;partdb;progress=5 7.


I would suggest setting permanent and transient space each equal to 100 percent on ldev 2. The 75 percent default on ldev 1 is fine as long as you don’t need the space. And if you did, your solution shouldn’t really be trying to squeeze the little extra you’d get by increasing the default maximum limits.

The reason for limiting ldev1 to 75 percent is to minimize the otherwise already heavy traffic on ldev 1, since the system directory must reside there, as well as many other high traffic “system” files.

You won't want to omit the ;CREATE and ;PARTDB options from the restore command. Doing so will certainly get the job done -- but perhaps not to your satisfaction. If any file that exists on your backup was created by a user that no longer exists, that file (or files) will NOT be restored.

Similarly, if you omit the ;PARTDB option, any file that comprises a TurboIMAGE database whose corresponding root file does not exist, will also not be restored.

I suppose it may be a matter of personal preference, but I would rather have all files that existed on my disks prior to disk crash also exist after the post disk-crash RELOAD. I could then easily choose to re-delete the users that created those files -- as well as the files themselves.

Another reason why the ;SHOW=OFFLINE option is used is so that one can quickly see the users that were re-created as the result of the ;CREATE option. Purging the "orphan" datasets would be slightly more difficult, since they don’t so easily stand out on the stdlist.

Finally, it’s critical that a second START NORECOVERY be performed. Otherwise, you cannot successfully start up your network.

Posted by Ron Seybold at 10:37 PM in Hidden Value, Homesteading | Permalink | Comments (0)

December 10, 2014

Getting Macro Help With COBOL II

GnuCOBOL An experienced 3000 developer and manager asked his cohorts about the COBOL II macro preprocessor. There's an alternative to this very-MPE feature: "COPY...REPLACING and REPLACE statements. Which would you choose and why?"

Scott Gates: COPY...REPLACING because I understand it better.  But the Macro preprocessor has its supporters. Personally, I prefer the older "cut and paste" method using a decent programmer's editor to replace the text I need. Makes things more readable.

Donna Hofmeister: I'm not sure I'm qualified to comment on this any longer, but it seems to me that macros were very efficient (and as I recall) very flexible (depending on how they were written, of course). It also seems to me that the "power of macros" made porting challenging. So if your hidden agenda involves porting, then I think you'd want to do the copy thing.

There was even porting advice from a developer who no longer works with a 3000, post-migration.

Tony Summers: When we migrated in 2008 we chose Acucobol partly because of its HP 3000 compatibility, including macro support. However had we gone down a different route, I had already proved that I could pre-process the raw code myself and expand the macros before calling the compiler.

Robert Mills, who started the discussion on the 3000-L, said in reply to Donna, “I admit that I do have a hidden agenda, but the main reason does not involve porting.”

For many years I have used macros to make my life easier. When I left the e3000, back in 2008, and did some work on other platforms I found I missed them. I'm now in semi-retirement and have been using the free version of Micro Focus COBOL (a couple of years) and GnuCOBOL (this last year) to write software for friends, family and my own use.

A couple of times since 2008 I had thought of writing by own macro preprocessor to emulate the one on the e3000. A few months ago I decided to do it and release it as open source under the GNU GPL. The development of preprocessor, using GnuCOBOL, is now completed and in final Beta Testing and I'm writing the manual. Was hoping that I could some additional reasons, from others, as to why you would use macros instead of the copy...replacing and replace statements.

Because a port of GnuCOBOL is a available on several platforms, and my preprocessor is written in GnuCOBOL, I see no problem in taking my macros with me nearly every wherever I go. If I end up doing work on a platform that does not support a feature that it is using it shouldn't be to difficult to develop a workaround.

As it turns out, GnuCOBOL is a newer version of OpenCOBOL — a compiler that Donna says bears a close resemblence to COBOL II. (OpenCOBOL has been ported into a commercial product, too, called IT-COBOL.) Adding that she obviously thinks macros are cool, she explained.

Do my mis-firing neurons recall that GnuCOBOL was formerly OpenCobol... which was actually very close to MPE’s COBOL?  (or something like that?)

I inherited a outstanding collection of macros at one job. Many of them were 'toolbox' functions.  Want to center a string and the overall length of the string doesn't matter?  Got a macro for that.  Want to use a 'db' call? Got a macro for that.  These went far beyond modifying code at compile time -- and that's what made them so valuable (at least to me).

Posted by Ron Seybold at 06:10 PM in Hidden Value, Homesteading, User Reports | Permalink | Comments (0)

December 08, 2014

IMAGE data schemas get visualized

Is there any program that will show the network of a TurboIMAGE database? I want to output the relationships among sets and items.

CFAWireframeIn 2011, Connie Sellitto researched the above question, while aiding new programmers who were charged with moving a pet organization's operations to a non-MPE system. Understanding the design of the database was important to this team. Sellitto mentioned a popular tool for PCs, but one not as essential as an IT pro's explanations.

You might try Microsoft's Visio, and you may need to have an ODBC connection to your IMAGE database as well. This produces a graphical view with search paths shown, and so on. However, there is still nothing like a detailed verbal description provided by someone who actually knows the interaction between datasets.

To sum up, we can refer to ScreenJet founder's Alan Yeo's testing of that Visio-IMAGE interplay

Taking a reasonably well-formed database into Visio and reverse engineering, you do get the tables and items. It will show you what the indexes in the tables are, but as far as I can see it doesn't show that a detail is linked to a particular master. Automasters are missing anyway, as they are really only for IMAGE.

My conclusion: if you have done all the work to load the databases in the SQL/DBE and done all the data type mappings, then importing in Visio might be a reasonable start to documenting the databases, as all you would have to do is add the linkages between the sets.

If you don't have everything in the SQL/DBE, then I would say we are back where we started.

ScreenJet knows quite a bit about moving 3000 engineering into new formats. It built the EZ View modernization kit for 3000 user screens that are still in VPlus. Yeo said the ubiquitous Visio might be overkill for explaining relationships.

If you have Adager, Flexibase, or DBGeneral -- or already have a good schema file for the databases -- just generate the schema files and import them into Word or Excel and give them to [your migrators]. If they can't put together the data structure from that, no amount of time you can spend with Visio is going to impart any more information.

Visio has free and open source competition, software which HP support veteran Lars Appel pointed out. "Perhaps Visio has similar 'database graph' features, such as the free or open source tools like dbVisualizer or SquirrelSQL."

Barry Lake of Allegro pointed out that users "may want to take a look at Allegro's DBHTML product, which creates a browser viewable HTML file documenting the structure of an IMAGE database." Allegro's site has an example DBHTML output on its website, although it doesn't draw pretty pictures.

At a more fundamental OS level, Michael Anderson points out to understand the structure of a TurboIMAGE database, "you could use QUERY.PUB.SYS, then issue the command FO ALL, or FO SETS."

Posted by Ron Seybold at 10:31 PM in Hidden Value, Homesteading | Permalink | Comments (0)

November 25, 2014

Open source SED manages 3000 streams

Open source resources make it possible to use SED, a stream editor built in the open source community. Since 2001 SED has worked on the HP 3000, thanks to Lars Appel, a former HP support engineer who ported Samba to the platform in the 1990s.

SED's main MPE page is on a page of Appel's. SED is an at your own risk download, but support is available through the 3000 community.

Dan Barnes, working on a problem he had to solve in his 3000 environment, asked:

The issue is incoming data from another platform that is being fed into MM 3000. This data occasionally has some unprintable characters, which of  course wrecks havoc on the MM application when it is encountered. To address this, the user, using a cygwin (Unix-like) environment on their Windows PC, developed a SED script. When they test the script in the cgywin environment it works just fine. But when done on the target HP 3000 it gets an undesirable result.

Barnes added that "The user thought that because MPE/iX is Posix-compliant, that this should work." He explained his user created the expression

sed -e 's/[\x7F-\xFE]/*/g' < COMSHD > COMSHD1

But Appel noted that hex 7F thru hex FE portion of the expression isn't supported on the MPE/iX version of SED. It's a limitation of MPE/iX, but there's a workaround.

Not sure if the regular expression usage here matches Posix or GNU specs, but my guess is the "\xNN" format, that seems to indicate a char by hex code, doesn't work.

How about something like using the command sed -e 's/[^ -~]/*/g' instead, i.e. map the characters outside the range space through tilde?

Posted by Ron Seybold at 10:05 AM in Hidden Value, Homesteading | Permalink | Comments (0)

November 17, 2014

HP's 3000 power supply persists in failure

Amid a migration project, Michael Anderson was facing a failure. Not of his project, but a failure of his HP 3000 to start up on a bad morning. HP's original hardware is in line for replacement at customers using the 3000 for a server. Some of these computers are more than 15 years old. But the HP grade of components and engineering is still exemplary.

"I was working with a HP 3000 Series 969, and one morning it was down," he reported. "All power was on, but the system was not running; I got no response from the console. So I power-cycled it, and the display panel (above the key switch) reported the following."

Proceeding to turn DC on

On the console it displayed garbage when the power was turned on, but the message on the display remained. I wasn’t sure what to replace. I was thinking the power supply — but all of the power was on. As it turned out, even in the middle of a power supply failure the 3000 was working to get out a message. The back side, the core I/O, FW SCSI, and so on, all appeared to have power. That is why I found it hard to believe that the power supply was the problem.

Anderson explained that Charles Johnson of Surety Systems replaced the power supply for the system.
He explained that (back in the day) HP engineered some of the best power supplies in world, lots of checks and verifications. Even though the power supply had actually supplied DC power to the various components, it was not able to verify it.

So the message "Proceeding to turn on DC power" remained on the front panel display, meanwhile the boot process on the console would hang, and if you do a <cntrl-B>, RS it would time-out with a msg:

"FATAL ERROR: System held in reset. POW_ON never came back (APERR 21)"

"Waiting until it's reasserted......"

Bill & Dave's Excellent Machine — even with a power supply failure, it still manages to get a message out (in plain English) attempting to explain the failure.

Posted by Ron Seybold at 08:41 PM in Hidden Value, Homesteading, User Reports | Permalink | Comments (0)

November 13, 2014

Thursday Throwback: IMAGE vs. Relational

As a precocious 18-year-old, Eugene Volokh wrote deep technical papers for HP 3000 users who were two or three times his age. While we pointed to the distinctions between IMAGE master and automatic datasets recently, Eugene's dad Vladimir reminded us about a Eugene paper. It was published in the fall of 1986, a time when debate was raging over the genuine value of relational databases.

While the relational database is as certain in our current firmament as the position of any planet, the concept was pushing aside proven technology 28 years ago. IMAGE, created by Fred White and Jon Bale at HP, was not relational. Or was it? Eugene offered the paper below to explore what all the relative fuss was about. Vladimir pointed us to the page on the fine Adager website where the paper lives in its original formatting.

COBO HallThe relationships between master and automatic and detail datasets pointed the way to how IMAGE would remain viable even during the onslaught of relational databases. Soon enough, even Structured Query Language would enter the toolbox of IMAGE. But even in the year this paper emerged, while the 3000 still didn't have a PA-RISC model or MPE/XL to drive it, there was a correlation between relational DBs and IMAGE. Relational databases rely on indexes, "which is what most relational systems use in the same way that IMAGE uses automatic masters," Eugene wrote in his paper presented at COBO Hall in Detroit (above). QUERY/3000 was a relational query language, he added, albeit one less easy to use.

Vladimir admits that very few IT professionals are building IMAGE/SQL databases anymore. "But they do look at them, and they should know what they're looking at," he explained.

Relational Databases Vs. IMAGE:
What The Fuss Is All About

By Eugene Volokh, VESOFT

What are "relational databases" anyway? Are they more powerful than IMAGE? Less powerful? Faster? Slower? Slogans abound, but facts are hard to come by. It seems like HP will finally have its own relational system out for Spectrum (or whatever they call it these days). I hope that this paper will clear up some of the confusion that surrounds relational databases, and will point out the substantive advantages and disadvantages that relational databases have over network systems like IMAGE.

What is a relational database? Let's think for a while about a database design problem.

We want to build a parts requisition system. We have many possible suppliers, and many different parts. Each supplier can sell us several kinds of parts, and each part can be bought from one of several suppliers.

Easy, right? We just have a supplier master, a parts master, and a supplier/parts cross-reference detail:

Relational-IMAGE Fig 1Every supplier has a record in the SUPPLIERS master, every part has a record in the PARTS master, and each (supplier, part-supplied) pair has a record in the SUPPLIER-XREF dataset.

Now, why did we set things up this way? We could have, for instance, made the SUPPLIER-XREF dataset a master, with a key of SUPPLIERS#+PART#.  Or,  we  could have made all three datasets stand-alone details, with no masters at all. The point is that the proof of a database is in the using. The design we showed -- two masters and a detail -- allows us to very efficiently do the following things:

  • Look up supplier information by the unique supplier #.
  • Look up parts information by the unique part #.
  • For each part, look up all its suppliers (by using the cross-reference detail dataset).
  • For each supplier, look up all the parts it sells (by using the cross-reference detail dataset).

This is what IMAGE is good at -- allowing quick retrieval from a master using the master's unique key and allowing quick retrieval from a detail chain using one of the detail's search items. 

However, let’s take a closer look at the parts dataset. It actually looks kind of like this:

PART# <-- unique key item

What if we want to find all the suppliers that can sell us a "framastat"? A "framastat", you see, is not a part number -- it's a part description. We want to be able to look up parts not only by their part number, but also by their descriptions. The functions supported by our design are:

  • Look up PART by PART#.
  • Look up PARTs by SUPPLIERS#.
  • Look up SUPPLIERs by PART#.

What we want is the ability to

  • Look up PART by DESCRIPTION.

The sad thing is that the PARTS dataset is a master, and a master dataset supports lookup by ONLY ONE FIELD (the key). We can't make DESCRIPTION the key item, since we want PART# to be the key item; we can't make DESCRIPTION a search item, since PARTS isn't a detail. By making PARTS a master, we got fast lookup by PART# (on the order of 1 or 2 I/Os to do the DBGET), but we forfeited any power to look things up quickly by any other item.

And so, dispirited and dejected, we get drunk and go to bed. And, deep in the night, a dream comes. "Make it a detail!" the voice shouts. "Make it a detail, and then you can have as many paths as you want to."

We awaken elated! This is it! Make PARTS a detail dataset, and then have two search items, PART# and DESCRIPTION. Each search item can have an automatic master dataset hanging off of it, to wit:

Relational-IMAGE Fig 2

What's more, if we ever, say, want to find all the parts of a certain color or shape, we can easily add a new search item to the PARTS dataset. Sure, it may be a bit slower (to get a part we need to first find it in PART#S and then follow the chain to PARTS, two IOs instead of one), and also the uniqueness of part numbers isn't enforced; still, the flexibility advantages are pretty nice.

So, now we can put any number of search items in PARTS. What about SUPPLIERS? What if we want to find a supplier by his name, or city, or any other field? Again, if we use master datasets, we're locked into having only one key item per dataset. Just like we restructured PARTS, we can restructure SUPPLIES, and come up with:

Relational-IMAGE Fig 3Note what we have done in our quest for flexibility. All the real data has been put into detail datasets; every data item which we're likely to retrieve on has an automatic master attached to it.

Believe it or not, this is a relational database.

If this is a relational database, I'm a Hottentot

Surely, you say, there is more to a relational database than just an IMAGE database without any master datasets. Isn't there? Of course, there is. But all the wonderful things you've been hearing about relational databases may have more to do with the features of a specific system that happens to be relational than with the virtues of relational as a whole.

Consider for a moment network databases. IMAGE is one example, in fact an example of a rather restricted kind of network database (having only two levels, master and detail). Let's look at some of the major features of IMAGE:

  • IMAGE supports unique-key MASTERS and non-unique-key DETAILS.
  • IMAGE does HASHING on master dataset records.
  • IMAGE has QUERY, an interactive query language.

Which of these features are actually network database features? In other words, which features would be present in any network database, and which are specific to the IMAGE implementation? Of the three listed above, only the first -- masters and details -- must actually be present in all databases that want to call themselves "network." On the other hand, a network database might very well use B-trees or ISAM as its access method instead of hashing; or, it might not provide an interactive query language. It would still be a network database -- it just wouldn't be IMAGE.

Why is all this relevant? Well, let's say that somebody said "Network databases are bad because they use hashing instead of B-trees." This statement is wrong because the network database model is silent on the question of B-trees vs. hashing. It is incorrect to generalize from the fact that IMAGE happens to use hashing to the theory that all network databases use hashing. If we get into the habit of making such generalizations, we are liable to get very inaccurate ideas about network databases in general or other network implementations in particular.

The same goes for relational databases. The reason that so many people are so keen on relational databases isn't because they have any particularly novel form of data representation (actually, it's much like  a  bunch  of old-fashioned KSAM/ISAM-like files with the possibility of multiple keys); nor is it because of some fancy new access methods (hashing, B-trees, and ISAM are all that relational databases support). Rather, it's because the designers of many of the modern relational databases did a good job in providing people with lots of useful features (ones that might have been just as handy in network databases).

What are relational databases: functionality

The major reason for many of the differences between relational databases and network databases is simple: age. Remember the good old days when people hacked FORTRAN code, spending days or weeks on optimizing out an instruction or two, or saving 1000 bytes of memory (they had only 8K back then) ? Well, those are the days in which many of today's network databases were first designed; maximum effort was placed on making slow hardware run as fast as possible and getting the most out of every byte of disk.

Relational databases, children of the late '70s and early '80s had the benefit of perspective. Their designers saw that much desirable functionality and flexibility was missing in the older systems, and they were willing to include it in relational databases even if it meant some wasted storage and performance slow-down. The bad part of this is that, to some extent, modern relational databases are still hurting from slightly decreased performance; however, this seems to be at most a temporary problem, and the functionality and flexibility advantages are quite great.

For even more IMAGE education, like the advantages of IMAGE over relational databases, and a tour of the flexibility that automatic masters provide, see the remainder of the paper on the Adager website.

Posted by Ron Seybold at 01:48 PM in Hidden Value, History, Homesteading | Permalink | Comments (0)

November 07, 2014

Manual and Automatic Masters, Detailed

A few days ago we included a Hidden Value question about how manual and automatic masters work in TurboIMAGE. Our ally and friend Vladimir Volokh called to note that in part of the question, the system manager had found "one detail data set that has thousands of entries which do not appear to be connected to any master.

It wasn't exactly a question, but in a reply on the 3000-L mailing list and newsgroup, Roy Brown gave a fine tutorial on how these features do their jobs for MPE and the 3000 -- as well as how a detail dataset might have zero key fields.

Manual masters can contain data which you define, like Detail sets can, along with a single Key field. Automatic masters contain only the Key field.

In both cases, there can be only one record for a given key value in a Master dataset.

A Detail dataset contains data fields plus zero, one, or many key fields. There can be as many records as you like for a given key value, and these form a chain accessible from the Master record key value. This chain may be sorted, or it may just be in chronological order of adding the records.

Zero key fields in a Detail dataset would be unusual, but is permissible.

Brown explained that "Where there are keys, referential integrity demands that there are no Detail record entries with a key field that is not found in either a Manual or Automatic master, both Key name and Key value. So a Detail data set with Key fields that are not present in a Master record would be a sign of a seriously corrupted database."

However, I doubt this is the case, and when you do a QUERY FORM command, you will see which fields in Detail datasets are Keys, which fields are used to establish Sort orders, and which fields are data pure and simple.

From the Key name, you can determine which Master set links the keys.

As I said above, it is possible to have a Detail dataset with no keys, but these usually contain only a very few records, since direct access to them without keys is cumbersome, and you would otherwise have to trawl right through one to find any given entry.

So a Detail dataset with thousands of unconnected entries would be very unlikely.

The FORM output will allow you to check how the Detail dataset that you think might have unconnected entries is actually linked in.

Posted by Ron Seybold at 09:55 PM in Hidden Value, Homesteading | Permalink | Comments (0)

November 04, 2014

Tips for Listing SCHEMAs, and FTP Listings

From my existing TurboIMAGE database, I want to generate a listing of data sets, data item names, and their relationships (master, detail). One detail data set has thousands of entries which do not appear to be connected to any master. 

Oh, and I cannot remember the difference between manual and automatic masters.

Francois Desrochers first replies, "Use Query's FORM command."

PASSWORD = >> password
MODE = >> 5

Manual masters: programs have to explicitly add entries before you can add related entries in detail sets. Programs have to explicitly delete entries when there are no related detail entries left. In other words, you have to do master dataset maintenance.

Automatic masters: entries are automatically created when a related detail set entry is created. Entries in the master are automatically removed when the last related detail entry is deleted. IMAGE takes care of the maintenance.

Consultant Ron Horner adds, "If you have a tool like Adager or DBGeneral, it can create a file of the database schema. The other way is by using QUERY to get a listing."

Horner also said, "The difference between a manual master and auto master is the following:

1. You have to add records to the manual master that contain the key data for any detail datasets that are linked to the master.

2. When working with automatic masters, you don't have to write data to them at all. IMAGE takes care of populating the master.

Ray Legault suggested using Allegro's free XSCHEMA, "A stripped down version of our commercial product DBHTML, this utility simply reads an Image root file and produces a DBSCHEMA-compatible text file as output."

Krikor Gullekian also noted that "With QUERY you can check the databases as long as you know the password. [Ed. note: Password advice is true, except when you're the database owner. No password is required then, just a semicolon.] FO SETS will give you a lot of details."

How can I tell the HP 3000's FTP server to use a standard 'ls -l', and not 'LISTFILE ,2' ?

Allegro's Donna Hofmeister said, The trick is to turn 'Posix' on. 

Turning Posix On

Keven Miller of 3K Ranger added "check out FTPDOC.ARPA.SYS, mainly the POSIX section. It mentions how to set the default to ON."

Posted by Ron Seybold at 04:19 PM in Hidden Value, Homesteading | Permalink | Comments (0)

October 28, 2014

Strategies for Redirecting App Spoolfiles

An HP 3000 manager wrote that a 24x7 application at his shop is stable and never goes offline unless it's required. But the everyday management had to include aborting the app once a week.

We take that application offline to close out the spoolfile that the application generates. Is there a way to keep the application running, and just redirect the output to a new spoolfile? We're using an N-Class server.

Robert Schlosser of Global Business Systems replied: Short of closing and reopening the application after n number of pages, you could have the application read (without wait and checking status codes) a message file. It could then close and open the output file on demand, and possibly even close down the application gracefully (no abort).

Our Homesteading Editor Gilles Schipper replied: I think the only way you could do that would be to actually modify the application program to periodically (say, for example, every 10 pages or every 100 pages) close then re-open the print file.

Olav Kappert of IOMIT International added:

If the program can be slightly modified, then I would suggest creating a message file as a conduit to the application. The program would do a read of the message file with the nowait option every once and awhile.  If the application encounters a keyword indicating a new spoolfile, then the program would close the spoolfile and reopen it.

An alternate method would involve the application being modified to close and open the file at a particular day and time during the week.

Posted by Ron Seybold at 05:34 PM in Hidden Value, Homesteading | Permalink | Comments (0)

September 23, 2014

Pre-Migration Cleanup Techniques

Migrations are inevitable. The Yolo County Office of Education is on its way to a Windows-based system, after many years of HP 3000 reliance. Ernie Newton of the Information and Technology Services arm of the organization is moving his 3000 data. He's doing a clean-up, a great practice even if you're not heading off of MPE.

I am cleaning up our IMAGE databases for the inevitable move to Microsoft’s SQL Server. One thing I've encountered is that Suprtool does not like null characters where there should be numbers.

I know that I have invalid characters, (non-numeric), in a field called ITEM-NUMBER.  But when I try to find those records, Suprtool chokes and abruptly stops the search. Here's what I get...


Error:  Illegal ascii digit encountered. Please check all data sources
Input record number: 1

Is there a way to run Suprtool to help it find these records? Query finds them just fine, but Query doesn't have to ability to do what I want to do. 

After being reminded that "Nulls are not numbers," by Olav Kappert, and "try to use a byte string to compare (like < "a" or > "z") or something like that," Robelle's Neil Armstrong weighed in.

You can find any character you want by using the Clean, $findclean and $clean feature. The first issue to deal with is to re-define the item-number as a byte type in order to use the function.

Armstrong explained, "The following shows how to find the fields with 'invalid' zoned-decimal characters. Remember that zoned fields use characters to indicate the sign. It's likely that you don't have negative Item numbers but they are valid.

get somedataset
def item-x,item-number,byte
{Setup what characters to look valid characters are 0-9, JKLMNOPQRST, ABCDEFGHI and the curly braces }
{ so Clean characters should be not the above }
clean "^0:^47","^58:^64","^85:^122","^126:^255"
if $findclean(item-x)
ext item-x
list st

"Note the clean command defines the 'decimal' characters to look for."

Posted by Ron Seybold at 05:50 PM in Hidden Value, Migration | Permalink | Comments (0)

September 22, 2014

Ways to Create PDFs from 3000 Output

Years ago -- okay, seven -- we reported the abilities of the Sanface Software solution to create PDF files out of HP 3000 output. But there are other ways and tools to do this, a task that's essential to sharing data reports between HP 3000s and the rest of the world's computers.

On the HP 3000 newsgroup, a veteran 3000 developer has asked,

Has anyone got any experience involving taking a file in an output queue and creating a PDF version of it?

"We use text2pdf v1.1 and have not had any problems since we installed it in October 2001," said Robert Mills of Pinnacle Entertainment. "I have e-mailed a copy of this utility and our command file to 27 people. Never knew that so many sites wanted to generate PDFs from their 3000s."

The program is a good example of 3000 source code solutions. This one was created as far back as the days of MPE/iX 6.0, a system release which HP has not supported since 2005.

Lars Appel, the former HP support engineer who built such things on his own time while working at HP Support in Germany -- and now works with Marxmeier on its Eloquence product -- has source code and a compiled copy of the utility.

Such solutions, and many more, are hosted on the Web server at 3K Associates, Check the Applications Ported to MPE/iX section of the Public Domain Software area at 3K's Web site.

You'll also find a link to GhostPCL up at the site, another Appel creation, one which he describes as

A program that reads PCL input files and converts them to a variety of output formats, including PDF or JPEG, for example. Combined with my little FakeLP Java program, you might even use it to capture MPE/iX network spooler output and generate PDF or JPEG from an MPE/iX spoolfile.

Open source solutions like these have been an HP 3000 community tradition. Way back in 2000, we reported in the print 3000 NewsWire about that FakeLP Java program, helpful in getting text2pdf to do its PDF magic.

A roadblock to using the text2pdf program: the spoolfiles had to be in text file format to work with it. But Lars Appel offered a free solution to make 3000 spoolfiles that don't rely on CCTLs ready for their PDF closeups:

"I have a small Java program that listens to a given port, for example 9100, and 'pretends to be a network printer' i.e. gets all the data sent and writes it to a flat file. This might be a start, as OUTSPTJ.PUB.SYS should have converted CCTL to plain PCL when sending to a JetDirect printer. However, this little Java program is just a quick and dirty experiment. Use at your own risk; it worked on my 3000, but your mileage may vary."

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cut here _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

// FakeLP pretends network printer to capture spooler PCL output


class FakeLP {

public static void main( String args[] ) throws Exception {

int port = 9100;
int next = 1;

if (args.length > 0) port = Integer.parseInt(args[0]);
if (args.length > 1) next = Integer.parseInt(args[1]);

ServerSocket serv = new ServerSocket( port );

while (true) {

System.out.println("FakeLP listener ready");

Socket sock = serv.accept();
byte[] buf = new byte[4096];
String name = "F" + (next++);

System.out.println("Capturing spoolfile to " + name);

InputStream si = sock.getInputStream();
OutputStream fo = new FileOutputStream(name);

for (;;)
int got =;

if (got != -1)
fo.write(buf, 0, got);


Posted by Ron Seybold at 06:43 PM in Hidden Value, Homesteading | Permalink | Comments (0)

September 19, 2014

Passing FTP Capabilities to MPE

Ws-FTP ProHP 3000s do lots of duty with data from outside the server. The 3000's FTP services sit ready to handle transfers from the world of Windows, as well as other systems, and PCs far outnumber the non-Windows computers networked to 3000s. Several good, low-cost FTP clients on Windows communicate with the 3000, even though MPE/iX still has some unique "features" in its FTP server.

Our former columnist John Burke once reported that his HP 3000 emitted a second line of text during an FTP session that could confuse the open source FTP client FileZilla:

FileZilla issues the PWD command to get the working directory information. On every other system I've tried, the result is something like 257 "home/openmpe" is the current working directory However, MPE responds with something like 257-"/SYSADMIN/PUB" is the current directory. 257 "MGR.SYSADMIN,PUB" is the current session. The second line appears to be confusing FileZilla because it reports the current directory as /MGR.SYSADMIN,PUB/, which of course does not work.

Back when it was a freeware, Craig Lalley took note of a worthy solution, WS-FTP from IP Switch. The product is now for sale but its client is not costly. And an MPE setting can remove the problems that can choke up FileZilla.

Lalley, who runs the 3000 consultancy Echo Tech, once offered this advice about WS-FTP. "I have used it for several years, without any problems. I also have used Bullet FTP and CuteFTP." About the built-in FTP in browsers, as far back as Internet Explorer, he added, "Don't go there."

Chris Thompson of The Internet Agency, another 3000-friendly vendor, echoed the praise of WS-FTP. Thompson also sells MPE software, the MPE/iX Enterprise Client. Alas, he noted that the much-praised Whisper Technology, now defunct, also had a laudable FTP product

WS-FTP is a really good product. Also, try FTP Surfer, which is freeware from Whisper Technology Limited. Usually we use this product to FTP to our 937. It's always worked well.

But as might be expected, there's a way to make HP's FTP behave in less unique and more compliant way. Lars Appel, who ported Samba to the HP 3000 before he left HP's support team, delivered the answer that makes FileZilla work with the 3000

Try the "SITE POSIX ON" command in your FTP session already (or the respective POSIX=ON setting in the SETPARMS.ARPA.SYS config file to change the default, in case the FileZilla session cannot issue "site"

Burke once reported that "POSIX = ON in the SETPARMS file did the trick, eliminating the message that confused FileZilla. I've been using FileZilla for all my ad hoc FTP needs for some time now — works great to all manner of Unix, Windows and Linux systems."

HP's James Hofmeister, who's led the effort to keep FTP up to date on the 3000, took issue with claims that the 3000 doesn't play well with Web-based FTP clients.

Lots of work went into an implementation of the FTPSRVR to support web access to the 3000... The "SITE POSIX ON" command can be sent by a FTP client and the 3000 FTPSRVR will emit Posix "standard" FTP output and will react like a Posix host (including file naming conventions).

It also is possible as documented to specify "POSIX=ON" mode in the file and achieve this functionality system-wide for all non-3000 client to 3000 FTPSRVR connections; again the FTPSRVR will emit Posix "standard" FTP output and will react like a Posix host (including file naming conventions).

Warning:  Before you specify "POSIX=ON" mode in the file, make sure you read the FTPDOC file closely; as you are warned that MPE file syntax will "no longer" work; The 3000 FTPSRVR is acting in Posix mode.

Posted by Ron Seybold at 04:05 PM in Hidden Value, Homesteading | Permalink | Comments (0)

September 16, 2014

Advice on uptime, net gateways and sockets

Is there a way to use the 3000's networking to check how long your system has been up?

James Hofmeister replies:

If you have SNMP running, a query to check system uptime is:

: snmpget public system.sysUpTime.0
Name: system.sysUpTime.0
Timeticks: (418638300) 48 days, 10:53:03

I get no awards for 48 days uptime, but I use my machines to duplicate, beta test and verify repair of customer network problems.

Is there a way to scan all the ports on my HP 3000 Series 996: How many are being used, and how many are available?

Mark Bixby replies:

SOCKINFO.NET.SYS can tell you which programs have opened which sockets.

NETTOOL.NET.SYS STATUS,TCPSTAT and STATUS,UDPSTAT can also give you useful information about sockets, particularly STATUS,TCPSTAT and CONNTABLE.

Or you can run an external tool and do a port scan against your 3000.  This is not recommended during production hours, since such port cans can sometimes confuse network applications.

When I try to configure a  on our MPE/iX 7.5 system, I get the following error when I try to validate my new NMMGR gateway configuration.

Searching for subsystem validation routine VALIDATENETXPOR

There are no other gateways configured so the CONFIGURED GATEWAYS (1) value look okay to me — so how can I increase the IPU MAX GATES value?

James Hofmeister replies:

In MPE/iX 5.5 and 6.0 (unpatched) the limit was 14 gateways. This was increased to 255 gateways with patches, and was included in base 6.5 and 7.x.

The fact that validate says “IPU MAX GATES (0)” would indicate to me that you have corruption of your configuration file in “at least” the field that holds this value.

I would suggest that you want to first keep a copy of this config file, then purge NMCONFIG.PUB.SYS and then rebuild your configuration with guided config.

Note:  You could do a copy subtree of the NETXPORT.PROT.IPU field from NMAUX1.PUB.SYS to NMCONFIG.PUB.SYS to update this field — but at this point I would expect problems in this config file with more than just this one field.

Posted by Ron Seybold at 06:27 PM in Hidden Value | Permalink | Comments (0)

August 26, 2014

See how perl's strings still swing for MPE

PerlheartThe HP 3000 has a healthy range of open source tools in its ecosystem. One of the best ways to begin looking at open source software opportunity is to visit the MPE Open Source website operated by Applied Technologies. If you're keeping a 3000 in vital service during the post-HP era, you might find perl a useful tool for interfacing with data via web access.

The 3000 community has chronicled and documented the use of this programming language, with the advice coming from some of the best pedigreed sources. Allegro Consultants has a tar-ball of the compiler, available as a 38MB download from Allegro's website. (You'll find many other useful papers and tools at that Allegro Papers and Books webpage, too.)

Bob Green of Robelle wrote a great primer on the use of perl in the MPE/iX environment. We were fortunate to be the first to publish Bob's paper, run in the 3000 NewsWire when the Robelle Tech long-running column made a hit on our paper pages.

You could grab a little love for your 3000, too. Cast a string of perls starting with the downloads and advice. One of HP's best and brightest -- well, a former HP wizard -- has a detailed slide set on perl, too.

The official website has great instructions on Perl for MPE/iX installation and an update on the last revision to the language for the 3000. First ported by Ken Hirsch in 2000, the language was brought to the 5.9.3 release in 2006.

An extensive PowerPoint presentation on perl by the legendary porter Mark Bixby will deliver detailed insights on how to introduce perl to your programming mix. Bixby, who left HP to work for the 3000 software vendor QSS, brings the spirit of open source advocacy to his advice on how to use this foundational web tool.

As an example, Bixby notes that "it's now possible to write MPE applications that look like web browsers, to perform simple HTTP GET requests, or even complicated HTTP POST requests to fill out remote web forms." It's no box of Godiva, or even the classic blue box from Tiffany's, but perl might be something you love to use, to show that 3000 isn't a tired old minicomputer -- just a great sweetheart of a partner in your mission-critical work.

Posted by Ron Seybold at 10:35 PM in Hidden Value, Homesteading | Permalink | Comments (0)

August 15, 2014

The 3000's got network printing, so use it

Ten years ago this summer, HP's 3000 lab engineers were told that 3000 users wanted networked printing. By 2005 it was ready for beta testing. This was one of the last enhancements demanded as Number 1 by a wide swath of the 3000 community, and then delivered by HP. The venerable Systems Improvement Ballot of 2004 ranked networked printing No. 1 among users' needs.

MPEMXU1A is the patch that enables networked printing, pushed into General Release in Fall, 2005. In releasing this patch's functionality, HP gave the community a rather generic, OS-level substitute for much better third party software from RAC Consulting (ESPUL). It might have been the last time that an independent software tool got nudged by HP development.

HP M1522N printerThe HP 3000 has the ability to send jobs to non-HP printers over a standard network as a result of the enhancement. The RAC third party package ties printers to 3000 with fewer blind spots than the MPEMXU1A patch. HP's offering won't let Windows-hosted printers participate in the 3000 network printing enhancement. There's a Windows-only, server-based net printing driver by now, of course, downloadable from the Web. The HP Universal Print Driver Series for Windows embraces Windows Server 2012, 2008, and 2003.

Networked printing for MPE/iX had the last classic lifespan that we can recall for a 3000 enhancement. The engineering was ready to test less than a year after the request. This software moved out of beta test by November, a relatively brief five-month jaunt to general release. If you're homesteading on 3000s, and you don't need PCL sequences at the beginning and end of a spool file, you should use it. Commemorate the era when the system's creator was at least building best-effort improvements.

MPE/iX 6.5 was still being patched when networked printing rolled out. That's a release still in some use at  homesteading shops. In contrast, plenty of later patches were only created and tested for the 7.0 and 7.5 PowerPatch kits.

Deep inside the Web is a white paper that former HP staffer Jeff Vance wrote, a guide he called "Communicator-like" after the classic HP technical documents. HP's taken down its Jazz repository of tech papers where NWPrinting.html once was available. But our open source software expert Brian Edminster tracked down that gem at the Client Systems website -- the company which was one of two to license HP's tech papers. You could check in with your independent support provider, to see if they've got the paper.

Networked printing was never as comprehensive as the indie solutions for the 3000, but at least it was delivered on the OS level via patches. The vendor still warned that adding new printers was going to be an uneven process.

HP will support this enhancement on a "best-effort" basis, meaning we will attempt to duplicate and resolve specific spooler problems -- but we cannot guarantee that all ASCII based printers are supported by this enhancement.

Of course, HP's support is long gone. But while best-effort might sound like a show-stopper so many years later, you'd be surprised how many printers of that 6.5 era are still attached at homesteading 3000 sites.

Where do you get the patch? That's where HP's still doing its work. These MPE/iX patches were given special dispensation from the pay-for-patches edict of 2010. They're still free by calling HP. That non-Windows printer and MPE might seem like old technology. But HP's still using telephones to enable the delivery of patches, so there's that Throwback -- and one you can find on days which are not Thursdays, too.

Posted by Ron Seybold at 03:12 PM in Hidden Value, Homesteading, Web Resources | Permalink | Comments (0)

August 08, 2014

Classic Advice: Adding a DLT to an HP 3000

I'm trying to add a DLT to a my HP 3000 939KS and it keeps reporting media as bad. I can FCOPY but not run an Orbit or MPE store. It does mount the tape normally. The MPE store gives the following error:


The server which this drive is being added to has DDS-3s on it, but we are adding another disk array, so we are going to outgrow what we have very quickly.

DLT4000DLT8000s have not been manufactured in perhaps 10 years. Even five-year-old drives are SDLTII or DLT VS160, or some form of LTO. Also, using HVD-SCSI is so last century. At any rate, the heads on the DLT drives do get used up depending on the media used. Try another DLT drive, if possible.

Unfortunately, this is the exact issue facing homesteaders and others who are delaying their migration off the HP 3000, especially if they have pre-PCI machines like the 939. The hardware to run with it can be difficult to find, but it's out there, although it can be of varying level of readiness. You have many options open to you, but as time goes by they will more difficult to implement.

1. Look for another DLT8000 or a DLT7000. Either of these tape drives will work, and you will not get any performance benefit from either one over the DDS-3, but you will get more storage on one tape. You might make sure it has HP-branded firmware; there have been a painful set of System Aborts, due to semi-random walks through driver state machines — initiated by non-certified firmware.

2. Instead of a DLT, consider getting more DDS-3 drives. One medium size N-Class can have 12 DAT24 drives -- they do either a 4x3 or 3x4 parallel storeset. No messing with "reel" switches.

3. Consider getting an HVD to SE/LVD SCSI converter and then trying a DDS-4 device. DAT40 with DAT24 media has worked well for some sites, but DAT40 with DAT40 media is only supported on A/N-Class. To get technical on you here, you may only configure the DLT (scsi_tape2_dm) driver "under" the NIO F/W SCSI HBA (fwscsi_dam).

4. Move to a PCI HP 3000 (the A-Class or a small N-Class,) then use the newer LVD devices. PCI systems will at least enable the usage of much newer used equipment — and even some new stuff, if you want to buy a XP10000/12000.

Posted by Ron Seybold at 03:31 PM in Hidden Value, Homesteading | Permalink | Comments (0)

July 30, 2014

Find :HELP for what you don't know exists

Last week we presented a reprise of advice about using the VSTORE command while making backups. It's good practice; you can read about the details of why and a little bit of how-to in articles here, and also here.

But since VSTORE is an MPE command, our article elicited a friendly call from Vesoft's Vladimir Volokh. He was able to make me see that a great deal of what drives MPE/iX and MPE's powers can remain hidden -- the attribute we ascribed to VSTORE. "Hidden, to some managers running HP 3000s, is the VSTORE command of MPE/iX to employ in system backup verification." We even have a category here on the blog called Hidden Value. It's been one of our features since our first issue, almost 19 years ago.

MPE commands exampleFinding help for commands is a straightforward search, if those commands are related to the commands you know. But how deep are the relationships that are charted by the MPE help system? To put it another way, it's not easy to go looking for something that you don't know is there. Take VSTORE, for example. HP's HELP files include a VSTORE command entry. But you'll only find that command if you know it's there in the operating environment. The "related commands" part of the entry of STORE, identifying the existence of VSTORE, is at the very bottom of the file.

Vladimir said, "Yes, at the bottom. And nobody reads to the bottom." He's also of the belief that fewer people than ever are reading anything today. I agree, but I'd add we're failing in our habits to read in the long form, all the way beyond a few paragraphs. The Millennial Generation even has an acronymn for this poor habit: TLDR, for Too Long, Didn't Read. It's a byproduct of life in the Web era.

But finding help on VSTORE is also a matter of a search across the Web, where you'll find archived manuals on the 5.0 MPE/iX where it was last documented. There's where the Web connects us better than ever. What's more, the power of the Internet now gives us the means to ask Vladimir about MPE's commands and the MPEX improvements. Vladimir reads and uses email from his personal email address. It's not a new outlet, but it's a place to ask for help that you don't know exists. That's because like his product MPEX, Vladimir's help can be conceptual.

Hold down the right-most or left-most mouse button and you'll see contextual help in plenty of applications. MPE commands don't have this feature, and while they don't seem to need it, conceptual help is missing, too. There's :HELP for many subjects, but conceptual help involves skipping over those TLDR habits.

Our original article about VSTORE used the command in context with a primer on when to create a System Load Tape. Do a VSTORE when you make an SLT, said Vladimir as well as our ally Brian Edminster. Creating context is high-order programming, something we can do more easily with our wetware than with software. It's about seeing relationships, connecting the dots.

"You can't ask for help for something you don't know exists," is how Vladimir posed the problem of contextual help in the MPE interface. Go to the %HELP of MPEX and you'll get related commands right away. For example, typing %HELP STORE will allows you to choose from the following topics:

1. %MPEXSTORE, MPEX command
2. MPE's :RESTORE help text
3. MPE's :STORE help text
4. MPE's :VSTORE help text
5. STORED, a file attribute variable

In comparison, you might not be aware of VSTORE's relationship to backups by using HP's :HELP files.

How did we learn about those %HELP options? The Internet led us to a 19-year-old technical paper written by Paul Taffel while he was in the Vesoft stables. The paper, hosted at Gainsborough Software, details the improvements to MPEX as a result of integrating the (then-new) Posix interface of MPE. Two-thirds of the way through an article of 2,800 lines, there's that %HELP information. (There's even a little joke about typing %HELP SENTENCE, and another about %HELP DELI in MPEX.)

It's all out there, somewhere, these opportunities to learn what you even don't know exists, but need to know. And you'd want to learn about efficient and effective use of MPE because? Well, because an HP 3000 might be a key part of your datacenter longer than expected -- and your best expert has already typed his final :BYE. In that 19-year-old article, Taffel expressed Vesoft's ideal about questions from the community.

We at VESOFT really encourage you to contact us with your favorite "I'd like to do this but I can't" problem.  MPEX has evolved largely as a result of the continued suggestions of our many thousands of users, and we hope to continue this process as long as you continue to come up with new problems.

After that message, there's a contact phone number for Vesoft, the one that still reaches the company's offices, unchanged after decades. But there's also current email to follow by this year for contextual help, by dropping a note into Vladimir's inbox. Your reply might include a call, a sample of MPE help that's so well hidden you don't know you need it.

Posted by Ron Seybold at 06:58 PM in Hidden Value, History, Homesteading | Permalink | Comments (1)

July 24, 2014

Using VSTORE to Verify 3000 Backups

Card VerificationHidden, to some managers running HP 3000s, is the VSTORE command of MPE/iX to employ in system backup verification. It's good standard practice to include VSTORE in every backup job's command process. If your MPE references come from Google searches instead of reading your NewsWire, you might find it a bit harder to locate HP's documentation for VSTORE. You won't find what you'd expect inside a MPE/iX 7.5 manual. HP introduced VSTORE in MPE/iX 5.0, so that edition of the manual is where its details reside.

For your illumination, here's some tips from Brian Edminster, HP 3000 and MPE consultant at Applied Technologies and the curator of the MPE Open Source repository,

If possible, do your VSTOREs on a different (but compatible model) of tape drive than the one the tape was created on. Why? DDS tape drives (especially DDS-2 and DDS-3 models) slowly go out of alignment as they wear.

In other words, it's possible to write a backup tape, and have it successfully VSTORE on the same drive. But if you have to take that same tape to a different server with a new and in-alignment drive, you could have it not be readable! Trust me on this -- I've had it happen.

If you'll only ever need to read tapes on the same drive as you wrote them, you're still not safe. What happens if you write a tape on a worn drive, have the drive fail at some later date -- and that replacement drive cannot read old backup tapes? Yikes!

Using the 'two-drive' method to validate backup (and even SLT) tapes is a very prudent choice, if you have access to that array of hardware. It can also often help identify a drive that's going out of alignment -- before it's too late! 

Unfortunately, SLTs have to be written to tape (at least, for non-emulated HP 3000s). However, your drive will last years longer if you only write to it a few times a year.

You can find HP's VSTORE documentation page from that 5.0 command manual on the Web, (thanks to MM Support for keeping all those those pages online).

Posted by Ron Seybold at 07:30 PM in Hidden Value, Homesteading | Permalink | Comments (0)

July 21, 2014

Maximum Disc Replacement for Series 9x7s

Software vendors, as well as in-house developers, keep Series 9x7 servers available for startup to test software revisions. There are not very many revisions to MPE software anymore, but we continue to see some of these oldest PA-RISC servers churning along in work environments.

9x7s, you may ask -- they're retired long ago, aren't they? Less than one year ago, one reseller was offering a trio for between $1,800 (a Series 947) and $3,200. Five years ago this week, tech experts were examining how to modernize the drives in these venerable beasts. One developer figured in 2009 they'd need their 9x7s for at least five more years. For the record, 9x7s are going to be from the early 1990s, so figure that some of them are beyond 20 years old now.

"They are great for testing how things actually work," one developer reported, "as opposed to what the documentation says, a detail we very much need to know when writing migration software. Also, to this day, if you write and compile software on 6.0, you can just about guarantee that it will run on 6.0, 6.5, 7.0 and 7.5 MPE/iX."

BarracudaSome of the most vulnerable elements of machines from that epoch include those disk drives. 4GB units are installed inside most of them. Could something else replace these internal drives? It's a valid question for any 3000 that runs with these wee disks, but it becomes even more of an issue with the 9x7s. MPE/iX 7.0 and 7.5 are not operational on that segment of 3000 hardware.

Even though the LDEV1 drive will only support 4GB of space visible to MPE/iX 6.0 and 6.5, there's always LDEV2. You can use virtually any SCSI (SE SCSI or FW SCSI) drive, as long as you have the right interface and connector.

There's a Seagate disk drive that will stand in for something much older that's bearing an HP model number. The ST318416N 18GB Barracuda model -- which was once reported at $75, but now seems to be available for about $200 or so -- is in the 9x7's IOFDATA list of recognized devices, so they should just configure straight in. Even though that Seagate device is only available as refurbished equipment, it's still going to arrive with a one-year warranty. A lot longer than the one on any HP-original 9x7 disks still working in the community.

One developer quipped to the community, five years ago this week, "On the disc front at least that Seagate drive should keep those 3000s running, probably longer than HP remains a Computer Manufacturer."

But much like the 9x7 being offered for sale this year, five years later HP is still manufacturing computers, including its Unix and Linux replacement systems for any 3000 migrating users. 

So to refresh drives on the 9x7s, configure these Barracuda replacement drives in LDEV1 as the ST318416N -- it will automatically use 4GB (its max visible capacity) on reboot.

As for the LDEV2 drives, there are no real logical size limits, so anything under 300GB would work fine -- 300GB was the limit for MPE/iX drives until HP released its "Large Disk" patches for MPE/iX, MPEMXT2/T3. But that's a patch that wasn't written for the 9x7s, as they don't use 7.5.

Larger drives were not tested for these servers because of a power and heat dissipation issue. Some advice from the community indicates you'd do better to not greatly increase the power draw above what those original equipment drives require. The specs for those HP internal drives may be a part of your in-house equipment documentation. Seagate offers a technical manual for the 18GB Barracuda drive at its website, for power comparisons.

Posted by Ron Seybold at 07:51 PM in Hidden Value, Homesteading, Migration, User Reports | Permalink | Comments (2)

July 09, 2014

How to Employ SFTP on Today's MPE

Is anyone using SFTP on the HP 3000?

Gavin Scott, a developer and a veteran of decades on MPE/iX, says he got it to work reliably at one customer a year or so ago. "We exchanged SSL keys with the partner company," Scott said, "and so I don't think we had to provide a password as part of the SFTP connection initiation."

At least in my environment, the trick to not having it fail randomly around 300KB in transfers (in batch) was to explicitly disable progress reporting -- which was compiled into the 3000 SFTP client as defaulting to "on" for some reason. I forget the exact command that needed to be included in the SFTP command stream (probably "progress <mumble>" or something like that), but without that, it would try to display the SFTP progress bar. This caused it to whomp its stack or something similarly bad when done in a batch job, due to the lack of any terminal to talk to.

As SFTP is a pure Posix program, I ended up making Posix-named byte-stream files for stdin and stdout, and generally did all the SFTP stuff from the Posix shell. The MPE job ended up being a bunch of invocations of SH -c to execute an echo command to make the stdin file, and then another SH -c to run SFTP with a ;callci setvar varname -- $? or something like that -- on the end to capture the Posix process exit code back into the CI.

I also parsed/grepped the stdout file after the SFTP completed/exited, in order to test for seeing the actual file transferring message. I also wanted to make sure that all of the stdin content had been processed, so I could detect unexpected early termination or other problems that might not show up in $?.

That's all from memory, as I don't have access to the scripts any longer. In the end, SFTP was completely reliable, after working through all of its little issues.

Posted by Ron Seybold at 08:50 PM in Hidden Value, Homesteading | Permalink | Comments (0)

June 18, 2014

The Long and Short of Copying Tape

Is there a way in MPE to copy a tape from one drive to another drive?

Stan Sieler, co-founder of Allegro Consultants, gives both long and short answers to this fundamental question. (Turns out one of the answers is to look to Allegro for its TapeDisk product, which includes a program called TapeTape.)

Short answer: It’s easy to copy a tape, for free, if you don’t care about accuracy/completeness.

Longer answer: There are two “gotchas” in copying tapes ... on any platform.

Gotcha #1: Long tape records

You have to tell a tape drive how long a record you with to read.  If the record is larger, you will silently lose the extra data.

Thus, for any computer platform, one always wants to ask for at least one byte more than the expected maximum record — and if you get that extra byte, strongly warn the user that they may be losing data.  (The application should then have the internal buffer increased, and the attempted read size increased, and the copy tried again.)

One factor complicates this on MPE: the file system limits the size of a tape record you can read.  STORE, on the other hand, generally bypasses the file system when writing to tape and it is willing to write larger records (particularly if you specify the MAXTAPEBUF option).

In short, STORE is capable of writing tapes with records too long to read via file system access. The free programs such as TAPECOPY use the file system; thus, there are tapes they cannot  correctly copy.

Gotcha #2: Setmarks on DDS tapes

Some software creates DDS tapes and writes “setmarks” (think of them as super-EOFs). Normal file system access on the 3000 will not see setmarks, nor be able to write them.

Our TapeDisk product for MPE/iX (which includes TapeTape) solves both of the above problems. As far as I know, it’s the only program that can safely and correctly copy arbitrary tapes on an HP 3000.

Posted by Ron Seybold at 04:53 PM in Hidden Value, Migration | Permalink | Comments (0)

June 03, 2014

Paper clips play a role in 3000's guardian

The HP 3000 was designed for satisfactory remote access, but there are times when the system hardware needs to be in front of you. Such was the case for a system analyst who was adding a disk drive to a A-Class HP 3000.

BentpaperclipCentral to this process is the 3000's Guardian Service Processor (GSP). This portion of the A-Class and N-Class Multifunction IO card gives system managers basic console operations to control the hardware before MPE/iX is booted, as well as providing connectivity to manage the system. Functions supported by the GSP include displaying self-test chassis codes, executing boot commands, and determining installed hardware. (You can also read it as a speedometer for how fact your system is executing.)

The GSP was the answer to the following question.

I need to configure some additional disk drives and I believe reboot the server. The GSP is connected to a IP switch and I have the IP address for it, but it is not responding. I believe I need to enable it from the console. Can this be done from the soft console, using a PC as the console with a console # command?

A paper clip can reset the GSP and enable access, says EchoTech's Craig Lalley.

Lalley added that a GSP reset is an annual maintenance step for him.
Look on the back of the CPU and you will see a small hole labeled GSP RESET.  You need your favorite techie paper clip. Just insert the paper clip, and you will feel it depress. It takes about a minute to reset. Don't worry, it only reboots the GSP, and will not affect the HP 3000.

I find it is necessary to reset the GSP about once a year.  It seems to correlate to when you really need to get access, and you can't get physical access to the box. Good old Murphy's law.

Lalley calls the GSP, which HP introduced with its final generation of 3000s, one of the most useful things in the A-Class and N-Class boxes.

The GSP is a small computer that is always powered on when the plug has power. With it, it is possible to telnet to and be the console. While multiple admins can telnet in and watch, only one has the keyboard.

It is possible to reboot, do memory dumps and even fully power down the HP 3000 from the GSP.  Use the command PC OFF to power down. The GSP is probably the best feature of the N-Class and A-Class boxes.

Allegro's Stan Sieler has a fine white paper online about MPE/iX system failure and hang recovery that includes GSP tips.

Posted by Ron Seybold at 11:03 PM in Hidden Value | Permalink | Comments (0)