April 10, 2017

3000 backup strategy for closing Sundays

ClosedOnSundayEaster Sunday is on this week's horizon. While it's a rare day of closure at our local HEB grocery chain, Sundays are another sort of closure for 3000 managers. Nearly all of them want their partial backups of the weekdays to wrap up before the backup begins that will serve the work week. If you do full backups every night and want to make the new strategy to do partials during the week and a full on Sunday, there's a way to make that work. Donna Hofmeister, one of the former OpenMPE directors, explained the strategy in a message to 3000 managers.

First you need to decide what kind of partial you want to do.  On Tuesday, do you want to backup all files changed since Sunday's full backup or do you want to backup all files changed since Monday's backup?  (and so on....)

There are some things to think about here. If your "line in the sand" is always Sunday, then you have to deal with knowing that by Friday/Saturday your "partial" backup is likely going to be sizeable and will take longer to run. On the other hand, if you ever have to do a big restore, your restore plan is plan is pretty simple -- you'll need your last partial and your last full backup.

If your "line in the sand" is always yesterday, then your "partial" backups will be relatively small and quick. The flip-side is your big restore could be very complicated, since you'll need every partial backup through Monday plus your full backup. I think most people set the backup date as "Sunday" and do partials from there. But there's a technical bit that's also important.

The bit concerns the command restore ....;date>=!my_lastfull. Donna went on to explain.

"!my_lastfull" is a CI variable that contains a date in "mm/dd/[yy]yy" format.  This is a value that your *full backup job* needs to establish.  So your full backup job should do something like:

         file lastfull,old
         echo !hpmonth/!hpday/!hpyear>*lastful

Your partial job will need to do something like this:

         file lastfull,old
         input my_lastfull < *lastfull

There's plenty of logic that needs to be added to the above examples. (You'll note that there is NO error checking/what-if handling, etc.) This should be enough to get you started.

On incremental vs. differential backups — the former is a backup of changed files since the last full OR partial. The latter is a backup of changed files since the last full.

HP's wisdom about date management for the built-in STORE facility for HP 3000s is in several places. 3K Ranger's Keven Miller volunteered his, and Neil Armstrong's TeamNA Consulting has a downloadable one on file as well.

Posted by Ron Seybold at 07:40 PM in Hidden Value, Homesteading | Permalink | Comments (0)

Get e-mail notice when the NewsWire blog gets a new entry. Just say "Blog Me" in a message to editor@3000newswire.com.

April 07, 2017

Friday Fine-Tune: Creating the perfect CSLT

Editor's note: A classic technique, detailed here by the NewsWire's Hidden Value editor emeritus John Burke.

PerfectTapeA question about creating a CSLT for a Disaster Recovery test turned into a general discussion about what the perfect CSLT should look like. A system manager wanted to use the STORE option on the SYSGEN TAPE command to store additional files onto the CSLT he was creating for his DR test but was running into trouble trying to specify STORE options as part of the SYSGEN TAPE command. In particular, he wanted to simply add ;SHOW to get a listing of all files stored.

The answer to his original question is to use an indirect file, as in


where the indirect file contains whatever STORE directives you want in addition to the file list.

One contributor recommended robust efforts to get a listing: “A backup tape is of limited value without a listing. For Disaster Recovery purposes it is also a good idea to have the original HP tapes and patches with you as it is possible to create an SLT that does not install or work on a different HP 3000 system.” This same contributor also suggested creating a disk file with a listing of all the stored files.

However, several people questioned whether this list of files which the thread originator had proposed storing was sufficient. Stan Sieler probably said it best:

“I’d put much more in the STORE section, at the minimum:


(To explain, NL, SL, XL are dumped in the CSLT portion, so no need to dump them in the STORE section; DUMPAREA is a 32Mb file created at INSTALL time and there’s absolutely no need to dump it to a tape.)

(or wherever you put your dumps.)

(It’s surprising how much you might want tools at an early stage.)

“If most of your user data is in one or two accounts, other than SYS, TELESUP, and the rest of the system might fit well onto one DDS/DLT, you might find it more useful to do:


(where USERS, SALES are the ‘user’ accounts)

“Why? It guarantees you’ll get everything you’re likely to need in a recovery/install situation (except, of course, for the major portion of the user’s data). I’d also specify:

;show;directory=MPEXL_SYSTEM_VOLUME_SET,…etc.; progress;partialdb

Posted by Ron Seybold at 07:14 PM in Hidden Value, Homesteading | Permalink | Comments (0)

March 30, 2017

Puts, Gets, and Serving Up Transfers Faster

Server TrayHP 3000s are exchanging files with other servers, a process that's included the FTP protocol for more than 15 years. This capability was once so magic that the arrival of Samba file exchange on MPE/iX was lauded as a breakthrough. FTP is quite a ways off the most current of transfer protocols. One manager's started a discussion about how to improve transfer speeds to and from the 3000, though. He's using DSCOPY as well, but prefers the PUTs and GETs of FTP.

The advice that's current about FTP/iX says that hard-coding the 3000's ports (100mb full duplex, or 10mb half) is one way to speed things up. Ensuring your traffic is not running through a proxy (called "being proxied) is another idea. Measuring the speed of a PUT against a GET is one step in discovering why the 3000's FTP might seem slow.

In 2008, MPE/iX gained a secure version of FTP—at least part of one. This SFTP functionality arrived at the end of the Hewlett-Packard lab era for 3000s, a period when new tools were not being placed into wide use. Sites were locking down their 3000 scope of operations to ensure stability. The port of this then-current functionality fell short of complete: only an FTP secured client got created. PUTs could be secured, as well as GETs. But only from Windows, Unix, or Linux hosts to the 3000. The 3000 wasn't going to dish out files using secured FTP. There are notes in place to carry the work forward, though.

MPE/iX tools and components are also out there to complete this securing of file transfers. OpenSSH is the best-known protocol. A quick-start bundle can be downloaded from the MPE-OpenSource website run by Applied Technologies. There are SFTP installation instructions at Applied, too. Someone who's got a need for securing FTP transfers will need to do the server side of the porting, which was completed on the client side by Ken Hirsh, Mark Bixby, and Mark Klein. Requests to speed up FTP are a sign this porting would be more than just an open source hobby project.

Brian Edminster, the senior consultant at Applied Technologies, explained that "with a bit of work, you could get OpenSSH v 3.7.1p2 working. The issue is that 'select' is busted under MPE/iX, and that's what's required for ssh to work correctly."

The fact remains: ssh cannot connect to a remote system and execute commands that produce any output. Ken Hirsch did the original port, but he only really needed the SFTP client -- so the issue with ssh wasn't addressed.   

Ken also posted on the 3000-L newsgroup in 2008, asking if there was any interest in getting an ssh and sshd/sftp-server working (server daemon) -- so the 3000 could do port forwarding, act as a SFTP server, receive inbound ssh connections, and so on. Apparently he didn't get enough response to carry forward.

Back in 2005, Hirsch posted his goal. 

I could get an interactive ssh client to work on MPE/iX.  I don't know how, but I know it's possible! It would not be possible to get an ssh server working in such as way that an ssh client could run any program. But it would be possible to get enough of the server running so that you could use the server to do port forwarding.

In 2008, he added the note which Edminster referenced. "If anybody knows a way to actually write to a terminal while there is a read pending," Hirsch said, "I could use OpenSSH as a server on the HP 3000. Apparently there are undocumented MPE/iX sendio() and rendezvousio() calls, of which I know nothing. There are also tread()/twrite() routines in libbsd.a that I think are intended for this, but there's no documentation for these, either."

Posted by Ron Seybold at 07:11 PM in Hidden Value, Homesteading | Permalink | Comments (0)

March 17, 2017

Friday Fine-Tune: Moving all disks at once

I want to take all my disks, system volume set, and the user volumes, and move them to another machine that has no disks. How can I best do this?

Lars Appel replies

You might use SYSGEN to create two different config groups in the SYS account, one for the old system (e.g. CONFOLD), and one for the new system (e.g. CONFNEW). To create the new config you might start with one of the existing config templates, e.g. CONF9x8.SYS for 3000/9x8 systems. Use BASEGROUP to open it, adjust IO to your needs and KEEP it as CONFNEW.

7935Just make sure that every disk gets an LDEV in the new config. Paths and LDEVs (except for LDEV 1) do not need to be the same on the new system. MPE/iX will notice (and "collect") all available volumes during bootup (as long as they are "visible" by a configured LDEV number).

Shutdown the system, plug disks into the new box, power it on, boot from the primary path (or enter the appropriate path for LDEV 1) and at the ISL prompt do a START NORECOVERY [NOSYSSTART] GROUP=CONFNEW.

You might also need to adjust NMCONFIG with NMMGR as a different system will probably have the LAN cards at a different physical path. In case you make a copy of NMCONFIG, make sure it stays on LDEV 1 (FILE;DEV=1).

We replaced LDEV1 on our HP 3000, did an INSTALL. Then we did
The MPE-filespace was restored normally, but the HFS filespace was not. RESTORE told me to use CREATE=PATH which I then did. The DIRECTORY is on my STORE tape - so what went wrong?

Wolfgang Kinscher, and Gilles Schipper reply:

If you are planning an install keep this in mind: Do not specify the option "DIRECTORY" with a partial (DATE>=) backup. STORE will not put the HFS directory structure on tape. Do the following instead: STORE ;*T;DIRECTORY stores all of your MPE/HFS directories on tape. Then do your partial backup without DIRECTORY option

After the install, first restore your DIRECTORY with RESTORE *T;;DIRECTORY and then restore your partial and your latest full backup respectively.

Posted by Ron Seybold at 09:13 PM in Hidden Value | Permalink | Comments (0)

March 10, 2017

Friday Fine-Tune: Going Beyond JBOD

By Gilles Schipper

Mod 20sOne of the most cost-effective ways of advancing the reliability of your legacy system may be to replace your existing “JBOD” disk system with a much more reliable disk system. MOD20 units, still a better deal than individual disks, can provide a good starting point to implement RAID. JBOD is an acronym meaning “just a bunch of disks” — which would characterize the majority of HP 3000 systems as they were initially sold. JBOD disk systems comprise a set of independent — typically SCSI-connected — disks, which are each seen by the HP 3000 as a single logical device number or LDEV. Each disk LDEV is associated with a “volume set” and the failure of a single disk renders the “volume set” to which it belongs inoperable and un-accessible.

Traditionally, most 3000 systems have comprised a single volume set (specifically, the required SYSTEM volume set, with the brevity-challenged label “MPEXL_SYSTEM_VOLUME_SET”).

Systems comprising a large number of “JBOD” LDEVs increased the likelihood of system downtime, since the failure of a single, old disk effectively resulted in a “down” system — requiring a time-consuming disk replacement and system reload before the system could properly function once again.

To mitigate such delicate exposure to a single disk failure, many installations implemented the “User Volume Set” feature built in to MPE/iX, then constructed multiple volume sets so that the failure of a single disk affected only the volume set to which it belonged.

For practical purposes, the only real benefit to this approach was to reduce the amount of time required to replace the disk and reload only the data residing on the affected volume set. (In reality, it was usually quite unusual for a system to continue normal, or even minimal operation with even a single unavailable volume set).

To further improve system reliability and minimize downtime an optional, additional-cost software  product was available in the form of software mirroring — aka “MPE/iX mirroring.”

This enabled the system administrator to configure non-system volume set disk drives to be associated with  identical corresponding “mirror” disks. The software was responsible for dynamically duplicating the contents of  both disk drive “mirrors” such the failure of one of the two mirror drives could be tolerated without affecting the continuous operation of the system. The damaged disk could then be replaced and the dynamic disk duplication would resume.

Only if both mirror pairs failed would there be a corresponding system outage and data loss. However, software mirroring was still far from ideal. Since it was unavailable for the MPEXL_SYSTEM_VOLUME_SET, the failure of a system disk, unprotected by mirroring software, would result in certain system down time.

Further, software mirroring exacted a price in terms of CPU and I/O overhead that could otherwise be utilized for actual “useful” processing. 

And, as a wise person once said, given a choice, a feature is almost always better implemented in hardware than software. This certainly applies to disk mirroring and nicely aligns with the the Nike MOD20 RAID disk system, which is (one of the) HP 3000’s solutions to the compromises associated with software mirroring.

The MOD20 features dual controllers, duplicated (even triplicated) power supplies, and up to 20 disk drives housed in a single frame/enclosure that provides significant improvements over the MPE/iX software mirroring functionality.

Each MOD20 provides for a maximum of 8 logical units (LUN’s) to be configured — each of which appears as a single logical device no. (LDEV no.) to the HP 3000. A maximally and optimally configured MOD20 will include 20 disk drives and be configured as follows:

14 disks to be defined as type RAID1, using up 7 LUNS—since each LUN comprises two separate mirrored disks. RAID level 1 is equivalent to simple mirroring whereby one disk is dynamically maintained as a duplicated mirror image on its mirrored twin disk, which must of identical size and model.

If one disk of the mirrored pair fails, the other disk can take over the responsibility of presenting the requisite data IO to and from the host system with no perceived performance degradation. The remaining 6 disks can be configured as a single LUN comprising 4 RAID 1/0 disks and 2 hot spares.

A RAID 1/0 configuration takes an even number of disks and duplicates the contents of half of them (as a group) onto the other half.

The hot spares would act as dynamic replacements for any disk in the MOD20 that fails, such that even the  failure of one or two disks would not prevent the entire disk subsystem from maintaining its fail-safe mirroring capability. Without the hot-spare feature, failure of a single disk would allow normal system activity to continue but without further fail-safe capability for the failing LUN only.

Chances of both disks in the same LUN failing are extremely remote. That is why I advise you to forgo the hot spare capability. Utilize a 6-disk RAID 1/0 LUN instead of a 4-disk RAID 1/0 LUN, giving you additional usable disk space overall.

Posted by Ron Seybold at 08:04 PM in Hidden Value, Homesteading | Permalink | Comments (0)

March 03, 2017

When and how to back up 3000 directories

Editor's Note: Homesteading Editor Gilles Schipper weighs in on  using the 3000's store directory option, rather than invoking the buldacct program, to make clean backups.

By Gilles Schipper
Homesteading Editor 

CityDirectoriesIt's common to see confusion surrounding the use of the ;directory store option versus the buldacct directory creation program. In order to benefit from the store ;directory option, one has to utilize the option almost perfectly in both the store and the restore following a system INSTALL. Consequently, it becomes much easier to fall back on the buldjob options to re-create the directory -- although that option is inferior.

In order to be able to effectively utilize the directory option, the first thing that must be done properly is to ensure that the appropriate ;onvs= option is also used in the case where user volumesets are utilized. Otherwise, the non-system volumeset directories do not get restored after the INSTALL since they are not on the tape.

But even if the store part is done correctly, the other opportunity to go wrong presents itself during the reload process.

The proper procedure during reload is as follows:

1. perform INSTALL
2. restore ;directory from tape
3. re-create disk and volumeset environment via VOLUTIL

Then -- and this where many go wrong,

4. Again restore ;directory from tape (this re-creates the volumset directory environment on the master volumes for all user volumesets for those utilizing it)
and then
5. restore files
6. reboot with start norecovery (to enable network functionality)

Of course, for those that do not utilize user volumesets, the directory option becomes much less error-prone. And, for those that utilize third-party backup utilities, the ;directory option -- as utilized in the MPE store command -- is generally replaced with a similar option in the various backup utilities.

The bottom line: for those that utilize the MPE store command to perform their backups, the properly-used ;DIRECTORY (and, if appropriate) corresponding ;ONVS= options) -- together with correct restore procedures as indicated above -- will get the desired result 100 percent of the time. Notwithstanding, of course, any tape issues occurring, which would be problematic no matter which directory re-creation option is used.

The bottom line is that the proper way to perform a full backup if you're using the MPE backup facility:

:file t;dev=tape
:store /;*t;partdb;directory;onvs=mpexl_system_volume_set,big;

Of course, upon further reflection, and better than the store command: use sysgen to create a backup tape that not only contains all files, but also the SLT -- so that this one tape alone can be used to INSTALL and RELOAD your system. The use of sysgen for such purpose will require use of an indirect file.

Posted by Ron Seybold at 09:12 PM in Hidden Value, Homesteading | Permalink | Comments (1)

February 24, 2017

Friday Fine-Tune: Opening Up MPE's Shell

Way back in the middle 1990s HP added the Posix shell to the HP 3000. The improvement meant customers who had Unix and MPE running in the same shop could train operators and managers with a single set of commands. Posix was a plus, making the 3000 appear more Unix-like (which seemed important at the time).

Over the years, however, Posix has been a feature waiting be discovered for most 3000 managers and operators. The computer's operating system was renamed from MPE/XL to MPE/iX just for this added Posix feature. But enough history; Posix is still on the 3000 and remains a powerful interface tool, an alternative to the CI interface that HP created for the system. You can even call Posix commands from the CI, a nifty piece of engineering when it can be done.

That's not always possible, though. A customer wanted to know how to "expand wildcard shells" using Posix. He tried from the CI and had this story to relate.

ls: File or directory is not found

So how do I do this? I need to be able to tell tar to archive all of the reels of a STD STORE set via a regexp. It does not work in tar, and it apparently does not in ls—so I speculate that there is something special about the innovation of Posix utilities from the CI that I am not aware of. What is it?

Jeff Vance, the 3000 CI guru at while at HP, replied "Wildcards on most (all) Unix systems, including Posix implementations, are done by the shell, not the individual programs or in-lined shell commands, like ls in your example. A solution is to run the shell and execute ll from within.

The magic Posix shell command to do the expansion:

Screen Shot 2017-02-27 at 11.22.22 PMAn interesting footnote if you've read this far: The Posix shell for the 3000 is one part of the operating system that was not built by HP. The shell was licensed by HP from MKS, and Hewlett-Packard paid royalties to MKS so Posix could work inside of MPE/iX. That was an issue that posed a potential snag for source code licensing from HP. But the outside license issues never ended up blocking emulation or source-license arrangements. Managers have used Posix on the 3000 as a way to get familiar with commands in Unix systems. In the great majority of instances, these commands are the same.

Posted by Ron Seybold at 11:10 PM in Hidden Value, Homesteading | Permalink | Comments (0)

February 03, 2017

Fine-tune Friday: Care and feeding of UDCs

Screen Shot 2017-02-06 at 1.53.00 PMMercury Insurance is a long-time HP 3000 shop still running a server in production. Last week Reggie Monroe reached out for a refresher on administration of HP 3000 User Defined Commands (UDCs). These are the HP3000's equivalent of scripting in Unix environments. UDCs are a better version of Command Files, according to Jon Diercks and his MPE/iX System Administration Handbook. UDCs are catalogued, Diercks says, so they can be loaded for individual user accounts.

UDC definition
Click for details

There's a superior PowerPoint slide deck online at the 3K Associates website that covers how to create and use UDCs. But the Diercks book (no longer in print, but available online) is more concise on the use of UDCs. It's also only available as an $80 book today on the used market; put yours in a safe place. Monroe's question asked about "a command to list all users, and the logon UDC associated with them, if one is set."

The initial answer was the command HELP SHOWCATALOG,ALL. This brings an administrator to

SHOWCATALOG [listfile][;USER=username[.accountname]]

But Alan Yeo pointed out that the MPE/iX command only locates system-level UDCs. 

You don't actually get what you think you asked for, so whilst :showcatalog ;user=@.@ sounds very hopeful, in fact it only shows the system level UDCs not account ones. As far as I'm aware the only place you can find them all is in the BULDJOB2 file in PUB.SYS. You do have a BULDJOB2 file don't you? And it's up to date?

And here's where Vesoft's utility does a job the 3000's OS cannot. VEAUDIT LISTUDC @.@ finds UDCs of all kinds.

We have chronicled much of MPEX during the 21 years of the NewsWire's publication. The utility was even the sole subject of the Inside Vesoft column back in the era when HP was starting to lock down 3000 futures. In 2002 Steve Hammond illustrated the distinction of UDC administration under VEAUDIT. It becomes important because security on a 3000 includes management of the UDC catalogs. And yes, there's a tool for the security, too.

VEsoft’s Vladimir Volokh told me he had been asked to find out if any users on a system had the VEsoft utility GOD in one of their UDCs and if it had the lockword embedded in the UDC. He gave me a series of two commands that did the trick and they had some added value to boot. Once I saw the commands, I was impressed with the simple elegance, but like a good programmer, I had to deconstruct it, break it down and reassemble the whole thing. If you’d like to play along, you need: MPEX, VEAUDIT (both available from, who else, VEsoft) and a healthy programmer’s curiosity (you’re going to have to provide that yourself).

The details of the exercise show "GOD.PUB.VESOFT’ [is found using VEAUDIT] and we have accomplished our mission. But wait. What are those CIERR907 files? Those are files in the list that don’t exist! But they are UDCs that have been set! Looks like you can do some housecleaning and those UDCs can be un-set. How about that — you got some value added, you killed two birds with one stone, (insert your favorite cliche here). Time to play system manager again."

When you add a third party tool to your administrator's box, you can make a purge of such files foolproof. MPE/iX cannot select to show a complete set files by attributes such as program capability. Or for that matter, by last accessed time, or file size, or file security. It's a long list of things that MPE makes an administrator do on their own. Missing something might be the path to looking foolish.

VEAudit and MPEX will root out UDCs and do a foolproof purge, including file names. VEAudit will list all of the UDCs on a server, regardless of user -- not just the ones associated with the user who's logged in and looking for UDCs. The list VEAudit creates can be inverted so the filename is the first item on each line. Then MPEX will go to work to do a PURGE. Not MPE's, but a user-defined purge that looks for attributes, then warns you about which ones you want to delete, or would rather not.

Posted by Ron Seybold at 11:49 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 27, 2017

Friday Fine-Tune: Memory and disk behavior

By Jeff Kubler
Kubler Consulting

Hard-disk-headAlong with the relationship between your CPU measurements and overall performance, memory and disk make up the other two components of your HP 3000 performance picture. Main memory is the scratch pad for all the work that the CPU performs. Every item of data that the CPU needs to perform calculations on or updating to must be brought into main memory.

The CPU must manage memory. It must cycle through the memory pages, marking some as Overlay Candidates (this means that new data from disk may be placed here), noting that some are in continued use, and swapping others out to virtual or what is called transient storage. Swapping to disk occurs when data is in continued use but a higher priority process needs room for its data.

To accommodate this higher priority process and its need for memory space, the Memory Manager will swap the memory for the lower priority process out to disk. The more activity the Memory Manager performs, the more CPU it takes to do this. Therefore it is the percentage of CPU used to manage memory that we use as a measurement.

Page Faults per Second

A Page Fault occurs each time a memory object is not found in memory. The threshold for the number of Page Faults per second that can be incurred before a memory problem is indicated varies with the size and the power of the CPU. Larger machines can handle more Page Faults per second while a smaller box will encounter problems with far fewer. We have found that the number of Page Faults per second a system can endure without problem rises with the relative performance rating of the machine.

An exceptional number of Page Faults should never be used as the sole indicator of memory problems but when observed should be tested with the memory manager percentage. If both agree, you have a memory shortage. There are some strange things that I have observed with Page Faults, so it does not stand alone as an indicator of memory shortage.

The number of Page Faults per second and the amount of CPU needed to manage Memory are always evaluated in conjunction with each other. That is to say the high Page Fault Rate will not be considered a problem if the Memory Manager Percentage is not above 4 percent.

Disk Environment

The Disk Environment is referred to as Secondary Storage. This is where all the data needed for system use is stored. Since Main Memory is not large enough to store all of the data that will be needed by all the processes, there must be a location for this larger pool of data. Even though the Disk Environment does not have the significance it once had, this area can still be a bottleneck. As the CPU speeds increase, bottlenecks become more significant.

Several different factors can affect the Disk Environment. One of these is data locality. Data locality pertains to two different types. There is data locality within Image datasets and data locality across the disk itself.

Data locality across disk: This refers to the location of separate pieces of files (called extents). When files are placed on the disk, they can be placed in contiguous sectors or sections of files, or they can be placed in non-contiguous locations or even on many different disks. When files are not in contiguous locations they are said to be fragmented. The advantage of contiguous location is that greater efficiencies are allowed in retrieving data. When files need to be read, the head movement of the disk drive is minimal if files are in contiguous locations. The head moves to the location and the retrieval begins.

As the disk fills up the system cannot find one contiguous location to build any new file. Therefore, the system breaks the file up into extents and places the file wherever it can. A system reload will put files back into contiguous location (usually back on the location of the files file label) or products such as Lund Performance Solutions De-Frag/X can be used to put the files back into contiguous location.

Operating systems allocate disk space in chunks as they create and expand files and transient disk space (swap areas, etc.). When files are purged, these chunks are released for reuse. Over time the disc space may end up fragmented into many small pieces, which can slow the performance and the reliability of the system.

To observe and correct MPE fragmentation, you can use the De-Frag/X product from Lund Performance Software or use the Contigvol command of MPE/iX's Volutil program. The latter creates contiguous free disk space on a volume. Contigvol work about as well as VINIT CONDense did -- that is, it's stable and reliable, but requires multiple passes to get the best results.

Data locality within IMAGE datasets is the other area of major concern. There there are two different types of datasets to be concerned with, detail datasets and automatic or master sets.

The Detail Datasets

This type of set holds the day to day data input. Detail sets begin with nothing in them. When records are added 1 is added to something called the high-water-mark, a number that tells how many records have been in the set, and the record is placed in the set.

The problem is that IMAGE automatically reuses space that is given up when a record is deleted. This space is often called the delete chain. New records are placed in the most recent location available on the "delete chain." This means that new records are not in the same physical locality as the rest of the records and may be far removed from the other records.

The ideal state for a detail database is one where the detail entries are sorted by the key field. This allows the data to be retrieved in the smallest amount of IOs making efficient use of the MPE systems pre-fetching of data. When this is not the case we can measure the dataset lack of efficiency with something called the Elongation factor. This is simply a measure of how many more IOs the user must perform to retrieve desired data.

The Master Datasets

These have unique identifiers (field names). There are two types of master sets, a manual master and an automatic master set. Manual masters have user-entered master entries while automatic masters have automatic entries placed in them to accommodate access to detail records. The issue of importance to performance here is something called the hashing algorithm. This is the method used by the database to calculate the location of the next record placed in the database. The intent is to cause the master set to be as equally distributed as possible.

The hashing algorithm uses the size of the set in its calculation. A poor size or a size that is not large enough will result in an unequally distributed database. A poor size is most easily described as one that does not consist of a prime number. This means that when the hashing algorithm calculates a location there is a higher potential that a record will already exist in that location. When this happens a secondary position must be calculated. When secondaries are placed in another block within the database, another IO must occur to retrieve needed data. Since IO to disk is the slowest type of access, we want to avoid this at all costs.

Posted by Ron Seybold at 07:14 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 06, 2017

Friday Fine-Tune: Logging, IP logins, SNMP

Due to a disk crash, I had to reload my HP 3000 system recently. I’ve just discovered that system logging has been suspended. How do I resume system logging?

Paul Christidis replies:

The reason for the suspension of logging was most likely due to a duplicate log file name. When the SLT was created the then-current log number was recorded, and when you restarted the system from your most recent SLT it tried to open the sequentially next log file. Said file already existed.

  • MOVE the existing log files to a hold area
  • Determine what logfile the system resumed on
  • Perform a series of SWITCHLOG commands until the logfile number advances to one more than the highest number in the hold area
  • Then move the held logfiles back to the pub.sys group — replacing the ones created by the series of ‘switchlog’ commands.

Is there a way to see the IP address associated with a particular login?

Any user with SM can do the following, for example:


The command :listf ci.pub.sys,8 will list all sessions and will show their associated IP address.

I’ve got an older model HP 3000 and I'd like to start monitoring it with SNMP for things like CPU utilized, jobs running or whatever other cool stat I can SNMP-grab. The problem I have is I can’t find the MIBs for it anywhere.

Andreas Schmidt replies:

First of all, I do not recommend the use of SNMP on the 3000, for performance but also security reasons. SNMP is not the securest protocol, as you know. Nevertheless, here are some hints:

• In the group NET.SYS you will find the SNMPUDC. This should be set in any case for MANAGER.SYS or on system level.
• Having set this, a SNMPCONTROL STATUS will show you the status of the SNMP subsystem.
• SNMPCONTROL START / STOP are self-explaining.
• The MIBs specific for MPE can be found in the document HP SNMP/XL User’s Guide

Posted by Ron Seybold at 06:53 PM in Hidden Value, Homesteading | Permalink | Comments (0)

December 16, 2016

Friday Fine-Tune: Login recovery strategies

We have changed passwords on MANAGER.SYS—and now we cannot locate the changed passwords, thanks to some staff reductions that made the new passwords unavailable. Any ideas on how to recover them?

John Stevens says

If you're logged on as OPERATOR.SYS, do LISTACCT on all the accounts that may have SM capability, then logon as the MGR/MANAGER of those accounts: TELESUP, and SUPPORT, and there are others; LISTACCT will find them. Login as those users (unless you don't have those passwords either) and LISTUSER MANAGER.SYS. Vesoft's MPEX might help ease some of this as well.

Duane Percox of QSS adds a simpler approach:

If you can log onto operator.sys:

file xt=mytape;dev=disc
file syslist=$stdlist
store command.pub;*xt;directory;show

Using your favorite editor or other utility search for the string: "ALTUSER MANAGER SYS"
You will notice: PAS=<the pwd> which is your clue.

Steve Ritenour suggests that a logon to the TELSUP account will unlock the passwords.

Some 3000 managers believe the subject itself should be filed in a place not easily found. "These responses are all well and good," said Bruce Collins of Softvoyage, "but shouldn't we be thinking twice about posting this kind of information (i.e. how to hack an HP 3000) to the 3000 newsgroup?"

Bill Lancaster disagreed. Secrecy about password recovery is not really a secret, he said.

A Google search with the right words will yield far more dangerous information about the 3000 that anything in this thread. The genesis of this information being on public networks came through BBS’s in the 1980s. I’m afraid the barn door is already open.

Can the password for MANAGER.SYS be reset?

Gilles Schipper says

Not easily. If you can log on as operator.sys, you should be able to store off the system directory to tape, as follows:

:file t;dev=tape
:store command.pub;*t;directory

Now that you have the directory on tape, you should be able to look around with fcopy (and the ;char and ;hex options) to find passwords for manager.sys.

John Stevens says

If you're logged on as OPERATOR.SYS, do LISTACCT on all the accounts that may have SM capability, then logon as the MGR/MANAGER of those accounts: TELESUP, SUPPORT, (I can't think of others right now, but LISTACCT will find them). Login as those users (unless you don't have those passwords either), and LISTUSER MANAGER.SYS MPEX might help ease some of this as well.

Posted by Ron Seybold at 08:40 PM in Hidden Value, Homesteading | Permalink | Comments (0)

December 09, 2016

Friday Fine-Tune: Driving Filesystem Checks

Space_odysseyIn the middle of a full backup, the HP 3000 at James Byrne's shop came to a system halt at 3 AM. It was the kind of halt that puts up those puzzling abort messages not even HP has fully documented. For example, about SA 1458, Robelle's Neil Armstrong said, "My experience with SA 1458 is that it is a catch-all abort. You need to look at the subsystem information and the only way to truly know the root cause would be to get a dump and analyze it." He referenced a webpage that breaks down the process of doing 3000 system failure analysis, too.

When a halt occurs during a backup, there's always the chance the 3000's filesystem has been injured. "I'd say run FSCHECK.MPEXL.TELESUP and check your filesystem," said Keven Miller of 3k Ranger. He added that a former HP support expert, Lars Appel, "instructed me that System Abort messages are in subsystem 98. From the MPE Error Messages Volume 2, Chapter 4, System Aborts, 1458 MESSAGE means A critical process is being terminated due to a trap."

Sure enough, power interruptions at Byrne's shop introduced damage to an Image database. 

We reached the point last summer were we were toying with going off-grid simply to avoid the repeated power interruptions. If this sort of thing is causing damage then we will have to consider it. And it seems to; we now have a broken backward chain in one of our Image databases. A thing that I cannot ever recall. Coincidence? We are doing a backup, and then I will be using Adager to go in and take a look.

FSCHECK is an included tool on the 3000. It's simply there to validate extents and scan the table cache for missing files. Better tools include not only the legendary Adager, but independent support suppliers for the 3000 owner. People who know the fast commands of tools like CSTM. "What does your support provider say?" asked one support vendor. Self-support can be backed up by 3000-L questions. Some of the advice about the halt even ran to looking at memory issues. A provider can help eliminate these possibilities.

Capricious power service from his utility — a government service that's been privatized — has extracted a price on Byrne's Series 918. 

I am simply trying to find out if there is any way of examining whether or not we actually have a failing drive. We have spares but if there really is no need then I would rather not take the system down again after such a short interval. It has been a bad fall for our poor old 918. The system HDD was toasted by a whipsaw set of power outages on October 11; now our data disc is suspected of being ready to let go as well.

While running FSCHECK, Byrne was advised to use the commands 

Check all Dev=all
FSCOUNT / 10000

He also received instruction from Mark Ranft on how to scan logs using CSTM to find disk errors.

Sign on as Manager.SYS. Do a LISTF LOG####,2 to find the start and ending Log files. Alter the log file number range in the commands and enter the commands in LOGTOOL.

list log=3404/3477 type=111 "device class"="hard disc",da,ca,"bus converter" out=LogOut1

list log=3404/3477 TYPE=111;'MGR CODE'= 241,242,900,901,951 out=LogOut2

The results are written to two files LOGOUT1 and LOGOUT2.  I had this set up to run weekly on my systems.  And if the files had errors in them, the job would email the results to me for review. You will see errors due to SCSI or FC resets on every boot, so check the timestamps to tell if the boot caused the error.

At Byrne's shop the Series 918 was recovered after Adager did its repairs. "During the Adager repair an infrequently occurring high-pitched but low volume sound was noted," he said. "This appeared to emanate from the 3000. We have not heard it since the Adager repair completed. A suspicion arose that we might have a disk about to lose a bearing."

Once the recovery was complete and the disk replaced, the trademark wisecracks of long-time 3000 vets began to arise. The sound during Adager's repairs "would have been the sound of the chains being dragged around the disk to put them back straight," said Alan Yeo, "and possibly the sound of a very tiny virtual Alfredo welding a few broken links back together."

Byrne noted that the problems with the Series 918 disks "started I was working on installing the 9.6 version of PostgreSQL on a new FreeBSD host. I wonder if the HP 3000 is throwing a temper tantrum? Naaah. Cannot be."

Posted by Ron Seybold at 09:20 PM in Hidden Value, Homesteading | Permalink | Comments (0)

December 02, 2016

Friday Fine-Tune: Print classes, STORE, FTP

I need to add classes to a printer on my HP 3000. I keep getting a message that the LDEV doesn't exist from SYSGEN's I/O section when I issue the ACLASS command. I can see the device exists, so what's the problem?

Letterpress-2Devices on your system that are configured through NMMGR must have their classes changed through NMMGR, which is probably why you're getting the error message. Run NMMGR and go to the PROFILE for the device. You should be able to add classes to the device at that point. After making changes in NMMGR, remember to VALIDATE these changes to NMCONFIG, and then cross-validate using the RDCC command from SYSGEN's SYSFILE section.

NMMGR changes take effect after the system has been reSTARTed and the power has been cycled on the DTCs. You should cut a System Load Tape (SLT) making these changes so you'll have them on tape for your next UPDATE.

If your device was configured through SYSGEN, the device should have its classes modified through SYSGEN. You're right, you do use the ACLASS command in the IO section. Changes to the IO section require a START to make them take effect.

In HP's MPE/iX manuals I am are directed to backup the system as follows:


What is the practical benefit over


People trot out the reason from the distant past: if you have to do a re-install it is good to have PUB.SYS (and maybe the TELESUP account) on the front of the tape so you can restore quicker.

Gilles Schipper woke us up with his response. “The reason one would perform the former over the latter is to ensure that all of the files residing in the SYS account be placed at the beginning of the backup. Whether that is a good thing is, in my opinion, dubious."

Gilles said, “Perhaps, in the days of half-inch, 1600-BPI magnetic tapes, it could provide for a quicker restore of files in the SYS account than, say, files in the YAHOO account. In any case, with tape technology, where retrieval of any group of files from any spot on DDS or DLT tape is quick, the raison-d’être for such file placement strategy disappears - assuming there ever really was a good reason.”

If you have user volume sets you MUST explicitly list them after the DIRECTORY keyword or else the directories for the user volume sets will not be stored.

Is there a way to use FTP to move an IMAGE database between two HP 3000s?

FTP by itself won't work, as it doesn't support user-labels, which are included with the IMAGE root file. DSCOPY, while not being as trendy as FTP, does know about user labels. The Web has freeware LZW compressor programs which know how to archive IMAGE datasets, so you can use FTP without having to change the file codes to non-priv values beforehand. Just archive the sets into one non-priv package, FTP it as a binary file, and then use LZW to re-expand the data-sets back to their original form. LZW programs, along with many other open source gems, are available at SourceForge

Posted by Ron Seybold at 04:22 PM in Hidden Value | Permalink | Comments (0)

November 25, 2016

Friday Fine-tune: Adding disks and IP blocks

Is it possible to add a disk drive "on the fly" without doing a reload?

Jeff Kell replied:

You generally have to shut the system down to install and cable the disk to avoid electrical/interface problems. The usual approach is to use SYSGEN to configure the new device on the path where it will reside, keep the new configuration, shutdown the system, install the disk and do a START NORECOVERY.

Once the disk is recognized by the system, you can add it to your running configuration as follows (assuming the new drive will be LDEV 5 in the system volume set):

> newvol mpexl_system_volume_set:member5 5 90 90 (DISC,SPOOL)

[For details, see "Volume Management", HP Part No. 32650-90045 or "Performing System Management Tasks" HP Part No. 32650-90004.]

This will add the volume to the system volume set, but it also has some side effects. Since the new volume is "empty" and the disk space allocation routines attempt to "balance" loads across drives, all of your new files and transient space will be allocated on the new drive until it's capacity approaches that of the other volumes. This will create an I/O bottleneck on that drive, at least initially.

You could selectively :RESTORE certain accounts (or the whole system) to try and balance the allocation. You could also perform an INSTALL and a :RESTORE for better efficiency, but at the cost of a great deal of time. There are also certain third-party utilities that will balance disk utilization across members of a volume set. These utilities work online on a running system and don't require any downtime.

The network configuration of our HP 3000 was originally set up with one block of IP addresses. Now I need to add another block of addresses. Where do I add these in NMMGR?

You can add an IP address using NMMGR the following way:

  1. After typing NMMGR, select "Open Directory" .
  2. Then select "Update Dir."
  3. Now select the "Add" option (F5.)
  4. You are placed in a screen where you can enter the IP Address of the machine. The type is generally set to 1( IP).
  5. Now press the "Save Data" (F6) option, back out of NMMGR, and you are done.

Posted by Ron Seybold at 09:00 AM in Hidden Value, Homesteading | Permalink | Comments (0)

November 23, 2016

Mailing news from the HP 3000: an old skill

Blue mailboxInternal mail hosts remain a crucial tool in datacenters, even some running MPE/iX. "You still host your own email?" is not a question you'd only pose to a crazy manager. An organization's security standards can be so high that no outside mail server will be trusted. In the earliest days of email, 3k Associates built and sold a beautiful native MPE mailing system, Netmail/3000. It's a smart mailserver, meaning it doesn't require that an organization's e-mail be piped through an Internet provider's mail server for final delivery. Then in the late 1990s, HP's lab started the long process of porting sendmail to MPE/iX.

Now some 3000 sites are looking at how to replace their 3000-based mailing software as they migrate. One of them contacted us this week to ask about an alternative to sendmail. Linux is their migration target, after a history using the 3000 that goes back to the days of HP Deskmanager. Tim O'Neill shared a story while asking about an alternative to sendmail.

I saw that FreeBSD Unix has its version of sendmail. Seeing reference to FreeBSD made me recall a story about FreeBSD running on an old HP 3000, maybe a Series 70 or an early Spectrum system. I think I have read that FreeBSD is at some sites still running in production mode, as MPE and MPE/iX are. It also made me wonder what the installed base of FreeBSD might be — and how that compares to the installed base of MPE and MPE/iX on old hardware and on Charon hardware.

FreeBSD, like MPE/iX, has some surprisingly large companies using it. You might have heard about one of them called Netflix. Of course the Charon HPA emulator from Stromasys makes every remaining product and archival 3000 a candidate for the kind of longevity we see in FreeBSD.

Sendmail has a colorful history. The Unix Hater's Handbook devoted a full chapter to the software's vulnerabilities; sendmail comes from the Unix heritage, after all. By 2003, HP was still patching sendmail to shut down security breaches, although the breaching wasn't nearly as serious on MPE/iX as on Unix variants including Linux. Sendmail's open source capabilities are now under the banner of ProofPoint, the company that purchased the sendmail resources in 2013.

Sendmail's worldwide release was last updated in 2014. HP announced it was testing sendmail to place in the Fundamental Operating System in November, 2001—a month that's famous in the 3000's history for other reasons. But the software moved along to an 8.13.1 release in FOS. It's only one major release behind the worldwide open source version, now advanced to an 8.14 release. Sendmail also includes encryption.

Sendmail has included encryption facilities since 8.11. That's where security capabilities descend onto the requirements. Encrypting mail is a common feature in commercial hosting solutions. Sendmail/iX sends mail created by and triggered from HP 3000 applications, given enough technical know-how.

There's a robust webpage about the 3000 mail solution that was started by Mark Bixby. He's the engineer responsible for lighting the fire of open source flames at HP. Keven Miller of 3K Ranger has updated and maintained the page and its knowledge about Sendmail/iX. The software itself is in your 3000's SENDMAIL account in a version-specific group named vuuff.

Posted by Ron Seybold at 06:22 PM in Hidden Value, Homesteading, Migration | Permalink | Comments (0)

November 18, 2016

Friday Fine-Tune: Tricks with command files

I'm working on a command file on my HP 3000. Is there any way to have it copy part of itself into a separate (temporary) file?

Jeff Vance replies:

MPE does not support the Unix concept of ‘here’ files, where input data for the command can reside in the same file as the command, except in the case of jobs. But even in a job, you may not include inline data for a script or UDC invoked by that job.

The SPOOKHELP script may be of some use. This single script contains the help text for all of the SPOOK commands plus the code to search for and display that text once HELP xyzzy is entered.

How can we execute a command after a user enters the :bye command in MPE?

Olav Kappert replies:

It is possible to execute many commands after the bye has been entered.  Simply create a UDC (maybe a cmd file) called bye.

The contents of the UDC for the command bye is up to you.  This would be useful if you want to do statistics before the session terminates.

John Pittman adds:

Don’t let them do a bye. We don’t allow any users access to OP system prompt at all. They get a logon no break UDC that runs a menu, and when they end the menu, they get logged off.

Inside that UDC at exit time, we build a string giving user, connection point (LDEV or IP of their PC) connect time, CPU date etc and append it to a log file. Then we know when anybody last used the system, how many users are using different connections, or when different user names are using the same connection point.

Posted by Ron Seybold at 11:10 PM in Hidden Value, Homesteading | Permalink | Comments (0)

November 11, 2016

Friday FineTune: Internal disk plus VA array

Human_pyramidFiberChannel Storage Area Networks and shared tape libraries became popular in the years after HP stopped making its MPE/iX hardware. The HP 3000 supports SAN from the XP series of RAID devices to the VA7100 disk arrays. But how much should you rely on a RAID or SAN device? Internal storage devices might seem to be yesterday's tools, but the modest drive inside an HP 3000 can still be very useful — even if a company has invested in the FiberChannel storage solution of the VA7100.

Moving to the VA solution has great benefits, as reported in a story about using a VA7100 array with the 3000s. But booting directly from a VA array — well, you'll need an N-Class server (native FiberChannel installed) or a very expensive HP A5814A-003 Fiber/SCSI router (if you can find such a thing) to employ VA in the 9x9s.

The Crossroads SA-40 Fiber/SCSI switch will link a VA array to 9xx 3000s. It just won't let you boot your MPE/iX system from any of its drives. Craig Lalley of EchoTech recommends the affordable Mod 20 arrays for boot capability.

Internal drives remain as important as the VA arrays for a Series 9xx HP 3000, or even to the XP line of HP arrays. Even important enough to even duplicate them.

A second bootable disk inside your 3000 can take some forethought, but can be essential to smooth recovery of an LDEV 1 failure. James Killam of HP once reported to the 3000 mailing list, "Keeping a bootable image of MPE on one of the internal drives... saved me once at 2 AM when we lost total connectivity to the XP array the system was attached to and we had some serious troubleshooting to do."

Donna Hofmeister, former OpenMPE board member, has noted:

I did one internal disc and it was expressly for memory dumps. A second internal disc with a bootable image would be wonderful insurance. It would take some planning to be able to manage it all, but there’s no reason why you can’t have two bootable discs. I’ll point out the obvious: if LDEV 1 is internal, and you have a multi-disc system volume, and the remaining system storage is on a disc array — uh, what’s the point? If LDEV 1 fails, you’re toast.

Hofmeister said she's had a drive fail on a VA array. "That array worked perfectly and switched over to the spare without a blink. Given that I had two systems sharing this array, I was more than pleased with how well it worked."

Drive failures are among the most likely of hardware problems using HP's 3000 devices. A second internal drive in a system can make a big difference in recovery time. The other way around drive failures is to make your way onto emulated PA-RISC systems from Stromasys.

Posted by Ron Seybold at 08:47 PM in Hidden Value, Homesteading | Permalink | Comments (0)

October 17, 2016

Rebuilding Systems Faster and Better

I'm looking at how to save as much time as possible in rebuilding an HP 3000's software and directories. My options seem to be using STORE, versus the sysgen tape command "tape store=@.@.@". What's the best way to go here?

Donna Hofmeister of Allegro replies

Construction-loaderUnless your system is small (like a 918 with 8-12GB of disc), you don't want to try to do a full backup via sysgen. If you really do a full backup then I prefer this syntax “store /;...” as it is self-documenting and you know that the Posix files will be backed up as well. (On older releases of MPE, @.@.@ did not back up Posix files <eek>)

You want to make sure that you run 'buldacct' periodically (and routinely). You also want to make sure that you are somehow backing up your directory (store /;*t;directory, for example). Between the two, you have belts and suspenders (for recovering your accounting structure).

On older releases of MPE, you want to make sure that the network is shut down prior to making your SLT tape. And it's still a good idea to have the system quiesced when making an SLT, since everything in the sys account (and .pub.sys in particular) will be locked while the tape is being made. Nothing quite like grumpy users to make your day.

Just as a matter of preference, you should normally do a BULDACCT @ at the beginning of the weekly full backup, then the DIRECTORY option along with ;ONVS=MPEXL_SYSTEM_VOLUME_SET, PRODUCTION_SET, etc as a belt and suspenders approach for that day we all hope never comes.

Mark Ranft of Pro 3K outlined the use of sysgen

Here are instructions for a complete backup of basic MPE/iX system via SYSGEN's TAPE option.  It is best if everything fits on a single DDS (or DDS-2 or DDS-3 or DDS-4) tape cartridge, but it will ask for a second (additional) tape(s) as needed.

Note:  I included a 'second_volume_set' which can be changed or removed

Note 2:  The line below is >80 characters, so you have to know how to create a file so that this does not wrap (or make other adjustments.) 

Step One - Create an indirect file containing the following as a single line... 

@.@.@;onvs=mpexl_system_volume_set, second_volume_set


Step Two - Create this job

:job jsltall,manager.sys;outclass=,1          
tape store=^fsltall.job.sys                  
:tellop END OF JSLTALL --------------  EOJ    

Feel free to add a BULDACCT to this.

Posted by Ron Seybold at 09:42 PM in Hidden Value, Homesteading | Permalink | Comments (0)

October 07, 2016

How to Make a Windows to 3000 FTP Move

Open-windowI need to move a file to our 3000 from a Windows server with FTP and Windows doing the put. The Windows file has longish variable length records, but I would like them converted to fixed length on the 3000. When I tried, this was the result:

PUT C:\Dev\MViewFTP\transdata\AP_HEADER_GW.CSV LMAPGW.IVD;rec=-1024,1,f,ascii,disc=1000;move

200 PORT command ok.

550 The FILE EQUATION STRING option (item# 52) is not in a valid file equation form. (FILE OPEN ERROR -449)

Keven Miller replies

Item #52 refers to HPFOPEN. From the intrinsic manual

52 File equation string:

Passes a character string that matches the file equation specification syntax exactly. (Refer to the FILE command in the MPE/iX Command Reference Manual.) This option allows the specification of options available in the FILE command.

I don't like the trailing ";move" in your command string.  I'd remove that. Also, you have comma after "ascii" and it should be a semicolon, like this


Tony Summers adds

I'm pretty sure ";MOVE"  is only valid if you're using the HP 3000 FTP client. Have a look at page 20 of the link below at TeamNA Consulting, another place where the MPE and HP 3000 manuals live

FTP/iX Manual, circa 2000

If you need to move the file during the FTP transfer,  maybe you need to use "RENAME".  

Posted by Ron Seybold at 06:36 PM in Hidden Value, Web Resources | Permalink | Comments (0)

September 26, 2016

3000-L connects again after a silence

Tin-can-telephoneAs if on cue after our report about its silence, the 3000-L mailing list and newsgroup sprang back to life over the weekend. One problem solved by the 415 readers was how to identify if a store to disk backup is a LZW compressed backup file. A Tracy Johnson report also confirmed that a LISTF,2 can report the time of each LISTF, by writing a specialized job.

Meanwhile, a 15-year-old HP 3000 with network connection troubles got advice from the newsgroup's readers. A Series 969 running MPE/iX 6.0 would not be the first thing you'd choose for interfacing to an internal website. But when a 3000 has data that a user needs over the Web, the server is the place to go.

Trouble started to surface when clients access a webpage which then opens a telnet session with the 3000, grabs the info, and then returns the data to the webpage.

We’ve been getting more and more errors over the last year, culminating in non-stop Could not initialize data in path with TCP, which then blocked anyone accessing us through our webserver. We’ve tried many changes but cannot seem to get past this.

When it locks up, the HP 3000 keeps running but won’t accept any new sessions. Which means our clients can’t run searches.  Which is very bad for us. Sometimes we can stopnet and startnet and it will work for a while, but then the errors start again. Eventually, we have to coolstart to be able to have clients log in.

Mark Ranft suggested "If they are already running in the CS queue, here is the likely cause.  Is there some new monitoring in place? If so, it may not be behaving well on the network.

What happens is someone uses a telnet or ftp connection to monitor whether the network is up on the 3000.  They send the SYN, the hp3000 answers with the SYN ACK, and then the 3000 receives a reset before the connection is complete.  This handshake sequence causes this exact error.

Also inetd and other HP3000 networking improved greatly in later versions of MPE/iX. If they must stay on 6.0, they should at least be sure to install the latest patches.

A third party support company served this troubled user until the support vendor folded, "and the only options we found weren't affordable." Getting the 3000 back up will trigger a revisit of that situation. Any server with critical customer data on it—and doesn't have a support vendor—relies on the largesse of the 3000 volunteers of 3000-L. That mailing list did go without a new message for more than a month, a troublesome response time for a critical server.

Posted by Ron Seybold at 07:44 PM in Hidden Value, Homesteading | Permalink | Comments (0)

September 14, 2016

Dancing the Samba services tune, MPE/iX-style

Samba DancersTen years ago this week we were promoting instructions on how to use Samba better on HP 3000s. Samba is "a group of programs that allows a Unix host to act as a fileserver for Windows platforms," according the MPE/iX documentation rolled out in 1999. The file-sharing and printer sharing software which has been a part of MPE/iX since the 6.0 release "allows Unix-like machines to be integrated into a Windows network without installing any additional software on the Windows machines. Many different platforms run Samba successfully; and there are nearly 40 different operating systems which support Samba." And many more now, a decade later.

HP brought features of Samba to the 3000 in a port called Samba/iX. "It is a solution for those wishing to access HP 3000 disk storage and printers (both networked and spooled from MPE/iX) from common PC client operating systems like Windows." Samba/iX allows access to disk and printer resources of MPE/iX by providing standard SMB file and printer services that are accessible from PC clients and their applications. An administration tool called SWAT makes Samba so much easier to use.

Samba 3.0.22 is distributed by the following MPE/iX base patches. Your independent support provider should be able to help you round one of these up. They've got the latest functionality.

  • SMBMXY6D (BT) for MPE/iX 6.5
  • SMBMXY6E (BT) for MPE/iX 7.0
  • SMBMXY6F (BT) for MPE/iX 7.5

The (BT) stands for Beta Test. HP never cut the 3.0.22 version loose as a general release (GR) version. For reference, the following are GR versions with less functionality.

  • SMBMXG3A (GR) for MPE/iX 6.5
  • SMBMXG3B (GR) for MPE/iX 7.0
  • SMBMXG3C (GR) for MPE/iX 7.5

Even a total 3000 network newbie can get Samba up and running. Samba must be running before you can run SWAT. Here's some useful info when getting SWAT going.

In SERVICES.NET you'll want a line that reads:
swat   901/tcp   # Samba/iX Web Admin Tool

In INETDCNF.NET you'll want:
swat stream tcp nowait MANAGER.SYS /usr/local/samba/SWAT swat
(adjust the path to your SWAT NMPRG)

 If you’re running an older version of Samba, you’ll need to modify ‘/usr/local....’ to point to where SWAT actually lives (and case is important).  The user needs to match the user in your samba daemon jobs. (For many, it’s MANAGER.SYS, for you it may be MGR.SAMBA) When you're connected to your MPE/Samba server through a browser to access SWAT, you'll be asked for a logon and password. This is a good thing.

After changing your services and inetdcnf files, all that you should have to do is give inetd a swift kick (e,g,  :inetd.net -c  ) Check inetd’s $stdlist after doing that and you should see that it brought in the new configuration.

In your browser point to http://xxx.xxx.xxx.xxx:901/ (where xxx.xxx.xxx.xxx is the IP of your 3000). Or you can use the name of your 3000 too.

When updating Samba and Apache config files, some are picky about how their records are terminated. Robelle's Qedit can make the needed adjustments. Be sure to know what version of MPE/iX you have installed, including patches.

Posted by Ron Seybold at 08:55 PM in Hidden Value, Homesteading | Permalink | Comments (0)

August 12, 2016

How to purge UDCs on the HP 3000 safely

Cheshire_catThe software vendors most likely to sell products for a flat rate -- with no license upgrade fees -- have been the system utility and administration providers. Products such as VEsoft's MPEX, Robelle's Suprtool, Adager's product of the same name -- came in one, or perhaps two versions, at most. The software was sold as the start of a relationship, and so the relationship focused on the understanding the product provided for people responsible for HP 3000s.

That kind of understanding might reveal a Lewis Carroll Cheshire Cat's smile inside many an HP 3000. The smile is possible if the 3000 uses UDC files and the manager uses only MPE to do a file PURGE. Of course, PURGE ships on all MPE systems. Using that means you'll have to rebuild the UDC catalog. But even that's not enough.

Stan Sieler of Allegro shared a story about this recently. "We recently encountered a site where—somehow—an HFS filename had gotten into COMMAND.PUB.SYS. You can't delete UDC entries with HFS filenames, nor can you add them. I had to edit the file with Debug to change the name into something delete-able." Then there's the rebuilding of the catalog. Keven Miller has contributed a program that sorts and reorganizes UDC files.

There is a more complete way to remove such things from a 3000's storage. You're careful about this because eliminating UDCs with only MPE might leave a user unable to use the server. That grin that lingers is the UDC's filename. 

User Defined Commands are a powerful timesaver for 3000 users, but they have administrative overhead that can become foolproof using the right tools. These UDCs need to be maintained, and as users drop off and come on the 3000, their UDCs come and go. There's always a chance that a UDC file could be deleted, but that file's name could remain in the filesystem's UDC master catalog. When that happens, any other UDCs associated with the user will fail, too. It might include some crucial commands; you can put a wide range of operations into a UDC.

When you add a third party tool to your administrator's box, you can make a purge of such files foolproof. You can erase the Cheshire Cat's grin as well as the cat. It's important because that grin of a filename, noted above, can keep valid users from getting work done on the server with UDCs. This is not the reputation anybody expects from a 3000.

First you have to find all of your UDCs on a system, and MPE doesn't make that as straightforward as you might think. Using SHOWCATALOG is the standard, included tool for this. But it has its limitations. It can display the system-level UDC files of all users in all accounts. But that's not all the UDCs on a 3000.

MPE, after all, cannot select to show a complete set files by attributes such as program capability. Or for that matter, by last accessed time, or file size, or file security. It's a long list of things that MPE makes an administrator do on their own. Missing something might be the path to looking foolish.

Employing a couple of third party tools from VEsoft, VEAudit and MPEX, lets you root out UDCs and do a foolproof purge, including file names. VEAudit will list all of the UDCs on a server, regardless of user -- not just the ones associated with the user who's logged in and looking for UDCs. The list VEAudit creates can be inverted so the filename is the first item on each line. Then MPEX will go to work to do a PURGE. Not MPE's, but a user-defined purge that looks for attributes, then warns you about which ones you want to delete, or would rather not.

By using MPEX -- the X stands for extended functionality -- you can groom your own PURGE command to look out for files that have been recently used, not just recently created. MPE doesn't check if a purged file is a UDC file. 

Such 3000 utilities provided the server and its managers with abilities that went far beyond what HP had built into MPE and its IMAGE database. Now that MPE is moving on, beyond HP's hardware, knowing these third party tools will transfer without extra upgrade fees is like ensuring that a foolproof MPE will be running on any virtualized HP 3000.

They're an extra-cost item, but how much they're worth depends on a manager's desire to maintain a good reputation.

In the earliest days of the sale of these tools, vendors were known for selling them for the price of the support contract alone. That's usually about 20 percent annually of the purchase price. If a $4,000 package got sold that way, the vendor billed for just $800 at first. It made the purchases easier to pass through a budget, since support at the manager-tool level was an easier sell. Think about it. Such third parties passed up $3,200 per sale in revenues in the earliest days. They also established relationships that were ongoing and growing. They were selling understanding of MPE, not just software.

This kind of practice would be useful for the community's remaining software vendors. This is not the time to be raising prices to sustain MPE computing, simply because there's a way to extend the life of the hardware that runs MPE. As the number of MPE experts declines, vendors will be expected to fill in the gaps in understanding. Those who can do this via support fees stand the best chance of moving into the virtualized future of 3000 computing.

Posted by Ron Seybold at 01:17 PM in Hidden Value, Homesteading | Permalink | Comments (0)

August 10, 2016

Measure 3000 performance for datacenters

Hp-1715a-oscilloscopeMeasuring the performance of an HP 3000 used to be a leverage point for increasing investments. By now the numbers help justify continuing to use the server in a datacenter with newer boxes. "We think of our HP 3000s as stable, and even reducing in usage over time," says one systems manager, "though actually as the company grows, the data requirements and load on the 3000s increases."

One way to measure a 3000's footprint is the amount of memory it requires. Memory upgrades cost nothing like what they did even 15 years ago. But any spending at all makes that 15-year-old server suspect. HP's Steve Macsisak recommended sessions x 4, plus jobs x 16, plus 64 MB as the criteria for memory usage.

An HP 3000 uses as much of its memory as possible to make processing efficient. The design of the PA-RISC architecture makes memory the most important element of performance, after IO speed. It's not that unusual to see a 3000 using 100 percent of its memory, according to field reports. There's also CPU usage to measure. 

CPU percentages can come via the REPORT command. Count up the CPU seconds used in the week, and divide by the total number of seconds available (604,800). But for all of this, it doesn't feel like a graphic report the rest of the datacenter gets from its Unix and Linux systems using SAR. There may be a program inside a 3000 that can help, even if the company never purchased performance tools from Lund. HP's Glance gives away its reporting power in its name, one manager has joked.

PloticusThere's freeware available to create handsome graphs like the one at left, suitable for showing in a meeting about datacenter resources. Ploticus/iX was written by Andreas Schmidt. It uses data from SCOPE.SYS. Ploticus even works with SAR's data.

Since there's no port of SAR for MPE/iX, something else must stand in. Some systems have HP Scope, the software that dives in deep enough to produce report-ready numbers. It's not the smooth path a 3000 gets from Lund's Clearview Performance Manager. Scope is the HP Performance Collection Software sold by the vendor while it still had an active 3000 business.

Scope includes a program, EXTRACT.SCOPE.SYS, that permits the software to EXPORT its results to a text file. The manual for the software says it has three components.

SCOPEXL, if you are using an MPE/iX system), UTILITY, and EXTRACT. SCOPEXL is the performance data collector for MPE Systems. It continuously collects and summarizes performance data. UTILITY and EXTRACT are the host programs that let you interact with SCOPEXL and manage the data that it collects.

HP's Scope documentation describes how to use the collection and management software: how the host components interact; detailed command descriptions for each program; and suggestions on how to use the programs to analyze and archive data efficiently.

Posted by Ron Seybold at 08:45 PM in Hidden Value, Homesteading | Permalink | Comments (0)

July 11, 2016

How to Use Input to Create Output Files

Intrinsics are a wonderful thing to power HP 3000 development and enhancement. There was a time when file information was hard to procure on a 3000. It was a long time ago, as I was reminded by Olav Kappert in a call about his HP 3000 history. "The high point in MPE software was the JOBINFO intrinsic," he said. Kappert started with the 3000 in 1979.

Fast-forward 37 years later and you'll find questions from a different programmer still working on a 3000, adding features to a system. The Obtaining File Information section of a KSAM manual on MPE/iX holds an answer to what seems like an advanced problem.

I'm still using our old HP 3000, and I have access to the HP COBOL compiler. We haven't migrated and aren't intending to. My problem is how to use the characteristics of an input file as HPFOPEN parameters to create an output file. I want that output file to be essentially an exact replica of the input file (give or take some of the data). I want to do this without knowing anything about the input file until it is opened by the COBOL program. 

I'm using FFILEINFO and FLABELINFO to capture the characteristics of the input file, after I have opened it. After I get the opens/reads/writes working, I want to be able to alter the capacity of the output file.

Francois Desrochers replies

How about calling FFILEINFO on the input file to retrieve all the attributes you may need? Then apply them to the output file HPFOPEN call.

Donna Hofmeister adds 

You might want to get a copy of the "Using KSAM XL and KSAM 64" manual. Chapters 3 and 4 seem to cover the areas you have questions about. Listfile,5 seems to be a rightly nifty thing.

But rather than beat yourself silly trying to get devise a pure COBOL solution, you might be well advised to augment what you're doing with some CI scripts that you call from your program.

In a lively tech discussion on the 3000-L list, Olav Kappert added, 

Since you want to do this without knowing anything about the input file until it is opened by the COBOL program, the only way is to use one of the MPE intrinsics to determine all the characteristics of the file in question. Then do a command build after parsing that information.

Michael Anderson added details on how the 3000's CI scripting can build upon the fundamentals of file information and COBOL.

I like Donna's plan.This is a strategy that will also help whenever you want similar functionality on a NON-MPE platform. Also, although COBOL is very capable, an external script might be a better tool. You don't always need a hammer.

This is hypothetical, to try to make a point. From your MPE CI prompt, type HELP FINFO. You should be able to set some variables (SETVAR FILEA "XXX"), and using FINFO add some more variables. Then from COBOL using HPCIGETVAR, string together a BUILD command (with a bigger LIMIT maybe), and call "HPCICOMMAND". You could string the build command from a command, into a single variable, then COBOL only needs to HPCIGETVAR once.

You can also write a script to do everything you want, and call HPCICOMMAND to run the script, pass it parms. It's pretty cool, and it makes your COBOL application more portable. (Same program, different script).

For example: On MPE I once wrote (using COBOL) a small utility to CALL DBINFO, extract all the meta-data from any IMAGE database, and then create, and write to the NEW KSAM COPYLIB, ending up with all the COBOL copylib modules needed for all datasets for any database, including call statements and working storage. My point to all this: I used CI scripting to create and write to the copylib. I actually used ECHO to write the copylib ksam file from a CI script. Now, seeing how I work more on HP-UX and Linux, plus OpenCOBOL and Eloquence, I should be able to compile this same program on Linux with minimal modifications, only changing the external script.

I use this method to access SQL databases, and much more, using OpenCOBOL and the Tcl/Tk developer exchange. This way I can run the same program, same script almost anywhere, no matter, Windows, Mac, or Unix.

Eric Sand, another veteran of the 3000, commented that this kind of challenge really shows off the range of possibility for solving development problems. "You can create almost any cause and effect in MPE that you can imagine," he said. "Reading about your concern gave me a little rush, as I mentally organized what I wanted to do to address your concern."

Posted by Ron Seybold at 09:12 AM in Hidden Value | Permalink | Comments (0)

June 29, 2016

Date format variable help for MPE/iX

What would be the easiest way to get a variable date in the format "06/29/16" on a MPE/iX 7.0 and 7.5?

Michael Anderson replies

First echo:

echo ![str("!hpyyyymmdd",5,2)]/![str("!hpyyyymmdd",7,2)]/![str("!hpyyyymmdd",3,2)]

Then setvar:

setvar mydate '![str("!hpyyyymmdd",5,2)]/![str("!hpyyyymmdd",7,2)]/![str("!hpyyyymmdd",3,2)]'

(Note the usage of single-quote and double-quote in the setvar command.)

Barry Lake adds

Please note that the HPYYYYMMDD variable is already a string variable:

Frodo: calc typeof(HPYYYYMMDD)
2, $2, %2

So you don't have to dereference it with ! inside double quotes. In other words, the following works just as well, is easier to read, and might even execute a bit faster:

Frodo: echo ![str(hpyyyymmdd,5,2)]/![str(hpyyyymmdd,7,2)]/![str(hpyyyymmdd,3,2)]

Also note that since you're asking for the value of HPYYYYMMDD three different times, there's the tiniest chance that if you executed this right at last part of the second at 23:59:59, then the value could roll over to a new date in the 2nd or 3rd call, and you could possibly get the wrong date, or even year. Unlikely, I know. But to avoid that, you'd want to capture the current date once, then operate on that, as in:

Frodo: setvar my_date hpyyyymmdd
Frodo: echo ![str(my_date,5,2)]/![str(my_date,7,2)]/![str(my_date,3,2)]

Stan Sieler adds

Kudos to Barry and Michael for apparently correctly guessing that our manager was interested in a CI-based solution. Next time, you might want to make that clear. I was thinking along the lines of SPL/Pascal/COBOL and the CALENDAR intrinsic, as well as whether I should ask the obvious questions or not. Of course, the easiest method is: ECHO 06/29/16

Any code that wants a coherent date and time using separate sources (e.g., CLOCK and CALENDAR intrinsics, or HPTIMEF (#1)) needs to be aware of midnight crossing. Thus, I use a helper function in Pascal like:

  procedure get_date_time (var cal : calendar_type; val clk : clock_type);
        cal := calendar;
        clk := clock;
     until cal = calendar; 
{avoid problems if ’clock’ called just after midnight} end;

Posted by Ron Seybold at 08:11 PM in Hidden Value, Homesteading | Permalink | Comments (0)

June 06, 2016

SLEEPER, Awakened for the Masses

Sleep-1389978_1280The contributed program SLEEPER is probably the software cited most often as proof of the riches of the Contributed Software Library. First created outside of the IT shops of Boeing (according to its first writer Ray Legault) the program was among the most classic of solutions for straightforward jobsteam management. You wouldn't mistake SLEEPER for something professional like Maestro, or even the free MasterOp. But SLEEPER was contributed to the 3000 community, not cast into the free wilds like MasterOp was after its commercial career ended. It was meant to be shared.

The trouble was, SLEEPER disappeared from the community's shelves when Interex died. The CSL tapes (and eventually CDs) went off the grid, another skid-mark left when the user group careened into the void in 2005. It's been 11 years, though, and it's finally time to at least make SLEEPER ready for a wake-up call. We've got the two simple source files to share.

Nobody has liability anymore for HP 3000 contributed software. SLEEPER was never released with support or a license; it was simply part of being an Interex member at a certain level. And let us take a minute to recall that Interex folded owing millions of dollars to members and vendors. After 11 years, it's time to make this software a community resource once more. "Hey, I know a guy who can get you that" has been the means to share the utility over the last decade.

Surprisingly, it's just a well-packed 184K of SPL and FORTRAN code. MPE magic never took up much space. That's one of the reasons it was magic. SLEEPER is also a fine example of how 3000 managers helped one another.

We've thrown caution to the winds (not exactly a new event here) and loaded those two files onto our website servers for distribution. It comes with no warranty, of course. Like always with the CSL gems, using it carries the same risk that any contributed utility or application did.

The FORTRAN file is available. And the SPL code is at hand, too. One well-honored MPE/iX engineer looked over the software programs and said they were extraordinary.

The SPL is some of the better looking old-style SPL that I've seen. The FORTRAN is very nice looking.

The SPL program runs in the batch job. The FORTRAN program maintains a configuration file that is presumably used by the SPL program.

So let the jobs of MPE/iX, including those running on standard HP hardware as well as those on virtualized 3000s, be managed a bit more easily. I await your comments on rousing the talents of this classic tool.

Posted by Ron Seybold at 10:06 PM in Hidden Value, Homesteading | Permalink | Comments (0)

June 01, 2016

Recovering Your Lost Logins

BWManIn the world of 2016, losing a password can be a time-bending experience. Apple's ID logins for its iCloud services, iTunes, and App Store only bear five attempts at most before they close out access to the phone. From there, you're on to a lengthy call to Apple. The smartphone is a tiny computer. The larger ones are not as strident about refusing repeated attempts, including the HP 3000.

They are secure, though. In these times when the servers can go dark for awhile as cold-start archive systems, though, a login can get misplaced. We're not supposed to write these things down, after all. What do you do when you have a question like the one below?

I restarted a 3000 9x7 after a few years in mothballs to run an old in-house app. I was able to boot up and login as OPERATOR.SYS, but cannot remember or find the password for MANAGER.SYS. Is there anyway to reset, clear, or overwrite the password file? I know the old machine is a very secure one, but now I am hoping there is a way around it.

What follows are a couple of suggestions to get back into the manager's driver's seat. However, you'll need to at least have a ticket to ride in OPERATOR.SYS.

Gilles Schipper shares what he knows.

There's not an easy way around the security. But since you can log on as OPERATOR.SYS, you should be able to store off the system directory to tape, as follows:

:file t;dev=tape
:store command.pub;*t;directory

Now that you have the directory on tape, you should be able to move around with FCOPY (and its ;char;hex options) to find the password for MANAGER.SYS. And that's all assuming you haven't implemented the directory encryption feature available with HP's security product Security Monitor/iX.

There's one other way. It's a good thing, because the alternatives do not include a rescue call to HP. There's nothing like Apple's password recovery support left for the HP 3000, as the vendor has long ago left the field. However, a good support contract for an MPE/iX server can be just the thing, and we know where you can get one.

Posted by Ron Seybold at 05:11 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 18, 2016

Tape drive changer a powerhouse for MPE

Autochanging HensAutochanging tape drives used to be the stuff of science fiction among 3000 managers, but those days passed by before HP cut off making Classic 3000 MPE V systems. Just because an autochanger is a 3000 storage option does not make it automatic to program, however.

A question posed to the community by Ideal Computer Services Ryan Melander reached for help on programmatically controlling such autochangers -- to select a slot, and load the tape and come to ready. "I am trying to configure an old DDS3 auto-changer, one that I don't believe will unload and load the next tape," he said.

Gilles Schipper noted that the command ad ldevno id=hpc1557a path=?? Mode=autoreply configures the device, and to advance tape after use, employ the command (from Devctrl.mpexl.telesup) ldev eject=enable load=online 

DAT tapesDenys Beauchemin mentioned HP's pass-through SCSI driver as a tool to drive the device's robot. The software was built by HP's labs and labeled as "not for the faint of heart" by engineers, but can assert a programmatic control over autochangers. Some third party programs such as Orbit's Software Backup+/iX can also do this work.

If ever there was a theme song for an autochanger at work, it would be a tune called Powerhouse. Children of the Fifties and Sixties will know it as soon as they hear it, if they've ever watched a Warner Brothers cartoon.

Some programming ideas came from Beauchemin, the engineer who developed at HiComp for the HiBack 3000 solution. "For the next tape to be brought online automatically, I seem to remember there had to be a special setting with the dip switches."

As for being able to control the robot itself, you definitely need to have the [HP] SCSI pass-through driver configured and loaded, and then you need a program to actually issue the IOCTL calls to the robot with the properly formatted SCSI commands. There was such a program a long time ago from a vendor, but that's all gone now.

Perhaps the high-test flutes and heavy octane horns of Powerhouse -- used in Duck Dodgers and the 24th and a Half Century -- can be put up on the MP3 player while fitting the driver to MPE. ("Oh drat these computers -- they're so naughty and so complex," says Marvin the Martian in one installment. "I could just pinch them.")

From our archives we found this advice from the late Jack Connor of Abtech, pointing to similar complex answers about controlling DDS changers.

Typically, there's a second SCSI port/address assigned for the transport control which allows the selection of specific tape. For MPE, stacker mode is typically selected, which tells the drive to just mount the next tape in line when requested. I don't know if the DDS autoloaders have a network connection available like the C7145NA DLT autoloaders do; with that device's web interface you can reload any tape, bypass a bad tape, and so on.

Back in 2011, John Pitman checked in to report that a much simpler solution to his changer's control needs popped up. "On re-examining my code for HPDEVCONTROL, I found I had catered for 1- and 2-digit device numbers in the string passed, but I had configured the drive as dev 777. This produced a string dev number of 77, which doesn't exist as a tape drive. Once I fixed this, it works like a treat."

Posted by Ron Seybold at 09:16 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 06, 2016

Options for STORE You May Not Know

StorehouseSTORE is the default backup tool for every HP 3000, but this bedrock of backup has options which might exceed expectations for a subsystem utility. Gilles Schipper, the support guru of three-plus decades on the 3000, told us about a few STORE sweet spots.

To start, using the :MAXTAPEBUF option can cut a four-hour backup to three hours or less. Schipper says that increasing the buffer size default to 32K, from the usual 16K, speeds up the backup when STORE sees MAXTAPEBUF. "That's a pretty good payback for one option."

Backups don't need to be specified with an @.@.@ command to be complete. "People should really be using the forward slash," he says, "because it's easy to accidentally omit the Posix file structure if you're not careful constructing your fileset backup." The slash is so much better that a backup specified by HP's TurboStore will replace any @.@.@ operation with "./" Combining @.@.@ with exclusions can lead to omitting files which should have been in a backup.

Schipper says that including a directory on a backup is smart, but private volumes in use on a 3000 need more than :directory as an option.

"If you're using private volumes and the directory option, you'll only get the system volume directory — and you will not get the private volume directories," he says. The backup must explicitly specify the volumes through the ONVS= command, using the long name of the private volume.

The partdb option on a STORE command ensure that any 3000 databases which are incomplete will get backed up. Without partdb, if a root file "doesn't have its corresponding data sets, the root file won't get stored. It's silly, but partdb ensures they get stored." A privileged file can also look like a partial database, so partdb brings those files into a STORE backup.

A 3000 with HP's TurboStore, rather than just the default STORE, can take advantage of the :online command. "It will give you zero downtime, if you have that version from HP," Schipper says. But that begins to drift away from the no-cost STORE options available to any HP 3000 administrator or owner.

Posted by Ron Seybold at 05:47 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 04, 2016

CPR for a Non-Responsive Console

On my HP 3000, after a short power blip, the console is now non-responsive. I can connect to the system's GSP port and the session is connected, but nothing is displayed. Neither <ctrl> A or <ctrl> B works. I type away, but get no response. I can then connect via VT-MGR and take the console :console !hpldevin and I receive all the console messages.

So, the messages are being sent (since I see them on the VT connection), but neither the physical console or the GSP gets any console messages. What can I try?

Gilles Schipper says

I believe a START NORECOVERY reboot is in order here. Since <ctrl> A <ctrl> B do not work, you will need to power-recycle the machine to effect a reboot. Presumably you would want to do this after gracefully stopping all jobs and asking online users to log off, if possible.

Depending upon which patch level your level of MPE is on, the :SHUTDOWN RESTART MPE command may also work from a logged-on session with at least OP capability.

Mark Ranft adds

If you haven't rebooted, I've seen similar issues. From the VT console can you try to do 'abortio 20' until it says no I/O to abort. A WHILE loop may make this easier. I've had luck with this in the past. But since Ctrl-B doesn't work, you may be out of luck.

Robert Thwaites notes

These are the simplest things to try first

<ctrl>Q (x-on)


Among the commonest issues: forgetting to do an x-on after a <ctrl>S (x-off) to stop output, so you can look at the line you are interested in. One time I saw another issue where someone had pressed <space> on the console and hadn't pressed <return>.

Posted by Ron Seybold at 10:37 PM in Hidden Value, Homesteading, User Reports, Web Resources | Permalink | Comments (0)

April 22, 2016

How to Transform MPE Spoolfiles to PDFs

HP 3000 data becomes more useful if it can be e-mailed as industry standard report documents. After more than two decades of pushing at it, Adobe has made its PDF the de-facto way to exchange documents, even complex ones.

Which might have prompted this question from HP consultant and Suprtool trainer Jeff Kubler:

Does anyone have a lead on a tool that converts spoolfiles to PDF files? Are there any Contributed Library tools?

It won't be in the Interex Contributed Library (because the programs have gone underground; ask your colleagues if they have a swap tape) but the txt2pdf product works nicely to make this conversion. Even in its most advanced version it's about $1,000, at last glance. Bob McGregor reports as much.

Jeff, txt2pdf does this. We have a job that runs that:

1. Checks a pseudo device we have setup for any spoolfiles that are on the device with an PRI >0

2. If it finds a spoolfile, we convert it to PDF and move it to a server

3  Sends an e-mail to the streamedby variable telling them the PDF doc is ready on the server.

4. Alters the priority to 0 to mark it processed

We've been using it for a couple years, and it works great — of course, once we got the bugs worked out. What's cool is if someone delete the file, we just adjust the priority to something greater than 0 and it gets reprocessed.

Posted by Ron Seybold at 08:52 PM in Hidden Value, Migration | Permalink | Comments (0)

April 15, 2016

Use GSP and More for 3000 ID, control

GSP boardThere's been plenty of change in the 3000 manager's life over the last 15 years. Some of the change is a little easier by exploring the use of the Guardian Service Processor. It can help in changing the location of HP 3000 consoles from one part of a shop to another. There's even a story out there that says the identity of the HP A- and N-Class hardware resides in the GSP.

Kent Wallace, while a 3000 manager for Idaho-Oregon healthcare delivery system Primary Health, needed to move his 3000 console.

I must move the 3000 console another 10 feet farther from the rack (it's an N-Class HP 3000/N4000-100-22). What are the 3-pin positions on the wire that I need to extend its RS-232 cable?

Our blog contributing editor Gilles Schipper chipped in with a solution offering even farther movement:

If you want to extend the range of the console to anywhere on the planet (at least where there’s Internet access) you could consider the HP Secure Web Console to replace the physical console. Depending upon the condition of your physical console, this solution may also save a bit of wear and tear on your eyeballs, instead of adjusting those pins.

Schipper wrote us a great article on setting up such a web console. The GSP is quite the tool.

Former HP support engineer Lars Appel offered another take on Schipper's strategy:

While Gilles is right about the possibility of using the web console, it would probably be easier to use the already built-in dedicated LAN port of the N-Class systems that gives access to the GSP by telnet.

I prefer the “telnet console” over the “web console” because it gives more freedom in the choice of terminal emulator — whereas the web console typically lacks features like “easy cut and paste” or special key mappings (e.g. German language ;-) or something similar.

This prompted Schipper to clarify his suggestion:

Lars is absolutely right about the built-in “secure-web-console” that comes with all N-Class and all but the earliest A-Class e3000s.

And, yes, the built-in is definitely more functional, allowing cut-and-paste as well as telnet access, whereas the external variety has only Java access to it via a web browser and no cut-and-paste.

So, if one has a choice, the built-in is definitely superior and available with only proper configuration. However, the external secure web console is available for all HP 3000s, and would still be most useful where is internal secure web console is not an option.

The GSP is a powerful part of an HP 3000 that runs whenever the server is plugged in. It's the maintenance control console commanding the ultimate class of the server to reboot, do memory dumps and even fully power down the 3000. Consultant Craig Lalley has noted the GSP has one fewer feature than its Unix counterpart, though.

"On HP-UX it is possible to reset the GSP from the host OS," he said. "I have not found a way from MPE."

From time to time a reset may be required for diagnostics services on A-Class and N-Class servers. If your 3000 gets loving care from a consultant or service provider outside your computer room, you may need a paper clip to keep up service levels.

The GSP can also reveal the 3000's speedometer, as profiled near the bottom of an Allegro webpage.

The gap between controlling the 3000 and HP's HP-UX Integrity GSPs is a common shortfall of HP designs. Even though the 3000's MPE/iX includes a Posix interface, HP didn't engineer enough Unix into the 3000 to enable some of the administration that HP-UX users enjoy. (That lack of Unix can sometimes be a good thing, however, when a security breach opens up in the Unix world.)

But when a 3000 needs a GSP reset, pressing a recessed button on the 3000's back will do the trick if a telnet command doesn't work. You can telnet to the IP address of the GSP, log in and do the reset. But you can also get someone to press the physical reset button at the back of the machine. It's recessed into the cabinet so you may need a magic paper clip bent just-so.

Lalley calls the GSP, which HP introduced with its final generation of 3000s, one of the most useful things in the A-Class and N-Class boxes.

The GSP is a small computer that is always powered on when the plug has power. With it, it is possible to telnet to and BE the console. While multiple admins can telnet in and watch, only one has the keyboard.

It is possible to reboot, memory dumps and even fully power down the HP 3000 from the GSP. Use the command PC OFF.

It is probably the best feature of the N-Class and A-class boxes. The problem is sometimes it needs to be reset, usually with a paper clip. Since the GSP is a different CPU, this reset can be done during business hours.


Posted by Ron Seybold at 08:35 PM in Hidden Value, Homesteading | Permalink | Comments (0)

March 18, 2016

Big files get zipped, moved on HP 3000s

ZipperA computer manager who's new to the HP 3000 is looking for CSL files this week. The Contributed Software Library is just an oasis to this IT veteran, something shimmering in his future that holds a highly useful thicket of utilities and more.

Someone in the 3000 community is bound to connect our new user with this CSL, for one reason: he's looking for MPE V programs to supplement his discovery of the emulated Classic HP 3000, simh. That's the MPE V-ready version of a virtual HP 3000: what amounts to a CISC skin for a 3000 on top of the simh code. Whenever the newbie connects with a CSL resource, if they've got their files on a 3000 they're bound to need to send about 24MB to him. That's going to require zipping them.

The act of zipping to compress for a transfer is an essential in 3000 management. Although the code for compressing files on HP 3000s is more than a decade old, like a lot of things on the system, it continues to work as expected.

Tracy Johnson, who manages the Invent3K server operated by OpenMPE, noted he's using the MPE/iX Posix shell's compress and uncompress. "It creates a file that ends in capital Z. Seems the compressed format is compatible with both GNU-zip and Winzip programs or any other Unix/Linux machine."

Lars Appel, who ported the Samba file sharing tool to MPE, offers a comprehensive answer. He points to an HP 3000 Web starter software kit that resides on a development server, open to the public.

Lars explained in a post a few years ago:

You can pick up the InfoIP zip/unzip programs (in a tar file) at my area in the Infocorp server. The link in that webpage that contains the zip/unzip programs is


Transfer it to the 3000 in bytestream or (fixed) binary format and then unpack with :/bin/tar "-xvzopf FILENAME". Place the two programs where you like; I typically have them in /usr/local/bin or (with uppercase filename) in a group or directory that is part of my HPPATH settings.

The webpage also contains a tar.Z file with /usr/local/bin/gzip


(gzip -d decompresses; creating a symbolic link gunzip is also useful)

Posted by Ron Seybold at 06:04 PM in Hidden Value, Homesteading | Permalink | Comments (1)

March 04, 2016

3000-L breaks silence with DTC primer

DTC NetworkOn the verge of four weeks without a new message, the 3000-L mailing list and newsgroup delivered a primer on the control of DTCs on MPE/iX networks. A longtime contributor to the community set up a question about using these venerable devices that connected HP's terminals and other devices to a 3000.

I have multiple HP 3000s sharing one DTC.  My problem is, which one controls the DTC?  In the event of  a power cycle, there is a race between them for which 3000 will download the new configuration.  I need system A, my A-Class, to be in charge. However, systems B or C are most likely to download first, leaving me with the manual step of unplugging the network from B and C, power-cycling the DTC, and then waiting for the A-Class to download the configuration. Is there anything that can be done to just leave the A-Class in charge?

Tracy Johnson replied, "It's always been a crap shoot. I'm of the opinion the first HP 3000 to notice the DTC needs downloading will do the job. Which usually means the less busy machine." As it was just the first answer on the newsgroup, there was still a need to do unplugging.

"For example," he added, "if you have a test machine on the side nobody's using, I would simply unplug the test machine's network cable until the DTC was reset. Since there will be no other machine in control, the regular machine will do the download.  When complete, I'd plug the network cable back into the test machine. However if both machines are in production and you have users active on both, you'll have to make a decision."

Lalley came back with an answer of his own, at least to eliminate the travel from chair to server.

"On the DTC's I guess it could be possible to do it without leaving your chair, if you TELNET to the console on B and CDTCCNTRL (has to be run from the physical console)  to stop the DTC subsystem, then restart it."

Mark Ranft at Pro3k had another idea.

Back in the good old days, we used DTC Switching - a feature that allowed a DTC prompt to appear at a terminal.  The user entered 'C HPA' to connect to host HPA.  I believe you could set a default system.  Initially DTC Switching was only available if you configured and downloaded the DTC using OpenView DTC Manager software running on a PC.  Later HP set up NMMGR to allow DTC Switching.

The Communicator for that release may have more details, but I only entered the MAC of the DTC on the one system that was supposed to download the DTC.  The other systems would have the DTC name, but not the MAC.  This allowed the system to have printers and other devices accessible on the DTC without concerns about which one downloaded the configuration.

Of course, the location of that MPE Communicator took all of 41 minutes to dig up. Barry Lake pointed at HP documentation (now hosted outside HP's baffling website) that covers Enhanced Host-Based DTC Management Functionality in Chapter 3. Plus, a manual on Configuring and Managing Host-Based X.25 Links.

Not too shabby for a mailing list that's more than 20 years old, but still manned by dozens of experts. It's the kind of expertise a good third party support provider offers—as usual, better than HP's today.

Posted by Ron Seybold at 06:33 PM in Hidden Value, Homesteading | Permalink | Comments (0)

February 26, 2016

21 days of radio silence on the 3000-L

Right_WirelessTelegraphThe slowing current of 3000 communication showed a fresh signal by the end of this month. As we write it's been 21 days since a message of any kind on the 3000-L MPE newsgroup. The resource that carried 45 messages during last February has 10 for the current month. All of this month's traffic was wrapped around finding resources: Brian Edminster of Applied Technologies and Vesoft support. Both were located.

However, the three weeks without a new message is new territory for the community's log of technical help and outreach by cohorts. Among those who were posting during 2015, several told us they're on the mailing list-newsgroup out of habit — rather than needing details for their datacenter's 3000s.

"I’m still on the list out of inertia, nostalgia and mild interest," said Dave Heasman, a UK IT manager. "My employer got rid of their 3000s and me in 2008. Bought a series of packages to replace a big bespoke brokerage/investment system."

Robert Mills said he "remained a member of the list, mainly as a lurker, to keep appraised of what was happening in the 3000 community. Except for three requests in September 2012, December 2014, and February 2015, I've only posted to the list when I felt that the 3000 knowledge I had would help somebody solve a problem." Mills said he retired when his company went insolvent in 2009, but he's kept his hand in IT.

"I have been involved with the GnuCOBOL (formerly OpenCOBOL) Project on SourceForge since October 2014, and decided to write a macro preprocessor that emulated the functionality available on the 3000," he said. "The preprocessor, CobolMac, is now in its 5th version (B.04) and has received good reviews by its users."

Others who contacted us said they haven't worked on the 3000 since the days that HP sold support for MPE/iX. "I have been a BizTalk developer full time since 2008," said Kent Wallace. "I needed to work, and this was the direction the world was going." The 3000-L still has more than 500 subscribers on its mailing list rolls, but much of the messaging comes from consultants and vendor experts, supplying answers to questions and tips. A total of 45 messages have passed through the list since the start of 2016. The IT pros like Wallace have taken the path to other platforms, first to HP-UX, then to Windows.

"I left my previous employer in Boise and I moved to a Microsoft shop, whose mainframe was HP-UX," Wallace said. "However, in 2015 we migrated off HP-UX and onto SQL Server on Microsoft Server 2008. We do health insurance and the purchased software, Trizzeto, was moved to MS SQL servers."

Another registered user keeps up with the community, but he can imagine a future where he'd be back on the MPE/iX front lines. "We're totally out of the 3000 business," said Ted Johnson of Wake Forest University, adding a sad-face emoji. "But I love seeing the 3000-L posts and keeping up. Who knows — maybe they'll get rid of me one of these days, and I'll end up back on a 3000."

The 3000-L, hosted at the University of Tennessee at Chattanooga where the late Jeff Kell launched it the early 1990s, holds more than two decades of traffic. 10 years ago the list was big enough to measure a signal-to noise-ratio, but by now it's almost entirely signal. When John Burke was a monthly columnist for the NewsWire who summarized its content in net.digest, he rounded up the following help in just one month's communications. For a 3000 owner managing a homesteading shop, the 3000-L's tips still carry some value.

Quick Cuts

• Do you want to know when a particular account or group was created? LISTACCT and LISTGROUP are no help. But “listfile /ACCOUNTNAME,3” for the account or “listfile /ACCOUNTNAME /GROUPNAME,3” for the group tell all. And then some.

• The number of sectors reported by the REPORT command for a group or groups is sometimes inaccurate, sometimes very inaccurate. Running the program FSCHECK.MPEXL.TELESUP and issuing the SYNCACCOUNTING command will fix this problem.

• In case you were wondering, despite many requests for the enhancement, TurboStore will NOT append store sets to tape. Well, it might if you use the proper incantations, but it is unsupported and highly dangerous because under certain circumstances you could overwrite a previous backup without knowing.

• Speaking of things you cannot do that you might like to do, the ALLOW command is not persistent across sign-ons unless you use the extremely dangerous “ALLOW @.@; commands” version. This is another example of an enhancement that has been requested for years, but now will never happen. Fortunately, there are a number of options, for sale and free (MPEX, CSL, etc.).

• CI integer variables are signed 32-bit entities. So be careful if you are doing some wild arithmetic in your CI scripts.

• Here is a little trick when using Apache’s indexing (for example to keep track of documentation) to index file displays. You can override the default ascending sort by name by appending “?N=D” to the url. Instructions on changing Apache’s default behavior are available on the Web.

• If you are trying to program VPlus applications and are interested in working examples programmed in your favorite language, look in the group HP32209.HPPL89 (which should be on every FOS tape). This group contains source code for the ENTRY program in a variety of languages including COBOL, Fortran, Basic and Pascal.

• To see the firmware (aka PDC) Revision of a system (CPU): Run cstm, and at the cstm > prompt, type ‘map’ and note the Dev Num of a CPU and then type ‘sel dev DEV_NUM’ (e.g., ‘sel dev 41’) and then type ‘info’ and then type ‘il’ and look at the output for the ‘PDC Firmware Revision’. Easy, huh? Thanks to Guy Paul of HP for this tip.

• SPFXFER will allow you to write to disk (undocumented “feature/bug”). But don’t do it, because SPFXFER cannot read the disk file it creates! Doing this could lead to a big oops.

• While it would certainly be a nice to have, MPE/iX CI scripts have no provision for inline comments. Sorry, don’t even bother trying.

Posted by Ron Seybold at 05:01 PM in Hidden Value, Homesteading, Migration, User Reports | Permalink | Comments (1)

February 19, 2016

How hot plug disks can replace DDS offsite

300 GB Ultra SCSII need to find an alternative to DLT and DDS tapes for offsite storage. Sure, there's DS2100 and Jamaica drives. But a few $35 300GB Ultra SCSI drives would hold a lot more data with less points of failure. I'll set up a BACKUP_VOLUME_SET and use the internal disks to do store-to-disk backups of the system. 

I've always used my A-Class and N-Class systems with fiber-attached disk. Are the internal disk drives hot-pluggable? 

Jim Hawkins, IO maven for HP 3000 systems at HP, replies with details.

There are multiple layers of changes for actual hot plugs or swaps to work.  

  • You need the disk HDD to handle this electrically.
  • You need HDD physical carrier and physical interface to comply.
  • You need the system physical interface and receptacle to comply. 
  • You need your Host System Bus Adapter (HBA) to electrically support this.
  • You need the OS to be aware enough of the HBA to not get flustered by absence of the device and deal with any notifications from the HBA of the activity.   

Given that the N-Class disk cage has a screw-based cover and the HDD carriers have no quick release levers (as compared with HASS/Jamaica or VA7400) I would state definitively that there is no hot-plug intention.  

At the same time, the SCSI bus is pretty low power and low voltage, so it would be generally not-too-unsafe to experiment. But you're also close to AC inputs and they are not low power.

Hawkins explored the not-too-unsafe scenario with theoretical possibilities.

Might you be able to pull/push a drive where you've closed the volume?  Likely it would work, but there may be all kinds of noise and stress on the SCSI bus which may not be well handled. However, I think each disk is on its own HBA channel which isn't shared with anything else, and so unlikely to abort someone else's IO.  

This takes us to the last issue: mechanical wear.  

These connectors were likely intended for more or less permanent mating of two components. Very likely they have a limited number of cycles that they are specified to hold-up. I've seen connectors that are specified for fewer than 25 cycles before you lose gold contact material.   This is okay for normal HDD where one might replace one or two per slot in a system lifetime, but not sufficient if you're doing nightly back-ups and swaps.  Connectors, where there is an expectation of a high number of pull/replace cycles, have special designs.  

Now a little good news here is that the N-Class was still pretty much old-school HP design, so likely they didn't pick up something cheap that saved them .2 cents per unit on gold plating. No idea though if the HDD connector is a 10-, 100-, 1000-cycle part. Your system, your risk. 

Consultant Mark Ranft points out that the HP design for 3000s seems to make the servers and components good candidates for exceptional mechanical wear tolerances.

It is especially helpful to understand the concept of mechanical wear on the connectors. HP always had excellent and innovative hardware engineering on their HP 3000 servers. Remember, you can drop them off a building and still self-test them

The Unix N-Class appears to allow hot-pluggable drives.   

The actual power supply and the fans are in the front of the N-Class.  The power receptacles in the back have internal cords that lead to the front.

Posted by Ron Seybold at 08:59 PM in Hidden Value, Homesteading | Permalink | Comments (0)

February 12, 2016

How to get specific about IP access for PCs

I want to give a 3000 a static IP, so I can permit a user to access the HP 3000 from that PC with that static IP. Is there a way to force a particular user ID to use a specific IP address?

Tracy Johnson replies:

A simple logon UDC should suffice:

IF HPREMIPADDR = "aaa.bbb.ccc.ddd" then
  ECHO Welcome.
  ECHO Evil message here.

Bob Schlosser adds:

You can set up a logon UDC that checks that the var HPLOCIPADDR is equal to the device (PC) that you want them to use. Something like this:

IF "!HPLOCIPADDR" <> "123.456.789.321"        change "123.456.789.321" to
your IP address

Using this, we verify that the user is on the correct (assigned) IP address, and log them off if not.

Chris Bartram, who's created e-mail solutions for the 3000 at 3K Associates, and hosted Web servers since early in the 1990s, adds:

The following is an excerpt from system UDCs I use on my HP 3000s that might give you some ideas.

The "VALIDATEIPADDR" call in this UDC calls another command file that actually does a validation of the logging-on user based on data in a control file to determine if he/she is allowed to log onto the system from the specific host/IP address they are coming from.

The variables the UDC sets will work whether the logging on user is coming in via Telnet or NSVT (or hardwired or via a modem).

The TELLOPs also leave a nice log on the system console (and log file) of the login, including where they came from, and what protocol was used to access the system.


setvar _network_node ''
if bound(hpstdin_network_node) then
  setvar _network_node '!hpstdin_network_node'

setvar _na ''
setvar _at 'HARDWIRED'
if bound(hpstdin_network_addr) then
  setvar _na '!hpstdin_network_addr'
elseif bound(hpremipaddr) then
  setvar _na '!hpremipaddr'

if bound(hplocport) then
  if !hplocport=23 then
    setvar _at 'TELNET'




setvar _node ups(ltrim(rtrim("!_network_node")))
setvar _addr ups(ltrim(rtrim("!_na")))
if '!_node'<>'' then
  tellop !_at, IP: "!_addr" Node: "!_node"
  tellop !_at, IP: "!_addr"

setjcw cierror=0
if !cierror<>0 then
  echo ************************************
  echo ************************************

Posted by Ron Seybold at 03:56 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 18, 2016

The GSP makes the A and N worthwhile

GSP boardIt's a powerful part of an HP 3000 that runs whenever the server is plugged in. The Guardian Service Processor (GSP) is the maintenance control console commanding the ultimate class of the server to reboot, do memory dumps and even fully power down the 3000. Consultant Craig Lalley of EchoTech has noted the GSP has one fewer feature than its Unix counterpart, though.

"On HP-UX it is possible to reset the GSP from the host OS," he said. "I have not found a way from MPE."

From time to time a reset may be required for diagnostics services on A-Class and N-Class servers. If your 3000 gets loving care from a consultant or service provider outside your computer room, you may need a paper clip to keep up service levels.

The GSP can also reveal the 3000's speedometer, as profiled near the bottom of a webpage from Allegro Consultants.

The gap between 3000 and HP's HP-UX Integrity GSPs is a common shortfall of HP designs. Even though the 3000's MPE/iX includes a Posix interface, HP didn't engineer enough Unix into the 3000 to enable some administration that HP-UX users enjoy. (That lack of Unix can sometimes be a good thing when a security breach opens up in the Unix world.)

But when a 3000 needs a GSP reset, pressing a recessed button on the 3000's back will do the trick if a telnet command doesn't work. You can telnet to the IP address of the GSP, log in and do the reset. But you can also get someone to press the physical reset button at the back of the machine. It's recessed into the cabinet so you may need a magic paper clip bent just so.

Lalley calls the GSP, which HP introduced with its final generation of 3000s, one of the most useful things in the A-Class and N-Class boxes.

The GSP is a small computer that is always powered on when the plug has power. With it, it is possible to telnet to and BE the console. While multiple admins can telnet in and watch, only one has the keyboard.

It is possible to reboot, memory dumps and even fully power down the HP 3000 from the GSP. Use the command PC OFF.

It is probably the best feature of the N-Class and A-class boxes. The problem is sometimes it needs to be reset, usually with a paper clip. Since the GSP is a different CPU, this reset can be done during business hours.


Posted by Ron Seybold at 10:43 AM in Hidden Value, Homesteading | Permalink | Comments (0)

January 13, 2016

Using Store-To-Disk for Backup Preservation

By Brian Edminster

Second of two parts

Yesterday I outlined some of the powers of the Posix program pax, as well as tar, to move MPE/iX backup files offsite. Here’s a warning. There are some file types that cannot be backed up by tar/pax while also storing their attributes:  ;CIR (circular) and ;MSG (message) files (and possibly others. I haven’t tested all possible file types yet.  Also, there is an issue with tar that is a fairly well known and has been discussed on the 3000 newsgroup. Occasionally it does not un-tar correctly.  It is unclear if and when this was fixed, but I’d love to hear from anybody that might be in the know, or which specific situations to avoid.

Regardless of these limitations, I’ve found a simple way around this. Use store-to-disk to make your backup, then tar to wrap it, so as to preserve the store-to-disk files’ characteristics, before shipping the files off-system. Later, when you retrieve your tar backups and un-tar them, you’ll get your original store-to-disk files back without having to specify the proper ‘;REC= , CODE= , and DISC=’ options on an FTP ‘GET’. I’ve been doing this for several months now on several systems, and I have not had any failures.

If you have a version of STORE that has compression, use it to reduce the size of backup.  If not, use the ‘z’ option in the tar/pax archive you create from your store-to-disk backup.  Do not use both.  They don’t play well together, and you may end up with a larger tar file.

But what about the tar archive size limit of 2GB?  There’s an easy way around this as well, as this limit is common on early Unix and Linux systems. Just pipe the output through ‘split’ to create chunks of whatever size you want. Below, there's simple examples for both directions.

Piping TarFigure 1, just below, is an example of the ‘cksum’ file produced.

Checksum 1 Grey

Below, Figure 2 is an example of a ‘cksum’ created of the files as they’re stored on the NAS. 

Checksum 2 GreyAs both the hashes and #bytes shown in each file are the same as on the MPE/iX server — we know the backups are transferred correctly.  The same technique can be used ‘in reverse’ to verify that when FTP’d back to the FTP server, they’re still intact.

When un-taring this backup, ‘cat’ the pieces together and pipe it through tar.   At least, that’s the way it’s supposed to work.  Yes, there is a known issue with the MPE/iX Posix shell’s built-in cat command. But I’ve so far been unable to successfully use the external cat command to successfully cat either.  Here’s how this should work for a 2-chunk tar backup:

sh>/bin/cat ./CS1STD1.ustar.aa ./CS1STD1.ustar.ab | tar -xfv - *

Unfortunately, for me at least, it always throws an error indicating bad format for the tar files.There is a work-around, however.  Note that while ‘cat’ing the tar ‘chunks’ didn’t work using the internal or external cat command, untar with multi-file option does work.  Even though it gives a minor error messages, files were returned to proper store-to-disk format, and the recovered store-to-disk backup is intact and has been used to recover the desired files. To do this, use tar like this: 

sh>tar -xfv ./CS1STD1.ustar.aa *  

Also note that when using tar in this way, it will ask for the name of the 2nd-nth component tar files, as it finishes reading each prior piece.  You must give the filename and press return to continue for each.  I believe that it should be possible to script this so that it’s fed the filenames, but I haven’t gotten around to doing that yet.  

Brian Edminster is president of Applied Technologies, a 3000 consultancy serving MPE/iX sites in contract and ongoing engagements.

Posted by Ron Seybold at 05:44 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 12, 2016

Backing Up Your 3000 Backup Files

By Brian Edminster
Applied Technologies 

Once store-to-disk backups on the 3000 are regularly being processed, it’s highly desirable to move them offsite — for the same reasons that it’s desirable to rotate tape media to offsite storage. You want to protect against site-wide catastrophic failures. It could be something as simple as fire, flood, or a disgruntled employee, or as unusual as earthquake or act of war.

Regardless of the most pressing reason, it really is important to keep at least some of your backups offsite, so as to facilitate rebuilding / recovering from scratch, either at your own facility, or at a backup/recovery site.

The problem comes in that the MPE/iX file system is far more structured than Unix, Windows, or any other non-MPE/iX file system-based storage mechanisms. While transferring a file off MPE/iX is easy via FTP, sftp/scp, or rsync, retrieving it is problematic, at least if you wish the retrieved files and the original store-to-disk files to be identical (i.e., with the same file characteristics: filecode, recsize, blockfactor, type, and so forth).

What would be optimal is automatic preservation of these attributes, so that a file could be moved to any offsite storage that could communicate with the MPE/iX system. Posix on MPE/iX comes to the rescue.

For FTP transfers between late-model MPE/iX systems this retrieval is automatic, because the FTP client and server recognize themselves as MPE/iX systems.  For retrieving files from other systems, HP has made that somewhat easier by making its FTP client able to specify ‘;REC= , CODE= , and ;DISC=’ on a ‘GET’:

Figure 1If you do not specify the ‘buildparms’ for a file being retrieved, it will default to the file-type implied by the FTP transfer mode: ASCII (the default), binary, or byte-stream (often called ‘tenex’ on Unix systems).  The respective defaults used are shown below:

Figure 2 GreyWhat follows is an example of automatic preservation of these attributes, so that a file could be moved to any offsite storage that could communicate with the MPE/iX system.  And this is yet again where Posix comes to the rescue, via the venerable ‘tar’ (Tape ARchiver), or ‘pax’ archiving utilities.

‘pax’ is a newer backup tool, designed to be able to read/write with tar format archives, newer ‘ustar’ format (that includes Extended Attributes of files). At the same time it has a more ‘normal/consistent’ command syntax (as Unix/Posix stuff goes, anyway), plus a number of other improvements. Think of it as tar’s younger (and supposedly more handsome) brother.

A little known feature of most ‘late-model’ tar and all pax commands is the ability for it to recognize and utilize Extended Attributes.  These will vary with the target implementation platform, but for the tar and pax commands included with releases after v5.5 of MPE/iX this capability is not only present — but contrary to the man command’s output and HP’s Posix Command Line manual, it’s the default! You use the -A switch to turn it off, returning tar to a bytestream-only tool.

While not externally documented, via a little experimentation I’ve determined that the following series of Extended Attributes value-pairs are in the MPE/iX Posix implementation of a tar or pax ‘file header’ for each non-Posix file archived:

MPE.RECORDSIZE= value in bytes
MPE.BLOCKFACTOR= integer value
MPE.RECORDFORMAT= integer value (0=unstructured?)
MPE.CCTL= integer value (0=nocctl)
MPE.ASCII= integer value (0=binary, 1=ascii)
MPE.FILECODE= integer value, absent for ‘0’
MPE.FILELIMIT= value in bytes
MPE.NUMEXTENTS= integer value, may be absent
MPE.NUMUSERLABELS= integer value (0=no user labels), and
MPE.USERLABELS=[binary content of user labels]

Brian Edminster is president of Applied Technologies, a 3000 consultancy serving MPE/iX sites in contract and ongoing engagements.

Posted by Ron Seybold at 02:38 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 05, 2016

Migrating 3000 Data from Spoolfiles to Excel

I need assistance with putting an output spool file from MPE/iX 7.5 into Excel or other readable format. The file is generated by Query, then processed by Editor, then sent to the printer. Instead of printing it, I want to put it into a readable format.

MigratingI do not have QEdit or any smart tools on MPE, so my approach thus far has been to move the file to a PC before doing anything.  However, that carries with it the initialization sequence for the printer to which the job is spooled. The job is set up to print on a PCL 5 laser, which means it has hundreds of lines of control before the data starts.

Tom Moore replies

I would put commas in between my columns (in the query, or using Editor). I FCOPY from the file to a new file with NOCCTL to get rid of carriage control byte. You could also remove the PCL 5 lines by subset in the FCOPY command. Depending on the data, I would use EDIT3000 to change all " ," to "," and all ", ","," to compress the file, removing the spaces before and after the commas inserted above, then save the file for download to the PC.

I would also consider using ODBC to directly extract from the IMAGE database, rather than Query and all the subsequent steps. The HP free ODBC driver would do the job very well.

Birket Foster of MB Foster notes

Not only did we make that free ODBCLink/SE as HP's lab resource from 1998 to 2006, but we have continued to develop the ability to work with data in all kinds of file formats. We do supply 32- and 64-bit versions for ODBC to the HP 3000.

UDALink-MPE was designed for the HP 3000. We provide data in several different formats including XLS for Excel, XML, CSV etc. We can have a discussion about what you are trying to do with data; perhaps UDACentral is the right product for your challenge and we can organize a demonstration for you.

Charles Finley adds

There seem to be at least three steps to what you are trying to do.

  • Remove the headers, footers and perhaps page numbers from the report.
  • Remove the ff or CNTL characters from the text file.
  • Import a space-delimited file to Excel.

There are any number of different scripting tools that can do this including various Unix tools. Here's a reference to an Excel solution that might get you started. In fact, if it were my problem to solve, I would likely do it all with Excel scripting.

John Hohn replies

  1. Output to a delimited file (tabs, pipes, etc).
  2. Download to your laptop or PC or wherever Excel is running
  3. Start macro recording in Excel
  4. Import/format the delimited file, save as .xls
  5. Turn recording off, save macro

Set the Excel file to auto-execute the macro every time the Excel file it's opened, i.e., re-input/format the delimited file. Then you can, for example, schedule delivery of a new version of this delimited file whenever you'd like, to your server. When people open it they would automatically get the formatted version of the new data.

Connie Sellitto of Hillary Software suggests

Hillary Software has a product, byREQUEST, which does just this.

It has the ability to suppress headings on pages after the first, and define the type of data in the columns (text, numeric, dates in various formats). It can remove blank pages and leading and trailing blank lines. It can even call an Excel macro to make the headings a different font, background color, etc — anything you'd want to do with a macro. In addition to Excel, byREQUEST can create a PDF file, Word, csv or Text.

Posted by Ron Seybold at 10:25 PM in Hidden Value, Homesteading, User Reports | Permalink | Comments (0)

December 22, 2015

Studying the Scripts for HP 3000s

A recent question on the 3000-L mailing list and newsgroup asked for help on scripting. The question aimed at the automation prospects through terminal emulation programs. Does Minisoft/92 script, a manager asked. Of course, and Tracy Johnson replied to give details as well as an example.

Minisoft scripts are plain text files with a file extension of .s92 and can be assigned to function keys f1 through f12. We use the keyboard mapping config menu to map it to Type "Script".  Once you choose "Script" a blank box appears below where you put the magic words DO SCRIPT followed by a path including the file name.

Joe FridayLast week, the use of scripts also surfaced while talking to Birket Foster about data migrations. A client of Fosters runs five scripts to clean up phone numbers during a transfer of data. Being able to reach for the scripts improved the quality of data, and the work was automatic. the power of scripting reminded me of a fine column written for us by Ken Robertson. Its subject was an introduction for Unix administrators to the use of shell scripts. But the writing was the kind of operational lore that can make a 3000 look more powerful to an admin new to the 3000.  Robertson wrote about it for the Newswire.

The marvels of scripting lie deep in the roots of MPE. When HP expanded the OS to MPE/XL in the 1990s, it added the Posix shell, which extended the 3000's scripting potential. The MPE/iX command interpreter has a generous command set, pushing the shell into the realm of a true programming tool. Its ability to evaluate expressions and to perform IO on files allows the end-user to perform simple data-processing functions. The Command Interpreter can be used to solve complex problems. Its code, however, is interpreted, which may cause a CI solution to execute too slowly for practical purposes.

For the average task, the MPE scripting language is easier to read and understand than most Unix scripts. For example, command line parameters in MPE have names, just like in regular programming languages. Of course, there are several script languages on Unix and only one on MPE. On Unix you can write shell scripts for any of the many shells provided (C shell, Bourne shell, ksh, bash, etc). Although there is also a Posix shell on MPE, most scripts are written for the CI. 

A command file can be as simple as a single command, such as a Showjob command with the option to only show interactive sessions (and ignore batch jobs):

1      showjob job=@s
2      //
/keep ss

You have created a command file called SS — when you type SS you will execute showjob job=@s

On MPE, the user needs read (r) or execute access (x) to SS. On Unix you normally must have x access, not just r access, so you do a chmod +x on the script. This is not necessary in MPE, although, if don’t want users to be see the script, you may remove read access and enable execute access.

Structure of a Command File (aka CI script)

A script is an ASCII file with maximum 511 byte records. Unlike Unix, the records may contain an ASCII sequence number in the last 8 columns of each line. The command file consists of 3 optional parts:

1. Parameter line with a maximum of 255 arguments:
parm sessionnumber
parm filename, length=”80”

2. Option lines:
option nohelp,nobreak
option list

3. The body (i.e., the actual commands)”
showjob job=!sessionnumber
build !filename;rec=-!length,,ascii
In MPE scripts, there is no inline data, unlike Unix ‘hereis’ files.


Notice in the example above that parameters are used with an exclamation (!), as opposed to the $ in Unix. The same is true for variables. Parameters are separated by a space, comma or semicolon. All parameter values are un-typed, regardless of quoting.

In a typical Unix script, the parameters are referenced by position only ($1, $2, $3, …). In an MPE script, the parameters have names, as in the function of a regular programming language, and can also have default values. In Unix you use $@ for all of the parameters as a single string; in MPE you use an ANYPARM parameter to reference the remainder of the command line (it must be the last parameter).

Here is a script to translate “subsys” and “err” numbers from MPE intrinsics into error messages. The subsys and error numbers are passed in as parameters:

parm p_subsys=108,p_error=63
setvar subsys hex(!p_subsys)
setvar error hex(!p_error)
comment the hex conversion allows for negative numbers
comment the #32765 is magic according to Stan!
setvar cmd “wl errmsg(#32765,!subsys);wl errmsg(!error,!subsys);exit”
debug !cmd

As you can see above, the Setvar command assigns a value to parameter or to a new variable. But there are also system pre-defined variables. To see them all do Showvar @;hp. To get information on variables, do help variable and to get help on a specific variable, say hpcmdtrace, do help hpcmdtrace (set TRUE for some debugging help).
In most MPE commands, you must use an explicit exclam ! to identify a variable: build !filename

However, some MPE commands expect variables, and thus do not require the explicit !. For example, Setvar, If, ElseIf, Calc, While, and for all function arguments, and inside ![expressions].

Warning: variables are “session global” in MPE. This means that if a child process, or scripts, changes a variable, it remains changed when that child process terminates. In Unix you are used to the idea that the child can do whatever it likes with its copy of the variables and not worry about any external consequences.

Of course having global variables also means that it is much easier to pass back results from a script! And this is quite common in MPE scripts.


Options allow you to list the commands as they are execute (option list), disable the Break key (option nobreak), enable recursion (option recursion), and disable help about the script (option nohelp).

The script body below shows active process information. This example shows many of the commands commonly used in scripts: If, While, Pause, Setvar, Input and Run. Other commands you will see are Echo, Deletevar, Showvar, Errclear.

    WHILE FINFO("SQMSG",19) <> 0
         IF POS("#",SQLINE) <> 0 THEN
           SETVAR PINLIST "!PINLIST" + "," + "!PIN"
    WHILE FINFO("SPMSG",19) <> 0
         IF POS(":",SPLINE) <> 0 THEN
           ECHO !SPLINE
    PAUSE 30

Handling Errors

In most Unix scripts, if a step fails, you check for an error with an If-conditional and then take some action, one of which is ending the script. Without an If, the script continues on, ignoring the error.

In MPE, the default action when a step fails is to abort the script and pass back an error. To override this default, you insert a Continue command before the step that may fail. You then add If logic after the step to print an error message and perhaps Return (back 1 level) or Escape (all the way back to the CI).

      build newdata
      if cierror<>100 then
         print "unable to build newdata file"
         print !hpcierrmsg
         comment - duplicate file, okay

You can set HPAUTOCONT to TRUE to continue automatically in case of errors, but this can be dangerous. The default behavior at least lets you know if an unexpected problem occurs.

Posted by Ron Seybold at 07:42 PM in Hidden Value, Homesteading | Permalink | Comments (0)

December 07, 2015

On MPE Chatting, B-Tree Plants and More

How we can chat on the HP 3000 system with the other users who have logged on?

Lars Appel replies:

You can try the TELL and TELLOP commands. For more information see :HELP TELL or :HELP TELLOP.

When I run dbutil.pub.sys, and type the command set MYDB btreemode1=on I get the message “Database root file must be at least “C”4 for SET <db> BTREEMODE1=O." Why can’t I set my btree mode?

Rene Woc replies:

Before you can “set btreemode1,” the database has to have Btrees. You add Btrees with “addindex MYDB for all (or specific master datasets)”. This command will also set root level to C4. To use “addindex” your system needs to be at least on TurboIMAGE version C.07.xx. So how do you find out what version of IMAGE you have? Use the version command in QUERY.

I need to take some groups off of the mirrored drives, and add (move) other groups onto the mirrored drives. Is it as simple to use the altgroup command and specify the volume set?

[Editor’s note: “mirrored drives” is a straw man that has nothing to do with the problem or answer.]

Craig Lalley and John Clogg reply:

It is simple but not that simple. What you need to do is create a temporary group on the target volume set. Copy files in the group you want to move to the temporary group. Delete the source group. Create the new group using

NEWGROUP xxx;homevs=volume_set
NEWGROUP xxx;onvs=volume_set

Note that it is a two-step command. Then rename the files from the temp group to the newgroup. John Clogg also noted that another approach would be to STORE the files, and restore them once the group was relocated. That way you could preserve creation and modification dates, and creator ID.

I have inherited a HP 3000, and I was trying to clean off an old TurboIMAGE database. I did a purge @.data and it purged about 100-plus files. However, it did not purge everything. They can’t be purged, even when logged in as manager.sys. The error that I get is CIERR 45, Privileged File access. I have tried to release it with no luck either. Any suggestions?

Steve Macsisak and Denys Beauchemin reply:

You can purge TurboIMAGE databases using the utility DBUTIL.PUB.SYS and the command PURGE (database name). The database name is the name of the root file, the one without numbers at the end. The command will purge all the datasets of the database. There are a few other ways to do this, but this is the most obvious.

[Editor's Note: Vladimir Volokh reports that if you have several databases, you must run this several times. Using MPEX, you can purge everything, so long as you are a manager of the account. All databases, all non-databases, everything will be purged with purge @.groupname. purge@.data (ispriv) will purge only databases in the group.]

I have two device numbers I don’t want to be in use and assigned to users on the network. LDEVs 5 and 6 are available for use and we don’t want them to be. Is there some place to configure these out of use?

Gilles Schipper replies:

Simply add these devices as network printer devices (id=hptcpjd path=none) in sysgen. This will prevent them from being used by any logon session.

Posted by Ron Seybold at 08:33 PM in Hidden Value, Homesteading | Permalink | Comments (0)

November 24, 2015

The Wide World of Connecting Storage

IO used to be more complex for IT. Sure, the array of choices for disk is vast today. But in the era when 3000s used to think they were lucky to get SCSI plugged into them, configuring disk connections was not simple. HP-IB protocol, built to link HP's instruments, was simple, used for all HP devices, and slow. But it was integrated and seamless compared to the SCSI of single-ended, fast/wide, and Ultra Fast.

Such was the case for one 3000 manager seeking advice from his colleagues. You never think about these things on a 3000 until the hardware breaks. Or backups fail. Or storage media gets rare. Aging hardware is one of several issues that require expertise, even if a 3000 runs the ultimate 7.5 version of MPE/iX. Our manager hunted for his help on the longest-running 3000 classroom in the world, the HP3000-L mailing list.

A single-CPU A-Class was moving away from DDS technology, the DDS-3 that was first launched in the '90s. There are other options for 3000 tape backup. But these options include single-ended, fast/wide, and other cable and termination combinations. DLT technology, introduced more recently but still a 1990s choice, runs with HP 3000s. It helps to get the ends right, though, if DLT is to have a new beginning on an old-school 3000.

"Until now they have done their backup on DDS," a manager talking to the 3000 newsgroup explained. "Lately they had a failure on the DDS drive, and have realized that it is getting difficult to get new tapes. They have decided to move to DLT8000, model C6378A, and have bought two of them. One is supposed to go live on the 3000, and the other to be stored as a spare device."

The DLT is hooked to the Ultra Wide SCSI interface on the A-Class. But ODE/Mapper doesn't recognize the device."

There was an error, and no DLT joy. Soon enough, one veteran consultant said, "You will have trouble connecting a fast wide SCSI device to an ultra-wide SCSI controller." It wasn't a rookie mistake, but the veterans who still prowl 3000-L had a solution and even a link to an inexpensive fix. So it goes, here in the fifth decade of HP 3000 mission-critical service. Answers are everywhere.

This wasn't an inexperienced 3000 pro, it seemed, when reading that he tried to "add the device in IOCONFIG by adding first the path 0/0/1/0.2, and then the device with the command: ad 8;path=0/0/1/0.2.0;ID=dlt;mode=autoreply."

SCSI on the 3000 sure isn't the world of USB, where just 2.0 and 3.0 cover the scope of IO choices. A $59 adapter card connected that DLT to the 3000. The IO challenge also prompted advice even a pro might not know — making a case for having fresher hardware than HP's to run MPE.

There was advice about using Mapper on the 3000 to troubleshoot an IO device from Michalis Melis.

Normally the path and the device should be recognized by running ODE/ Mapper without even loading the operating system. You do not have to go to SYSGEN. If Mapper fails you have a problem before the OS loads.

Craig Lalley made the link between two incompatible kinds of SCSI interfaces.

You are trying to hook up a Fast/Wide SCSI device to an Ultrawide interface. The C6378A can only connect to a HVD Fast Wide SCSI interface (A4800A SCSI card comes to mind). Remember, the A-class does not support Dual-Head cards, so your only option is A4800A. You need either a DLT8000 with a Fast-Wide interface, or you need a cheap A4800A HVD (High Voltage Differential) SCSI card. You can daisy chain devices to the card, but I would only use one tape at a time.

Lalley also tipped his hat to Keven Miller, who supplied the link to that $59 adapter card.

Then Denys Beauchemin, who has been among one of the more prolific contributors to the 3000-L, delivered detailed advice about connecting backup devices. His background reaches back to the first decade of 3000 use, including years spent with Hi-Comp on backup software development.

Fast/Wide SCSI (FWSCSI) is essentially HVD SCSI on SCSI-2 standard. This means that the signal is a differential in the voltage between various wires (HVD is High Voltage Differential) and Ultrawide SCSI is SE (Single Ended) SCSI, on the SCSI-2 standard which makes is wide (16 bits), like the FWSCSI.

So what is needed is a converter to power the signal from the Ultra Wide SCSI interface on your server to the FWSCSI interface on the DLT device. I have a number of those somewhere here, but they were for SE SCSI, not UltraSCSI.  They might work for that, since all they did was provide the powered signal and the cable is the one that converted from wide to narrow.

Another thing to consider is that since HP nicely crippled the A-class, that 3000 system would not be able to keep the DLT8000 streaming. And that device hates not streaming, so much so that it will enter shoeshine mode and perform abysmally. Just a parting gift from HP to the MPE community. You should hear what they're doing to the VMS crowd.

That last comment comes from Beauchemin's current duties as migration manager for the OpenVMS users who are leaving that platform. VMS had a steady Internet community to help Digital users, just as the 3000 has 3000-L. People like Beauchemin, largely working outside the 3000 world, are still providing advice for homesteaders -- even while assisting in migrations. After migration there is much to manage, but simply migrating off Hewlett-Packard's 3000 hardware makes using MPE/iX less complex.

Posted by Ron Seybold at 08:07 PM in Hidden Value, Homesteading, User Reports | Permalink | Comments (0)

November 17, 2015

Putting PDF Into a 3000's Data Flow

PDF logoHP 3000 experts know of a wide array of techniques to create PDF files from the server's data, then move them via FTP to a Windows server. While the simplest answer for getting reports into PDF format and then out via Windows is probably Hillary Software's byRequest, there are other commercial solutions. There's also several bolt-together techniques if you've got very limited budget to homestead.

Bob McGregor reports:

We use txt2pdfPRO by Sanface. We had a job that would run and check a pseudo device for spoolfile output, and if the pri > 0, would run the sf2html process, convert to PDF and then FTP to a Windows server. The process would then delete spoolfiles=0 on the pseudo device the next day. Setup took a bit... but once done, worked well.

Lars Appel, author of the Samba/iX file sharing tool, adds:

I wonder if it might make sense to configure a "dummy" network printer on MPE/iX and have it send spooler output to a little socket listener on the Windows system (similar to the FakeLP example from the 3000-L archive) and then invoke GhostPCL on the Windows side for generating the PDF output.

The "dummy" network printer would let the MPE spooler take care of the PCL conversion and also perform the "file transfer" automagically. The GhostPCL software is probably easier to get (or build / update) on Windows than on MPE (okay, I admit that it did also build on MPE long ago.)

John Pitman employed that concept of Appel's in a combination of an off-the-shelf FTP solution, a freeware PDF converter, along with a good deal of innovative 3000 integration.

Nominate a spooled ldev as always suspended (74 in our case, arbitrary). Users can choose this device as their printer in their menu, and all subsequent reports (until changed to another real printer ldev) will go to this device — and therefore NOT physically print. Some reports that are commonly used to import to Excel have been modified to make headings tightly lined up with the data columns, and only print one page heading, to ease the import process.

Run a job on 3000 that every few minutes scans for spoolfiles for this ldev, then copies them to Posix space specific to 74 (for generality), with the creating user and account in the file name (e.g. mgr_stock_O12345.txt), then delete the original spoolfile.

A scheduled program on the Windows box (every minute) connects via FTP to the 3000. When it finds a spool files as above example, it checks for a Windows destination dir of MGR_STOCK, and copies the file to it as O12345.txt, and deletes the 3000 copy of the file. The account name enables segregation of reports for different applications in our case. If the file is > 1MB (an arbitrary size of your choice), it's zipped. It could as easily be converted to any desired form — for example to PDF via the shareware cutepdf.

It could also readily email the output to a user, given access to a mail server, and a way to develop the email address. Users have a client to access the FTP server and obtain their .txt or .zip files

This has been running for more than 10 years with almost no issues. Occasionally a large file might hang FTP, but canceling and restarting the copy usually fixed it. I have seen report selection errors produce 500 MB TXT files.

You might use several suspended ldevs for different types or groups  of users. We ran this on four 3000s in different locations, each with their own separate windows boxes using BP-FTP server. This means that users in Australia can run a report on the Houston or China system to the local printer 74, pause, connect their FTP client to the relevant FTP server, and download the report without having to print it.

The process also enables soft storage of month end reports, which can be very useful for comparative purposes, auditing, and general historical reference. We now have about eight years of this information stored, with backups and CD copies. Much more compact than paper, and cheaper.

Michael Anderson of consulting firm J3K Solutions added that there is also a open source PDF tool, pdfcreator, a manager can use to set up a network PDF printer. "Some assembly required, and batteries not included," he noted.

Another vote came in for the Advant/X software from Tracy Johnson, the OpenMPE volunteer who manages the Invent3K shared open source server. Johnson notes that this STR Software product "while intended to convert spool files and then e-mail or fax them, can be used short of the transmission process."

Posted by Ron Seybold at 05:44 PM in Hidden Value, Homesteading | Permalink | Comments (1)

November 11, 2015

Protecting a 3000 by Eliminating Its Services

Ron as PrivateHere on this day when we celebrate people who have served in the armed forces, a question emerged about enabling HP 3000 JINETD services. Or disabling them, to make a 3000 more powerful and secure. (Yes, it seems to defy the logic about more services being better, one that we can hear in national defense debates. We didn't have such debates at Signal Corps training for the Second Battalion.) The solution to the 3000 service problem included advice on how to trim back risk as well as performance drains on a 3000.

Grigor Terterian said he was having a Series 979 freeze up, because JINETD was receiving a call "for echo udb." Mark Ranft and Denys Beauchemin said the fastest repair would be to comment out echo in the inetdcnf file. Ranft got specific with an example.

:print inetdcnf.net
# Internet server configuration database
#echo  stream tcp nowait MANAGER.SYS internal
#echo  dgram  udp nowait MANAGER.SYS internal
#daytime  stream tcp nowait MANAGER.SYS internal
#daytime  dgram  udp nowait MANAGER.SYS internal
#time  stream tcp nowait MANAGER.SYS internal
#time  dgram  udp nowait MANAGER.SYS internal
#discard  stream tcp nowait MANAGER.SYS internal
#discard  dgram  udp nowait MANAGER.SYS internal
#chargen stream tcp nowait MANAGER.SYS internal
#chargen dgram  udp nowait MANAGER.SYS internal
telnet  stream tcp nowait MANAGER.SYS internal
#bootps  dgram  udp wait   MANAGER.SYS /SYS/NET/BOOTPD bootpd
#tftp  dgram  udp wait   NET.SYS   /SYS/NET/TFTPD tftpd
ftp  stream tcp nowait MANAGER.SYS /SYS/ARPA/FTPSRVR ftpsrvr

In the example above, only telnet and ftp services are enabled, Ranft said. This led Art Bahrs, a Certified Security Professional, to add that the services you leave on are the ones that can cause trouble, if you don't need them enabled.

Bahrs, who's also a retired Marine, celebrated his Veterans Day with this advice.

You should never, ever, no time, (did I mention 'never'?) run services you don't use or have a business or production need for.

Two reasons: First is security minded. If you have a service active, it is just another way to be hacked. Second is that an active, running service uses machine power, which is wasteful of electrons if there’s no need for it.

Ranft added his experience with inetd on MPE/iX:

Note that the command:

inetd.net.sys - c

will have inetd re-read the configuration.

Your success with this may vary. I've had lots of trouble with inetd in the far past. They got a lot better with the latest (final, for MPE/iX) set of patches. But on occasion, a scheduled restart (inetd.net.sys -k) and re-stream will probably help prevent issues.

I run my inetd with the logging feature.

!job jinetd, manager.sys
!run inetd.net.sys;pri=cs;info="-l"

This allows one to see the offending IP address in the $STDLIST.

Received call for: telnet tcp
telnet/tcp: Connection from unknown ( at Fri Nov 6 19:56:28 2015
Received call for: echo tcp
echo/tcp: Connection from unknown ( at Wed Nov 11 12:56:45 2015
Received call for: echo udp
echo/udp: Connection from unknown ( at Wed Nov 11 12:57:07 2015
Received call for: echo udp
echo/udp: Connection from unknown ( at Wed Nov 11 12:57:25 2015

As one veteran to others, I honor the services of all on this day, and thank you for your efforts toward our security. Long may it wave.

Posted by Ron Seybold at 09:59 PM in Hidden Value, Homesteading | Permalink | Comments (0)

October 20, 2015

Patching an HP 3000 with Patch/iX, Stage/iX

Editor's Note: Today a system manager asked for directions on using AUTOPAT on an HP 3000. That's very old school tech even for a classic system like the 3000 (TOH to Alan Yeo for digging out that link). More than 10 years ago, our contributor John Burke detailed the use of the more-modern MPE/iX patch tools.

By John Burke

Patch bicycleEven after HP introduced Patch/iX and Stage/iX to MPE/iX, these HP tools were poorly understood and generally under-used. Both are tremendous productivity tools when compared with previous techniques for applying patches to MPE/iX. There's a good reason for using the newer tools. Prior to the introduction of Patch/iX and Stage/iX, system managers did their patching with AUTOPAT, and you had to allow for at least a half-day of downtime. Plus, in relying on tape, you were relying on a notoriously flaky medium where all sorts of things could go wrong and create “the weekend in Hell.”

Patch/iX moves prep time out of downtime, cutting downtime in half because you create the CSLT (or staging area) during production time. Stage/iX reduces downtime to as little as 15-20 minutes by eliminating tape altogether and, furthermore, makes recovery from a bad patches as simple as a reboot.

This article is based upon the Patch Management sessions I presented at three Interex Solutions Symposiums. The complete set of 142 slides (over 100 screen shots) and 20 pages of handouts are downloadable from my website. The complete presentation takes you step-by-step through the application of a PowerPatch using Patch/iX with a CSLT and the application of a downloaded reactive patch using Patch/iX and Stage/iX. Included is an example of using Stage/iX to recover from a bad patch.

Why should you care about Patch Management? Keeping a system running smoothly includes knowing how to efficiently and successfully apply patches to the system. HP supplyed bug fix patches to MPE/iX and its subsystems through 2008, including two PowerPatches a year through 2006. Some patches add functionality to older 3000s. Patch/iX is a tool for managing these patches.

Patch/iX can be used to apply reactive patches, a PowerPatch, or an add-on SUBSYS tape with a PowerPatch. Patch/iX is actually a bundle including the PATCHIX program and a number of auxiliary files that are OS release dependent.

Patch/iX allows you to:

• Qualify all patches in a set of patches;

• Install reactive patches at the same time as a PowerPatch;

• Selectively apply patches from a PowerPatch tape; and,

• Create the CSLT (or staging area for Stage/iX) while users are still on the system; i.e., when it is convenient for you without incurring downtime.

Stage/iX is an OS facility for applying and managing the application of patches. Stage/iX reduces system downtime for applying patches to the length of time required for a reboot and provides an easy and reliable method for backing out patches. Stage/iX includes an interface to Patch/iX that creates the “staging area” and two utilities:

• STAGEMAN, which allows you to manage all aspects of patch staging, including which version of the OS will be used for the next update; and,

• STAGEISL, an ISL utility available from the ISL prompt whenever the system is down. It contains a subset of STAGEMAN functionality that allows you to recover from most problems.

Steps in staging

The set of all operating system files, for example NL.PUB.SYS, etc., are considered the current Base OS. Stage/iX creates and manages staging areas, which are HFS directories that hold versions of files that are different from the Base. More than one staging area can exist at a time. Each staging area contains the difference, or delta, between the Base OS and a patched version of the OS.

When a staging area is activated on the next boot, the files in the staging area are moved into their natural locations while the Base versions of the files are saved in a Stage/iX archive HFS directory. To back out a patch, the reverse takes place and the system is restored to its original state.

Once you are satisfied with the new and patched OS, you can COMMIT the staging area to the Base, deleting the staging area directory and all archived Base files. Note that Stage/iX and Patch/iX allow new patches to be staged and applied in a cumulative fashion. In other words, if you create a new staging area while another staging area is active, the new staging area will contain all the changes between the Base and the active staging area plus all the new changes.

Whether or not you use Patch/iX and Stage/iX, the key to successful OS patching is preparation. Information is the key to preparation. The System Software Maintenance Manual (S2M2) for your particular release of MPE/iX is the bible for all patch management activities. It contains a checklist for each possible update and patch activity along with detailed sections corresponding to checklist items. A hardcopy version and a PDF version on CD usually ship with each major OS release.

The S2M2 for each OS release is also available and downloadable from docs.hp.com. Information about specific patches is available at the IT Resource Center (itrc.hp.com). A PowerPatch usually comes with some patch specific documentation – make sure you have it available before you start.

Finally, before you ever sit down at the keyboard, create a Patch Book for the specific patch activity you will be attempting. You can do it with the hardcopy manual and a copy machine, but I prefer to use the PDF version, printing out the two-page checklist and each section that makes up the checklist to create my Patch Book.

How to get, apply patches

Suppose you just got the latest Patch Digest and there is a patch you need to install or you’ve read a thread on the 3000-L mailing list that references a patch you think you should install. Let’s look at how you can get and apply the patch using the ITRC, Patch/iX and Stage/iX. Note that the complete reference manual for both Patch/iX and Stage/iX can be found in the appendices of every S2M2 since, and including, the one for MPE/iX 5.5.

Before proceeding too far, check HPSWINFO.PUB.SYS to ensure the patch has not already been applied. [Note that Patch/iX will tell you if a patch, or even a superceding patch, has already been applied, but it only takes a moment to check HPSWINFO and it could save you some time.] Each patch has an eight character ID. For example, consider TIXMXC3B. The first three characters indicate the subsystem; in this case TIX stands for TurboIMAGE. The next four characters are internal to HP’s patch management mechanism. The final character is the version identifier; in this case “B” indicates the second version of this patch.

First off, identify the proper checklist, in this case Checklist B, and create your Patch Book. Next, review the information about the patches at the ITRC; in particular, look for any patch dependencies.

You need to make sure you have the latest version of Patch/iX installed on your target system. This is critical to your success. All sorts of bad things can happen if you use an old or incomplete version of the Patch/iX bundle. To check the version of Patch/iX, sign on as “MANAGER.SYS,INSTALL” and type in PATCHIX VERSION, The program will respond with something like: Patch/iX Version B.01.09

[Ed. note: getting patches from HP is a free process, but you may have some work in front of you to find a place inside HP support that recognizes the 3000 as a server, and knows how to deliver its patches. If you have a maintenance contract with a good third party — Pivital Solutions, for example — they can help lead you to the right HP cubbyhole.]

Once you have the current Patch/iX and your patches, you are ready to run Patch/iX and create your staging area. There are four steps in any run of Patch/iX:

• “Select Activities,” where you define what type of patching activity you want to perform. You have the choice of adding a PowerPatch, adding reactive patches from tape, adding reactive patches from download or adding SUBSYS products.

• “View Patches” (optional): You can actually view information about all the patches that have been applied previously to your system. Note that this can easily number in the hundreds for a system that is kept reasonably up to date.

• “Qualify Patches,” where Patch/iX does a lot of work to determine which patches of the set you supply it with can and/or should be applied.

• Create the stage, the tape, or both that will be used to actually change the OS.

This is all done while normal production continues and places a minimal load on your system. Once you have created your stage with Patch/iX, you run STAGEMAN to activate your staging area with the SET command. The next time you boot your system (and this can be done remotely from your home at 3 AM Sunday morning if you like), your changes will take effect. Total downtime is the time it takes to do a SHUTDOWN followed by a START NORECOVERY.

What if something goes wrong? If you have problems after successfully rebooting your system and you want to back out your patches, simply run STAGEMAN and use the SET command to make the Base the active stage, reboot your system and you are right back where you started. Suppose you cannot even boot the system successfully after setting the stage? Simply boot to the ISL prompt and use STAGEISL to set the active stage to BASE, reboot and, again, you are back to where you started.

Once you are satisfied with your changes after reasonable testing you can again run STAGEMAN and this time use the COMMIT command to make the active stage the Base and free up the disk space occupied by the old Base.

Of course I have only touched on the key points in the Patch/iX and Stage/iX process. There are a number of details I have left out that are covered in your Patch Book. My goal has been to demonstrate the ease with which you can patch an HP 3000 OS using Patch/iX and Stage/iX. Try it once and you will never go back to applying patches via AUTOPAT — nor will you ever again use a CSLT tape to apply patches if you can possibly help it.

Posted by Ron Seybold at 05:54 PM in Hidden Value, Homesteading | Permalink | Comments (0)

October 07, 2015

Reloading, Redux: How To

Loading cementIt used to be the worst thing that could happen to an HP 3000 was a reload of its data. Adager gained its central place in the 3000 manager's toolbox because it would prevent the need for a reload after database restructuring. Although the worst thing to happen to today's 3000 is the loss of an expert to keep it healthy and sustained, reloads are still a significant event.

Not very long ago, a 3000 manager was looking for a refresher on how to do a system reload. Ernie Newton explained why he needed one and shared what he knew. Advice about adding the DIRECTORY parameter, and using BULDJOB1 to set up the accounting structure, followed when Ernie said

We suffered a double drive failure on our Raid 5 disk array yesterday and I'm thinking that I may have to do a reload. It's been almost 15 years since I've done one. If I recall, I do a load when bringing the system up, then do a restore @.@.@.

Our resident management expert Gilles Schipper provided detailed instruction on doing a reload. We hope it's another 15 years for Ernie until he's got to do this again.

Here are the instructions, assuming your backup includes the ;directory option, as well as the SLT:

1. Boot from alternate path and choose INSTALL (assuming alternate path is your tape drive) 
2. After INSTALL completes, boot from primary path and perform START NORECOVERY. 
3. Use VOLUTIL to add ldev 2 to MPEXL_SYSTEM_VOLUME_SET. 
4. Restore directory from backup (:restore *t;;directory) 
5. openq lp
6. Perform a full restore with the following commands
:file t;dev=7(?)
:restore *t;/;keep;show=offline;olddate;create;partdb;progress=5

Gilles adds, "I would suggest setting permanent and transient space each equal to 100 percent on LDEV 2. The 75 percent default on LDEV 1 is fine as long as you don’t need the space. And if you did, your solution shouldn’t really be trying to squeeze the little extra you’d get by increasing the default maximum limits."

The reason for limiting LDEV to 75 percent is to minimize the otherwise already heavy traffic on LDEV 1, since the system directory must reside there, as well as many other high traffic “system” files.

You won't want to omit the ;CREATE and ;PARTDB options from the restore command. Doing so will certainly get the job done -- but perhaps not to your satisfaction. If any file that exists on your backup was created by a user that no longer exists, that file (or files) will NOT be restored.

Similarly, if you omit the ;PARTDB option, any file that comprises a TurboIMAGE database whose corresponding root file does not exist, will also not be restored.

I suppose it may be a matter of personal preference, but I would rather have all files that existed on my disks prior to disk crash also exist after the post disk-crash RELOAD. I could then easily choose to re-delete the users that created those files -- as well as the files themselves.

Another reason why the ;SHOW=OFFLINE option is used is so that one can quickly see the users that were re-created as the result of the ;CREATE option. Purging the "orphan" datasets would be slightly more difficult, since they don’t so easily stand out on the stdlist.

Finally, it’s critical that a second START NORECOVERY be performed. Otherwise, you cannot successfully start up your network.

Posted by Ron Seybold at 04:10 PM in Hidden Value, Homesteading | Permalink | Comments (0)

September 23, 2015

Where Do Those DBEs Go In MPE?

Where can I get help with storing and restoring an Allbase DBE?

Gilles Schipper replies

SQLUTIL.PUB.SYS should let you access the DBE. IMAGESQL is to a DBE what QUERY/DBUTIL is to TurboIMAGE.

Denys Beauchemin adds

ISQL.PUB.SYS is used to access the data in a DBE. If a TurboIMAGE database is attached to a DBE, there is a DBTC file for each Turbo database and an ATCINFO file with the DBE.

How can I find all the IMAGE databases on a system?

Michael Anderson replies:

IMAGE database root files all have a unique filecode value, -400. IMAGE datasets all have a value of -401. So if you want to find all IMAGE databases on a MPE system I would use the following command:


This will give you all the IMAGE root files and only the IMAGE root files. If you use something like “listf @01.@.@” you get the first dataset in the database, but also any file where the filename ends with “01”, and that may or may not be a database file. Also, the MPE file system allows filecode representations to be alpha (“PRIV”) and Numeric. When you see “PRIV” as a filecode it simply means that the numeric value of the filecode is negative, and again this can include non-database file types.

What is the command to kill a session that refuses to die?

Ernie Newton and Sam Knight reply:

If it is a VT or Telnet session, try NSCONTROL KILLSESS=#Snnnn

What is the maximum sized LUN that MPE 7.5 will recognize?

Guy Paul replies:

You should be able to have up to a 146 GB LDEV. Trial and error over the years has taught me to put no more than 70Gb/FWSCSI on a busy system. If you have Hyper-Volume extension, I would suggest slicing the mechs up a little smaller.

I have a couple of configuration questions about an N-4000. Does it make sense that an A6795A (PCI 4x 2Gb/s Single-port Fibre Channel Adapter) would be installed in PCI slots 1 or 2? The system's docs indicate that PCI slots 1 and 2 are 2x PCI slots; so the fibre channel would be limited to 1Gb/s. Does it make sense to have two Fiber Channel cards for a single VA7110?

Craig Lalley and Guy Paul reply:

The VA7110 does have two fiber connections, and both CAN be used. However, in order to be consistent with HP’s maintenance criteria, you should have one connected to a Command View workstation. Everything in Command View can be done through the serial port on the VA7100, except for logging. It is also a lot easier to define LUN’s graphically (Command View) as opposed to the command line (cryptic) language on the serial port.

Also on the VA7110, one of the fiber channels is “passive” and is for the command view workstation. I would not install a A6795A HBA into slots 1 or 2. Slots 3-12 are the twin-turbo (4x) slots you should use.

Posted by Ron Seybold at 09:02 PM in Hidden Value | Permalink | Comments (0)

September 18, 2015

Passing audits: MPE privileges can be keys

Migrated HP 3000 data can become forgotten while making provisions for an audit. Since some HP 3000s work as mission-critical servers, these active, homesteading systems must weather IT and regulatory audits. The 3000 is capable of passing these audits, even in our era of PCI, HIPAA and Sarbanes-Oxley challenges — all more strenuous than audits of the past.

However, establishing and enforcing a database update procedure is a step onto filling the gap in the security of an MPE/iX system. HP 3000 managers should take a hard look at how their users employ System Manager (SM) privileges. (Privileged Mode, PM, and System Supervisor OP should also be watched. Overall, there can be 21 capabilities to each user.) In their most strict definition, those privileges can expose a database. Hundreds of users can be created at Ecometry sites; even seasonal help gets SM users, according to one consultant's report, users which are seldom deleted after the holiday has passed. One site had a script to create new users, and each had PM capability, automatically.

Privileges are often a neglected aspect of 3000 operations, especially when the system's admin experts have moved on to non-3000 duties, or even to other companies. (Then there's the prospect that nobody knew how to use privileges in the first place.) Some SM users have disturbed the integrity of 3000 databases. It's easy to do accidentally. A creator of a database can also update a 3000 database — a capability that can foul up a manager's ability to pass some audits.

VEAudit from VEsoft, using its LISTUSER @.@ (CAP("SM")) filter, can give you a report of all of the SM users on your HP 3000. You can even ask for the SM users where password="". (Now there's a good list to find: SM users who have no passwords.) There is no MPE command that will do such things, we are reminded by VEsoft co-founder Vladimir Volokh. Even after more than three decades of his business as a 3000 software vendor, he also offers consulting on MPE operations and management, and still travels the US to deliver this. 

If you are worried about arbitrary access via QUERY, you can "disable subsystem access" via DBUTIL. This will, of course, only disable the access on QUERY.

Some less-adept auditors can also demand that a database's password be changed every 90 days. It's quite impossible to do, considering the database password is built into every application program.

So a database's security might be compromised through SM privileges, but it depends on the meaning of "update." This term can be construed to be as restrictive as using DBUPDATE to change an entry. It can also refer to UPDATE access DBOPEN MODE 2. 

To get very specific, an update can mean that the modify date has been changed in the file label of one or more IMAGE-related files. In a very general definition, an SM user can update the database simply by way of a restore from tape. (OP privileges permit this, too.)

Auditors sometimes ask broad questions, the sort of inquiry that fits better with the everyday use of HP 3000s in an enterprise. But for MPE/iX experts, "update" means any kind of modification capability.

So you can answer your auditor's question and say "no, SM privileges don't permit any of our users to update a database in another 3000 account." This answer is true, to the extent that the auditor's concern is about changing data — not just making a minor date change or using DBOPEN MODE 2. For auditors without MPE/iX and IMAGE expertise, well, they might not go so far in their examinations.

As for the SM user's ability to muck up an IMAGE database, it’s a mistake that is not difficult to make. An SM user who obtains a database password can corrupt an IMAGE database just by using the restore command. We’ve heard a story that such a user might explain, "Oops, I thought I was signed onto the test  account."

It's important to make a system fool-proof, because as Vladimir says, "fools are us." 

Posted by Ron Seybold at 04:31 PM in Hidden Value, Homesteading | Permalink | Comments (0)