July 07, 2017

Fine-tune Friday: opening disk, adding HASS

I need contiguous file space for my XM log file. How do I get this?

Many operations on the HP 3000 require contiguous disk space. Other files also require contiguous space; for example, consider the contiguous disk space on LDEV 1 required for an OS update. If you do not have one of the several third-party products that will create contiguous disk space on a drive, you may still be able to get enough free space by using CONTIGVOL.

However, occasionally, CONTIGVOL will stop with a message of “*Warning: Contigvol - Inverse Extent Table Full, Internal resource limit.” What can you do? Run it again. HP’s Goetz Neumann reported the message "is a warning that an internal table has filled up. It appears CONTIGVOL only handles looking at 40,000 extents at a time. You can run CONTIGVOL multiple times if the first run does not condense the free space enough because of this limitation.

I am adding two drives to a HASS (Jamaica) enclosure that already has several drives. How do I do this?

Gilles Schipper, Lars Appel and Chris Bartram reply:

First, a note of caution. If you dynamically add disk drives to, say, your MPEXL_SYSTEM_VOLUME_SET, you could find yourself in a pickle if you subsequently perform a START RECOVERY by accident or design. So while you can add drives dynamically as a convenience, it is a good idea to schedule a SHUTDOWN, START NORECOVERY as soon as possible to “fix” the new drives in your base configuration.

You do not even have to take down the system to add the drives to an HASS enclosure. The following steps will do the job.

• Set proper SCSI IDs. Make sure the SCSI addresses of the HASS enclosures are what you believe them to be. Do not make any assumptions. You need to set the SCSI address dip switches properly and ensure they are unique for the controller they are attached to. You will probably need a little flashlight to check the settings.

• Plug in the new drives.

• Use IOCONFIG to add the appropriate paths and device IDs. Note that the ldevs cannot be in use by, for example, vt or telnet sessions. So, you may still need to do this “off hours.”

• Use VOLUTIL to NEWVOL or NEWSET. For example, 

:volutil
>newvol mpexl_system_volume_set:member99 99 100 100 

(This example is for LDEV 99 — the “99” in member99 does not need to correspond to the LDEV number, it only needs to be unique for that volume set.)

It might be a good idea to first run the drives in a NEWSET for a while, exercising them a little. You could also use that extra volume set to exercise seldom used VOLUTIL commands or NEWACCT options like ONVS/HOMEVS. Finally, SCRATCHVOL them and add them to the desired volume set.

Posted by Ron Seybold at 09:55 PM in Hidden Value, Homesteading | Permalink | Comments (0)

Get e-mail notice when the NewsWire blog gets a new entry. Just say "Blog Me" in a message to editor@3000newswire.com.

June 30, 2017

Fine-Tune Friday: Jazz refuge, Query-JCL tip

Editor's Note: We're taking Monday July 3 off to celebrate Independence Day, and we'll be back on July 5 with a new report.

Where did the files HP hosted on jazz.external.hp.com go? So many articles reference that HP 3000 labs site.

The Jazz server contents were moved to several servers. A system at Speedware (a company now called Fresche Legacy) has much of what was hosted on Jazz. Those Jazz links at Speedware (now Fresche Legacy, and deeply absorbed with IBM iSeries work) are tucked away under hpmigrations.com. Not exactly the place where you'd look for homesteading tools, but available anyway.

How can I supply to QUERY a variable from within a JCL job? The physical paperwork for a file on the 3000 is being copied to digital format. We mark the files as deleted, (logically, not physically). Tracking this destroyed paperwork is done manually (a tedious and error prone process). How can I write something on the 3000 to create a comma delimited file of files shredded the day before?

John Long replies

This may help:

:SETVAR PREVDATE, !HPYYYYMMDD - 1
:SHOWVAR PREVDATE
PREVDATE = 20170607

I'm not sure how (or even if) query reads variables. (Nothing in the manual about it). Which is why you might have to 'build' your jobstream daily and replace the date in the 'USE file.'

Keven Miller shares this QUERY XEQ file

!JOB JCLQRY,MGR.KEVENM,DB
!
!setvar qdate1 "20170606"
!setvar qdate2 "20090710"
!
!echo find date=![qdate1] > q1
!echo find date=![qdate2] > q2
!
!query
b=stdb
;
5
s=upsell
xeq q1
r
d1,user-id,10
d1,date,20
d1,time,30
end

xeq q2
r
d1,user-id,10
d1,date,20
d1,time,30
end

exit
!
!EOJ


Then the spoolfile


:JOB JCLQRY,MGR.KEVENM,DB.
Priority = DS; Inpri = 8; Time = UNLIMITED seconds.
Job number = #j25.
WED, JUN 7, 2017, 2:38 PM.
HP3000 Release: C.60.00 User Version: C.60.02
MPE/iX HP31900 C.16.01 Copyright Hewlett-Packard 1987.
All rights reserved.
STREAMED BY KEV,MGR.KEVENM (#S118) ON LDEV# 9
STREAM DATE: WED, JUN 7, 2017, 2:38 PM

:
:setvar qdate1 "20170606"
:setvar qdate2 "20090710"
:
:echo find date=![qdate1] > q1
:echo find date=![qdate2] > q2
:
:query

HP32216D.03.20 QUERY/NM WED, JUN 7, 2017, 2:38 PM
COPYRIGHT HEWLETT-PACKARD CO. 1976

b=stdb
PASSWORD =
MODE =5
s=upsell
xeq q1

find date=20170606
0 ENTRIES QUALIFIED
END OF XEQ FILE

Posted by Ron Seybold at 07:31 PM in Hidden Value | Permalink | Comments (0)

June 23, 2017

Friday Fine-Tune: A 3000's Intrinsic Savvy

Homer-at-blackboardAs the clock counts down to the 10-year deadline for calendar services changes, our thoughts turn to HPCALENDAR. That's the intrinsic HP wrote for the 6.0 and 7.x releases of the 3000 OS, a new tool to solve an old problem. Alas, HPCALENDAR is fresher than the bedrock CALENDAR, but it's only callable in the 3000's Native Mode.

But poking into the online resources for MPE Intrinsics, I learned that once more HP's re-shelved its 3000 docs. Things have gotten better: everything now lives on the much-better-focused HP Enterprise website. You can, for the moment, locate the guidelines to intrinsics for MPE/iX at hpe.com.

The Intrinsics Manual for MPE/iX 7.x is also a PDF file at Team NA Consulting. Independents like Neil Armstrong help the community that's using HP's resources for 3000s these days. It used to be much simpler. In the 1990s, the Interex user group ran a collection of well-written white papers by George Stachnik. We're lucky enough to have them with us today, cut loose from ownership and firewalls. One is devoted to the system's intrinsics.

By the time The HP 3000--for Complete Novices, Part 17: Using Intrinsics was posted on the 3K Associates website, Stachnik was working in technical training in HP's Network Server Division. He'd first written these papers for Interact, the technical journal devoted to 3000 savvy for more than two decades. Even though Interact is long out of print, Stachnik's savvy is preserved in multiple web outposts.

Stachnik explains why intrinsics tap the inherent advantage of using an HP 3000.

When an application program calls an MPE/iX intrinsic, the intrinsic places itself in MPE/iX's "privileged mode." The concept of privileged mode is one of the key reasons for the HP 3000's legendary reputation for reliability. Experienced IT managers have learned to be very wary of application programs that access system internal data structures directly. They demand that MPE/iX place restrictions on HP 3000 applications, to prevent them from doing anything that could foul up the system. This is what led to the development of the intrinsics. Application programs running in user mode can interact with the operating system only by invoking intrinsics.

Even if your company has a migration in mind, or doesn't have an unlimited lifespan for the 3000, knowing how intrinsics work is an intrinsic part of learning 3000 fine-tuning that might be inside classic applications. Tools can help to hunt down intrinsics, but it helps to know what they do and what they're called. You can fine-tune your 3000 knowledge using Stachnik's papers and HP's Intrinsic documentation.

Posted by Ron Seybold at 08:34 PM in Hidden Value, Homesteading | Permalink | Comments (0)

June 16, 2017

Friday Fine-Tune: Cleaning Up Correctly

Classic 3000 Advice
By John Burke

Good intentions about maintenance sometimes stumble in their implementation. As an example, here’s a request for help on cleaning up.

Cleanup-tools“We have a 989/650 system. Every weekend we identify about 70,000 files to delete off the system. I build a jobstream that basically executes a file that has about 70 thousand lines. Each line says ‘PURGE file.group.account’. This job has become a real hog. It launches at 6 AM on Sunday morning, but by 7 PM on Sunday night it has only purged about 20,000 files. While this job is running, logons take upwards of 30 seconds. What can I do?”

This reminds me of the old joke where the guy goes to the doctor and complains “Gee, doc, my arm hurts like hell when I move it like this. What can I do?” The doctor looks at him and says “Stop moving it like that.” But seriously, the user above is lucky the files are not all in the same group or he would be experiencing system failures like the poor user two years ago who was only trying to purge 40,000 files.

In either case, the advice is the same; purge the files in reverse alphabetic order. This will avoid a system failure if you already have too many files in a group or HFS directory, and it will dramatically improve system performance in all cases. However, several people on the 3000-L list have pointed out that if you find you need to purge 70,000 files per week, you should consider altering your procedures to use temporary files. Or if that will not work, purge the files as soon as you no longer need them rather than wait until it becomes a huge task.

If all the files are in one group and you want to purge only a subset of the files in the group, you have to purge the files in reverse alphabetical order to avoid the System Abort (probably SA2200). PURGEGROUP and PURGEACCT will be successful, but at the expense of having to recreate the accounting structure and restoring the files you want to keep. Note that if you log onto the group and then do PURGEGROUP you will not have to recreate the group.

Craig Fairchild, MPE/iX File System Architect explained what is going on. “Your system abort [or performance issues] stem from the fact that the system is trying desperately to make sure that all the changes to your directory are permanently recorded. To do this, MPE uses its Transaction Management (XM) facility on all directory operations.

“To make sure that the directories are not corrupted, XM takes a beginning image of the area of the directory being changed, and after the directory operation is complete, it takes an after image. In this way, should the system ever crash in the middle of a directory operation, XM can always recover the directory to a consistent state - either before or after the operation, but not in a corrupted in-between state.

“On MPE, directories are actually just special files with records for each other file or directory that is contained in them. They are stored in sorted alphabetical order, with the disk address of the file label for that file. Because we must keep this list of files in alphabetical order, if you add or delete a file, the remaining contents of the file need to be “shifted” to make room, or to compact the directory. So if you purge the first file alphabetically, XM must record the entire contents of the directory file as the before image, and the entire remaining file as the after image.

“So purging from the top of the directory causes us to log data equal to twice the size of the directory. Purging from the bottom of directory causes XM to log much less data, since most of the records stay in the same place and their contents don’t change. The system abort comes from the fact that more data is being logged to XM than it can reliably record. When its logs fill completely and it can no longer provide protection for the transactions that have been initiated, XM will crash the system to ensure data integrity.”

Goetz Neumann added, “PURGEGROUP (and PURGEACCT) do not cause a SA2200 risk, since they actually traverse the directory in reverse alphabetical order internally. This is useful to know for performance reasons. Since these commands cause much smaller XM transactions, it is faster to empty a group by logging into it and then PURGEGROUP it, instead of using PURGE @.

“There is a little-known tool to help prevent you from running into these situations in the first place: DIRLIMIT.MPEXL.TELESUP. A suggested (soft) limit for directory files would be 2MB. This would limit MPE to not have more than 50,000 files in one group, and (very much depending on the filenames) much less than 50,000 files per HFS directory. (These are XM protected just as well, and tens of thousands of files in an HFS directory is not a good idea from a performance standpoint, either.)

“Another way to reduce the risk of SA2200 in these situations would be to increase the size of the XM system log file (on the volume set that holds the group with the large number of files), which is available in a VOLUTIL command.

Posted by Ron Seybold at 08:59 AM in Hidden Value, Homesteading | Permalink | Comments (0)

June 09, 2017

Friday Fine-Tune: Split up MPE/iX versions

Multiple-personalitiesHas anybody had success trying to split a single HP 3000 system into two different OS versions by using two sets of disks? There is no need for sharing of information between the two operating systems, and they would run independently of each other at different times of the day.

Guy Paul replies:
This is certainly possible, as we have done this in the past for customers who couldn’t tolerate any downtime for OS upgrades. Hence, we came up with a solution to have a duplicate set of SYSVS discs that we upgraded while they were still on the old OS. Come day of the ‘real’ OS upgrade, we brought them down, stored off any modified files, switched over to the new OS, restored any modified files and they had an OS upgrade in about 45 minutes. So it is possible.

You should probably consider using BULDACCT to synchronize the accounting structure.

Gilles Schipper adds:
This should be entirely possible. I do this sort of thing all the time. By simply booting from the appropriate boot path, you can do exactly as you wish. In fact, I have even shared common volume sets among different LDEV 1 system volume sets, with different MPE versions.

What's the name and syntax of the Posix utility that allows you to assign a new file, give the file the same name as the one you're replacing—and the new file will replace the old file when the last user closes it?

Andreas Schmidt replies:
You want to use mv, a utility to rename and move files and directories. The syntax is

mv [-fi] file1 file2
mv [-fi] file... directory
mv -R|-r [-fi] directory1 directory2

Where do I start to configure my HP Laser printer to be a network printer off the HP 3000. Don’t I need steps for I need steps for the NMMGR process?

Jeff Woods replies:
Odd as it might seem, network printing isn’t configured using NMMGR but instead depends on SYSGEN devices with the ID HPTCPJD, plus a text file called NPCONFIG.PUB.SYS defining each printer LDEV.

(This might have more than one LDEV associated with each IP address, perhaps to give different environments such as landscape vs. portrait or duplex printing or special margins or line spacing as needed).

The process is pretty well documented in Chapter 3 of HP's Native Mode Spooler Reference Manual (tip of the hat to NA Consulting for the manual link).

Posted by Ron Seybold at 06:32 PM in Hidden Value | Permalink | Comments (0)

June 02, 2017

Sendmail fine-tunes, if you still need delivery

By Andreas Schmidt,
with Mark Bixby, and Jens von Bülow

Relying on the HP 3000, you may want to use this box for incoming and outgoing e-mails as well. This is possible using a collection of software bundled with the HP 3000, or available in the public domain:

• Sendmail/iX, the mail transport agent well known on HP-UX that was ported to MPE/iX;

• Syslog/iX, the event logging subsystem required by Sendmail/iX;

• MAILX.HPBIN.SYS, the mail reader, and;

• Qpopper, the POP3 protocol for downloading.

Sendmail/iX and Syslog/iX have been ported to MPE. MAILX.HPBIN.SYS is part of the Posix shell part of each MPE/iX release since 4.5. The combination of all four of these utilities will enable your HP 3000 to receive Internet e-mails sent to you@host.domain and send Internet e-mails into intranets.

Syslog/iX

Syslog is the standard event logging subsystem for Unix. It consists of a server daemon, a client function library, and a client command line utility. It is possible to log to files, terminal devices logged on users, or forward to other syslog systems. Syslog can accept data from the local system via an AF_UNIX socket, or from any system on the network via an AIF_INET UDP socket on port 514. The sendmail mail transport package is one of the Internet tools which log to syslog. Syslog/iX is bundled with MPE/iX in the SYSLOG account. If somebody was a little too aggressive about cleaning up unused FOS files, you can restore the SYSLOG account from the backup of your OS. Otherwise, you can locate your FOS tape and manually extract and install the SYSLOG account.

Sendmail/iX

Sendmail is a mail transport that accepts fully formatted e-mail messages from local host system users, queues the messages, and then delivers the messages to local or remote users. It listens on TCP port 25 for incoming SMTP messages from remote systems, and delivers these messages to local host system users by appending the message text to the user’s mailbox file.

Sendmail is not a mail user agent. It does not have the ability to compose or to read e-mail. To cover this functionality, HP bundled the program /SYS/HPBIN/MAILX into the shell utilities. Sendmail is also not a POP3 server that will enable network clients to access Sendmail/iX mailboxes.

MAILX.HPBIN.SYS

This program helps read and send electronic mail messages. It has no built-in facilities for sending messages to other systems. But combined with other programs (a mail routing agent and a transport agent like Sendmail/iX) it can send messages to other systems. MAILX only offers limited support for various message headers (i.e. Subject:, From:, To:, Cc:, etc). If you need to do anything fancy, like MIME headers, you’ll need to call SENDMAIL.PUB.SENDMAIL directly and pass it a fully formatted message containing all headers and body text.

To read messages from your mailbox in /usr/mail/ type :MAILX.HPBIN.SYS

To send messages use :MAILX.HPBIN.SYS [options] user1 user2 ... An :EOD finishes the message text.

The files in /usr/mail/ are named USER.ACCOUNT and are accessible only for this user.

Qpopper

Qpopper is a server that supports the POP3 protocol for downloading Internet e-mail from software clients. Qpopper does not include a message transfer agent or SMTP support but normally works with standard Unix mail transfer agents such as sendmail. On MPE/iX it works therefore perfectly with Sendmail/iX.

The installation procedure basics are:

• The link /usr/local/bin/popper must point to /SYS/ARPA/POPPER.

• In SERVICES.NET.SYS, port 110/tcp must be reserved for pop3 service.

• INETDCNF.NET.SYS must start this service via pop3 stream tcp nowait MANAGER.SYS /SYS/ARPA/POPPER popper.

• For relaying via Sendmail/iX, a file /etc/mail/relay-domains must exit in mode 644 (-rw-r—r—) owned by MGR.SENDMAIL.

Having successfully installed this, you may now change your Internet browser so that your HP 3000 is the incoming POP3 server. You may do as we did: we created a new account POP3 with plain vanilla users per mailbox. The PC e-mail client needs to be configured in the following way:

Server Name: your POP3-enabled HP 3000

Server Type: POP3 Server, User Name: USER1.POP3 (e.g., SCHMIDA.POP3)

You may want to remember to set a password and an adequate check time for new e-mail. It’s up to you whether you want to download the new messages to the PC and not to keep on the host or not.

A nice feature is the aliasing in Sendmail/iX. Your HP 3000 acts as a POP3 and SMTP server for all Internet e-mail software agents.

Is there a proper way to shut down sendmail?

• Use the Posix kill signal from SERVER.SENDMAIL or any user with SM capability. (The following can be easily turned into a job.)

kill $(head -n 1 /etc/mail/sendmail.pid)

• Only use :ABORTJOB as a last resort! (This is true for all of the Posix things that got ported to MPE)

If you don't need to run a mail server (e.g. sendmail) on your 3000, you shouldn't. In most cases, using a mail client will be "just the ticket." Point the client at your in-house (SMTP) mail server and enjoy.

 

Posted by Ron Seybold at 07:29 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 26, 2017

Friday Fine-Tune: Tape to Disk, Posix time fix

C1539 tape driveEditor's Note: Monday is the Memorial Day holiday here in the Newswire's office. We'll be back with a new article on May 31. With every day that passes, HP's original hardware gets older and more likely to fail. Virtualization of hardware brings newer hardware into service with 3000s and helps with this problem, a risk that's greatest on the 3000's moving components. Especially tape drives like the C1539 above; even HP's discs are less likely to fail. Enjoy this advice on how to put stored data from tape into a store-to-disk file, as well as keeping dates accurate in MPE/iX's Posix name space.

I have some information on a tape. How do I create a store to disc file with it?

There are a few solutions. The first and easiest is to simply restore the info to a system (RESTORE *T;/;SHOW;CREATE;ACCOUNT=WORKSTOR) where WORKSTOR is an account you create to pull the data in.  Then a simple FILE D=REGSFILE;DEV=DISC and STORE /WORKSTOR/;*D;whatever else should create the disc store.

The second is to use FCOPY. The STORE format is FILE TAPEIN;DEV=TAPE;REC=8192,,U,BINARY.

John Pitman adds, "If you mean copy it off tape to disk store file, I’m not sure if that can be done. In my experience with tapes, there is a file mark between files, and EOT is signified by multiple file marks in a row. But it may be possible. If you do a file equate and FCOPY as shown below, you should be able to look at the raw data, and it should show separate files, after a file list at the front.

FILE TX;DEV=TAPE;REC=32767
FCOPY
FROM=*TX;TO=;CHAR;FILES=ALL

Here is our current store command, and the message it provokes. MAXTAPEBUF speeds it up somewhat

STORE  !INSTOREX.NEW.STOCK2K;*DDS777;
FILES=100000;DIRECTORY;MAXTAPEBUF

Why is the date/time in the Posix shell way off from the time on MPE, and what can be done to fix it? It’s over three weeks off.

Homesteading Editor Gilles Schipper replies:

First, check to ensure your timezone offset is correct and there are no pending time clock changes.

:showclock

SYSTEM TIME: TUE, OCT 19, 2010,  5:46:38 PM
CURRENT TIME CORRECTION: 0 SECONDS
TIME ZONE:  4 HOURS  0 MINUTES WESTERN HEMISPHERE

If the incorrext timezone and/or time correction is non-zero, you can fix both with the :SETCLOCK command.

Next, ensure that the TZ variable is appropriately set. This can be done with a system logon UDC that executes the following:

comment the following is for Eastern Time
SETVAR TZ "EST5EDT"
comment use the following for california
comment SETVAR TZ "PST8PDT"

Posted by Ron Seybold at 06:11 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 12, 2017

How to Step Through a CSLT Reinstall

Stepping-stonesAssume you've seen your Series 918 crash to its bones. You replace the 3000 and need to reinstall from CSLT. Many a 3000 site hasn't done this in a long time. This is when you're getting in touch with your independent support company for advice and walking the steps to recovery.

Oh. You don't have a support company to call. One more thing that's been dropped from the budget. You could ask on the 3000 newsgroup for help, so long as your downtime isn't serious. This support strategy is one way to go, and James Byrne got lucky this week. One expert walked him through the critical steps.

"I am working my way through the check-lists for reinstalling from a CSLT," Byrne said, "and I have come to the conclusion this stuff was written more to obscure than to illuminate." The HP documentation advised him to boot from disk, something he couldn't do. "Fortunately this is a backup 3000, and nothing too bad can happen yet."

Gilles Schipper, our esteemed homesteading contributor, provided the answers. The problem lay in a bad boot drive, but how do you discover that's true? We'll get to that in a bit. First, the CSLT reinstall.

"Assuming you have no user volume sets:

1. Mount CSLT in appropriate drive and boot from alternate path
2. From prompt, type INSTALL
3. At completion, boot from primary path and START NORECOVERY (this is your second "boot".)
4. Mount whichever tape contains your directory (could be the CSLT or your latest full backup - if directory on both, use whichever is more current)
5. Log on as MANAGER.SYS and restore directory (:file t;dev=?;restore *t;;directory - note 2 consecutive;)
6. Use VOLUTIL to add additional discs to MPEXL_SYSTEM_VOLUME_SET.
7. Mount backup tape and:
:restore *t;/;keep;create;olddate;partdb

Reboot with START NORECOVERY

Ah, but what to do if you have user volume sets?

"So," Schipper said, "with user volume sets:

Modify step 6 to read as follows:
6. Use VOLUTIL to add additional discs to MPEXL_SYSTEM_VOLUME_SET, as well as any additional user volumesets that you have.
The approprite VOLUTIL commands you will need are:
1. NEWVOL command to add additional volumes to the existing MPEXL_SYSTEM_VOLUME_SET
2. NEWSET command to add first (master) volume of user-volumeset1, say
3. NEWVOL command to add additional volumes to user-volumeset1
4. repeat 2 and 3 for each additional user volumeset

Then, prior to step 7, add step 6A, which is the repeat of step 5, which is to AGAIN restore the directories from the tape containing the directories so that they are restored to the master volumes of their respective user volumesets.

And then step 7 remains as before.

The bad LDEV1 discovery? You can always swap out a known good drive. Byrne did this and noted he had to restock his cache of backup drives. "Only three left," he said. "Time to order more, perhaps." Advice on checking for bad drives before a boot:

"Instead of INSTALL, type ODE," Schipper said. "Then run Mapper. That should show you all of your devices including hard drives. Although that may even show drives that cannot be INSTALL'ed to. Also, check all of your SCSI terminators."

Stan Sieler added this to the remedies for a bad disk. "Try doing:

  start norecovery message

"That 'message' option will cause 'start' to generate a **LOT** of output. Hopefully, the last few dozen lines might provide a clue to the problem."

Do all the backup that you need when protecting that HP 3000. The success of its backups falls in the lap of the system's managers. (Running CHECKSLT from the TELESUP account will verify if a CSLT is still good enough to boot your system. You'll want to check that CSLT to ensure it'll run on any tape drive, not just the one it usually runs on. Alignment issues kick DDS drives out of service regularly.) Don't forget about keeping your CSLT healthy.

Posted by Ron Seybold at 03:53 PM in Hidden Value, Homesteading | Permalink | Comments (0)

April 28, 2017

Friday Fine-tune: directories and tombstones

ByetombstoneA 3000 manager wanted to know about adjusting privileges on their server. When the community's veterans started to respond, extra information rose up. Some of was about the management of files in MPE/iX, the kind of legacy recorded on what's known as a tombstone.

Tombstones are data used to solve 3000 problems and establish file access. HP says in its manual for programming in MPE/iX that "It's frequently necessary to obtain status information on a file to determine the cause of an error." A File Information Display is frequently called a tombstone, providing:

  • Actual physical and operational file characteristics.
  • Current file information, pertaining to end of file, record pointer, and logical and physical transfer count. Information on the last error for the file and the last HPFOPEN or FOPEN error.
  • When a file is opened, the final characteristics may be different from those originally requested because of defaults, overrides, :FILE commands, and the file label.

You can use the PRINTFILEINFO intrinsic to print a tombstone. It requires that you specify the file number returned when the file is opened by HPFOPEN or FOPEN. The tombstone can display either a full or short format.  If the file is open, it provides a full display. Otherwise, it provides a short display. Calling this intrinsic does not automatically abort the program.

You can call the PRINTFILEINFO intrinsic from programs written in COBOL II/XL and HP FORTRAN 77/iX. When calling from COBOL II/XL, use the FD filename. You can call the name PRINTFILEINFO directly from HP FORTRAN 77/iX programs. You can obtain the required file number by using the FNUM intrinsic.

Tombstones came up after one list member resurrected an answer about privileges from a 11-year-old post. Ray Shahan, still managing archival systems for Republic Title of Texas, heard his name in discussion about TD and RD privileges and how to control them. He quipped about not being heard from in ages.

"I have been asked by our security group to remove TD and RD privileges from our HP 3000," Reggie Monroe wrote this week. "These are for Reading and Traversing Directories. Does anyone know what the impact of this would be, if any?"

Tracy Johnson replied that "Unless your users have access to Posix files, you can categorically state you don't have any to remove."

There is an old comp.sys.hp.mpe posting where Ray Shahan wants to add TD and RD privileges. Just do the opposite, though that may be a bad thing if applied to MPE groups and accounts treated as directories.

The original TD and RD posting

The advice from the 2005 discussion included using Posix to enable "execute" permissions on all directories needed to get to the directory you want. So the opposite would be to disable those permissions. The ALTSEC command does this.The process will also include adding ACDs to the directory.

Once considered a new feature of MPE/iX, Access Control Definitions are pseudo bits of information on the HP 3000.

ACDs are ordered lists of pairs.The pairs are made up of access permissions and user specifications that control access to objects. Objects are passive entities that contain or receive information, such as files, directories, and devices. Each entry in the ACD specifies object access permissions granted to a specific user or group of users. In addition to being granted access to an object protected by an ACD, users can also be granted access to read the ACD itself.

ACDs can be applied to any MPE/iX files using the ALTSEC command. This command was enhanced to support directories. If a file has an ACD, this method of specifying access to the file takes precedence over other security features, such as lockwords and the file access matrix. ACDs cannot be placed on root, account, group, or directories.

 

Posted by Ron Seybold at 11:37 AM in Hidden Value, History, Homesteading | Permalink | Comments (0)

April 10, 2017

3000 backup strategy for closing Sundays

ClosedOnSundayEaster Sunday is on this week's horizon. While it's a rare day of closure at our local HEB grocery chain, Sundays are another sort of closure for 3000 managers. Nearly all of them want their partial backups of the weekdays to wrap up before the backup begins that will serve the work week. If you do full backups every night and want to make the new strategy to do partials during the week and a full on Sunday, there's a way to make that work. Donna Hofmeister, one of the former OpenMPE directors, explained the strategy in a message to 3000 managers.

First you need to decide what kind of partial you want to do.  On Tuesday, do you want to backup all files changed since Sunday's full backup or do you want to backup all files changed since Monday's backup?  (and so on....)

There are some things to think about here. If your "line in the sand" is always Sunday, then you have to deal with knowing that by Friday/Saturday your "partial" backup is likely going to be sizeable and will take longer to run. On the other hand, if you ever have to do a big restore, your restore plan is plan is pretty simple -- you'll need your last partial and your last full backup.

If your "line in the sand" is always yesterday, then your "partial" backups will be relatively small and quick. The flip-side is your big restore could be very complicated, since you'll need every partial backup through Monday plus your full backup. I think most people set the backup date as "Sunday" and do partials from there. But there's a technical bit that's also important.

The bit concerns the command restore ....;date>=!my_lastfull. Donna went on to explain.

"!my_lastfull" is a CI variable that contains a date in "mm/dd/[yy]yy" format.  This is a value that your *full backup job* needs to establish.  So your full backup job should do something like:

         file lastfull,old
         echo !hpmonth/!hpday/!hpyear>*lastful

Your partial job will need to do something like this:

         file lastfull,old
         input my_lastfull < *lastfull

There's plenty of logic that needs to be added to the above examples. (You'll note that there is NO error checking/what-if handling, etc.) This should be enough to get you started.

On incremental vs. differential backups — the former is a backup of changed files since the last full OR partial. The latter is a backup of changed files since the last full.

HP's wisdom about date management for the built-in STORE facility for HP 3000s is in several places. 3K Ranger's Keven Miller volunteered his, and Neil Armstrong's TeamNA Consulting has a downloadable one on file as well.

Posted by Ron Seybold at 07:40 PM in Hidden Value, Homesteading | Permalink | Comments (0)

April 07, 2017

Friday Fine-Tune: Creating the perfect CSLT

Editor's note: A classic technique, detailed here by the NewsWire's Hidden Value editor emeritus John Burke.

PerfectTapeA question about creating a CSLT for a Disaster Recovery test turned into a general discussion about what the perfect CSLT should look like. A system manager wanted to use the STORE option on the SYSGEN TAPE command to store additional files onto the CSLT he was creating for his DR test but was running into trouble trying to specify STORE options as part of the SYSGEN TAPE command. In particular, he wanted to simply add ;SHOW to get a listing of all files stored.

The answer to his original question is to use an indirect file, as in

sysgen>TAPE MODE=VERBOSE DEST=OFFLINE STORE=
^CSLT.INDIRECT.SYS

where the indirect file contains whatever STORE directives you want in addition to the file list.

One contributor recommended robust efforts to get a listing: “A backup tape is of limited value without a listing. For Disaster Recovery purposes it is also a good idea to have the original HP tapes and patches with you as it is possible to create an SLT that does not install or work on a different HP 3000 system.” This same contributor also suggested creating a disk file with a listing of all the stored files.

However, several people questioned whether this list of files which the thread originator had proposed storing was sufficient. Stan Sieler probably said it best:

“I’d put much more in the STORE section, at the minimum:

/SYS - /SYS/MPEXL/DUMPAREA - /SYS/PUB/NL - /SYS/PUB/SL -/SYS/PUB/XL

(To explain, NL, SL, XL are dumped in the CSLT portion, so no need to dump them in the STORE section; DUMPAREA is a 32Mb file created at INSTALL time and there’s absolutely no need to dump it to a tape.)

/TELESUP - /TELESUP/DUMPS
(or wherever you put your dumps.)

/ALLEGRO, /LPSTOOLS, /REGO, /ROBELLE, /VESOFT
(It’s surprising how much you might want tools at an early stage.)

“If most of your user data is in one or two accounts, other than SYS, TELESUP, and the rest of the system might fit well onto one DDS/DLT, you might find it more useful to do:

/ - /USERS - /SALES

(where USERS, SALES are the ‘user’ accounts)

“Why? It guarantees you’ll get everything you’re likely to need in a recovery/install situation (except, of course, for the major portion of the user’s data). I’d also specify:

;show;directory=MPEXL_SYSTEM_VOLUME_SET,…etc.; progress;partialdb

Posted by Ron Seybold at 07:14 PM in Hidden Value, Homesteading | Permalink | Comments (0)

March 30, 2017

Puts, Gets, and Serving Up Transfers Faster

Server TrayHP 3000s are exchanging files with other servers, a process that's included the FTP protocol for more than 15 years. This capability was once so magic that the arrival of Samba file exchange on MPE/iX was lauded as a breakthrough. FTP is quite a ways off the most current of transfer protocols. One manager's started a discussion about how to improve transfer speeds to and from the 3000, though. He's using DSCOPY as well, but prefers the PUTs and GETs of FTP.

The advice that's current about FTP/iX says that hard-coding the 3000's ports (100mb full duplex, or 10mb half) is one way to speed things up. Ensuring your traffic is not running through a proxy (called "being proxied) is another idea. Measuring the speed of a PUT against a GET is one step in discovering why the 3000's FTP might seem slow.

In 2008, MPE/iX gained a secure version of FTP—at least part of one. This SFTP functionality arrived at the end of the Hewlett-Packard lab era for 3000s, a period when new tools were not being placed into wide use. Sites were locking down their 3000 scope of operations to ensure stability. The port of this then-current functionality fell short of complete: only an FTP secured client got created. PUTs could be secured, as well as GETs. But only from Windows, Unix, or Linux hosts to the 3000. The 3000 wasn't going to dish out files using secured FTP. There are notes in place to carry the work forward, though.

MPE/iX tools and components are also out there to complete this securing of file transfers. OpenSSH is the best-known protocol. A quick-start bundle can be downloaded from the MPE-OpenSource website run by Applied Technologies. There are SFTP installation instructions at Applied, too. Someone who's got a need for securing FTP transfers will need to do the server side of the porting, which was completed on the client side by Ken Hirsh, Mark Bixby, and Mark Klein. Requests to speed up FTP are a sign this porting would be more than just an open source hobby project.

Brian Edminster, the senior consultant at Applied Technologies, explained that "with a bit of work, you could get OpenSSH v 3.7.1p2 working. The issue is that 'select' is busted under MPE/iX, and that's what's required for ssh to work correctly."

The fact remains: ssh cannot connect to a remote system and execute commands that produce any output. Ken Hirsch did the original port, but he only really needed the SFTP client -- so the issue with ssh wasn't addressed.   

Ken also posted on the 3000-L newsgroup in 2008, asking if there was any interest in getting an ssh and sshd/sftp-server working (server daemon) -- so the 3000 could do port forwarding, act as a SFTP server, receive inbound ssh connections, and so on. Apparently he didn't get enough response to carry forward.

Back in 2005, Hirsch posted his goal. 

I could get an interactive ssh client to work on MPE/iX.  I don't know how, but I know it's possible! It would not be possible to get an ssh server working in such as way that an ssh client could run any program. But it would be possible to get enough of the server running so that you could use the server to do port forwarding.

In 2008, he added the note which Edminster referenced. "If anybody knows a way to actually write to a terminal while there is a read pending," Hirsch said, "I could use OpenSSH as a server on the HP 3000. Apparently there are undocumented MPE/iX sendio() and rendezvousio() calls, of which I know nothing. There are also tread()/twrite() routines in libbsd.a that I think are intended for this, but there's no documentation for these, either."

Posted by Ron Seybold at 07:11 PM in Hidden Value, Homesteading | Permalink | Comments (0)

March 17, 2017

Friday Fine-Tune: Moving all disks at once

I want to take all my disks, system volume set, and the user volumes, and move them to another machine that has no disks. How can I best do this?

Lars Appel replies

You might use SYSGEN to create two different config groups in the SYS account, one for the old system (e.g. CONFOLD), and one for the new system (e.g. CONFNEW). To create the new config you might start with one of the existing config templates, e.g. CONF9x8.SYS for 3000/9x8 systems. Use BASEGROUP to open it, adjust IO to your needs and KEEP it as CONFNEW.

7935Just make sure that every disk gets an LDEV in the new config. Paths and LDEVs (except for LDEV 1) do not need to be the same on the new system. MPE/iX will notice (and "collect") all available volumes during bootup (as long as they are "visible" by a configured LDEV number).

Shutdown the system, plug disks into the new box, power it on, boot from the primary path (or enter the appropriate path for LDEV 1) and at the ISL prompt do a START NORECOVERY [NOSYSSTART] GROUP=CONFNEW.

You might also need to adjust NMCONFIG with NMMGR as a different system will probably have the LAN cards at a different physical path. In case you make a copy of NMCONFIG, make sure it stays on LDEV 1 (FILE;DEV=1).

We replaced LDEV1 on our HP 3000, did an INSTALL. Then we did
RESTORE *T;/;DIRECTORY;SHOW;KEEP;OLDDATE
The MPE-filespace was restored normally, but the HFS filespace was not. RESTORE told me to use CREATE=PATH which I then did. The DIRECTORY is on my STORE tape - so what went wrong?

Wolfgang Kinscher, and Gilles Schipper reply:

If you are planning an install keep this in mind: Do not specify the option "DIRECTORY" with a partial (DATE>=) backup. STORE will not put the HFS directory structure on tape. Do the following instead: STORE ;*T;DIRECTORY stores all of your MPE/HFS directories on tape. Then do your partial backup without DIRECTORY option

After the install, first restore your DIRECTORY with RESTORE *T;;DIRECTORY and then restore your partial and your latest full backup respectively.

Posted by Ron Seybold at 09:13 PM in Hidden Value | Permalink | Comments (0)

March 10, 2017

Friday Fine-Tune: Going Beyond JBOD

By Gilles Schipper

Mod 20sOne of the most cost-effective ways of advancing the reliability of your legacy system may be to replace your existing “JBOD” disk system with a much more reliable disk system. MOD20 units, still a better deal than individual disks, can provide a good starting point to implement RAID. JBOD is an acronym meaning “just a bunch of disks” — which would characterize the majority of HP 3000 systems as they were initially sold. JBOD disk systems comprise a set of independent — typically SCSI-connected — disks, which are each seen by the HP 3000 as a single logical device number or LDEV. Each disk LDEV is associated with a “volume set” and the failure of a single disk renders the “volume set” to which it belongs inoperable and un-accessible.

Traditionally, most 3000 systems have comprised a single volume set (specifically, the required SYSTEM volume set, with the brevity-challenged label “MPEXL_SYSTEM_VOLUME_SET”).

Systems comprising a large number of “JBOD” LDEVs increased the likelihood of system downtime, since the failure of a single, old disk effectively resulted in a “down” system — requiring a time-consuming disk replacement and system reload before the system could properly function once again.

To mitigate such delicate exposure to a single disk failure, many installations implemented the “User Volume Set” feature built in to MPE/iX, then constructed multiple volume sets so that the failure of a single disk affected only the volume set to which it belonged.

For practical purposes, the only real benefit to this approach was to reduce the amount of time required to replace the disk and reload only the data residing on the affected volume set. (In reality, it was usually quite unusual for a system to continue normal, or even minimal operation with even a single unavailable volume set).

To further improve system reliability and minimize downtime an optional, additional-cost software  product was available in the form of software mirroring — aka “MPE/iX mirroring.”

This enabled the system administrator to configure non-system volume set disk drives to be associated with  identical corresponding “mirror” disks. The software was responsible for dynamically duplicating the contents of  both disk drive “mirrors” such the failure of one of the two mirror drives could be tolerated without affecting the continuous operation of the system. The damaged disk could then be replaced and the dynamic disk duplication would resume.

Only if both mirror pairs failed would there be a corresponding system outage and data loss. However, software mirroring was still far from ideal. Since it was unavailable for the MPEXL_SYSTEM_VOLUME_SET, the failure of a system disk, unprotected by mirroring software, would result in certain system down time.

Further, software mirroring exacted a price in terms of CPU and I/O overhead that could otherwise be utilized for actual “useful” processing. 

And, as a wise person once said, given a choice, a feature is almost always better implemented in hardware than software. This certainly applies to disk mirroring and nicely aligns with the the Nike MOD20 RAID disk system, which is (one of the) HP 3000’s solutions to the compromises associated with software mirroring.

The MOD20 features dual controllers, duplicated (even triplicated) power supplies, and up to 20 disk drives housed in a single frame/enclosure that provides significant improvements over the MPE/iX software mirroring functionality.

Each MOD20 provides for a maximum of 8 logical units (LUN’s) to be configured — each of which appears as a single logical device no. (LDEV no.) to the HP 3000. A maximally and optimally configured MOD20 will include 20 disk drives and be configured as follows:

14 disks to be defined as type RAID1, using up 7 LUNS—since each LUN comprises two separate mirrored disks. RAID level 1 is equivalent to simple mirroring whereby one disk is dynamically maintained as a duplicated mirror image on its mirrored twin disk, which must of identical size and model.

If one disk of the mirrored pair fails, the other disk can take over the responsibility of presenting the requisite data IO to and from the host system with no perceived performance degradation. The remaining 6 disks can be configured as a single LUN comprising 4 RAID 1/0 disks and 2 hot spares.

A RAID 1/0 configuration takes an even number of disks and duplicates the contents of half of them (as a group) onto the other half.

The hot spares would act as dynamic replacements for any disk in the MOD20 that fails, such that even the  failure of one or two disks would not prevent the entire disk subsystem from maintaining its fail-safe mirroring capability. Without the hot-spare feature, failure of a single disk would allow normal system activity to continue but without further fail-safe capability for the failing LUN only.

Chances of both disks in the same LUN failing are extremely remote. That is why I advise you to forgo the hot spare capability. Utilize a 6-disk RAID 1/0 LUN instead of a 4-disk RAID 1/0 LUN, giving you additional usable disk space overall.

Posted by Ron Seybold at 08:04 PM in Hidden Value, Homesteading | Permalink | Comments (0)

March 03, 2017

When and how to back up 3000 directories

Editor's Note: Homesteading Editor Gilles Schipper weighs in on  using the 3000's store directory option, rather than invoking the buldacct program, to make clean backups.

By Gilles Schipper
Homesteading Editor 

CityDirectoriesIt's common to see confusion surrounding the use of the ;directory store option versus the buldacct directory creation program. In order to benefit from the store ;directory option, one has to utilize the option almost perfectly in both the store and the restore following a system INSTALL. Consequently, it becomes much easier to fall back on the buldjob options to re-create the directory -- although that option is inferior.

In order to be able to effectively utilize the directory option, the first thing that must be done properly is to ensure that the appropriate ;onvs= option is also used in the case where user volumesets are utilized. Otherwise, the non-system volumeset directories do not get restored after the INSTALL since they are not on the tape.

But even if the store part is done correctly, the other opportunity to go wrong presents itself during the reload process.

The proper procedure during reload is as follows:

1. perform INSTALL
2. restore ;directory from tape
3. re-create disk and volumeset environment via VOLUTIL

Then -- and this where many go wrong,

4. Again restore ;directory from tape (this re-creates the volumset directory environment on the master volumes for all user volumesets for those utilizing it)
and then
5. restore files
6. reboot with start norecovery (to enable network functionality)

Of course, for those that do not utilize user volumesets, the directory option becomes much less error-prone. And, for those that utilize third-party backup utilities, the ;directory option -- as utilized in the MPE store command -- is generally replaced with a similar option in the various backup utilities.

The bottom line: for those that utilize the MPE store command to perform their backups, the properly-used ;DIRECTORY (and, if appropriate) corresponding ;ONVS= options) -- together with correct restore procedures as indicated above -- will get the desired result 100 percent of the time. Notwithstanding, of course, any tape issues occurring, which would be problematic no matter which directory re-creation option is used.

The bottom line is that the proper way to perform a full backup if you're using the MPE backup facility:

:file t;dev=tape
:store /;*t;partdb;directory;onvs=mpexl_system_volume_set,big;
maxtapebuf[;progress=5;show;online[=xxxx]]

Of course, upon further reflection, and better than the store command: use sysgen to create a backup tape that not only contains all files, but also the SLT -- so that this one tape alone can be used to INSTALL and RELOAD your system. The use of sysgen for such purpose will require use of an indirect file.

Posted by Ron Seybold at 09:12 PM in Hidden Value, Homesteading | Permalink | Comments (1)

February 24, 2017

Friday Fine-Tune: Opening Up MPE's Shell

Way back in the middle 1990s HP added the Posix shell to the HP 3000. The improvement meant customers who had Unix and MPE running in the same shop could train operators and managers with a single set of commands. Posix was a plus, making the 3000 appear more Unix-like (which seemed important at the time).

Over the years, however, Posix has been a feature waiting be discovered for most 3000 managers and operators. The computer's operating system was renamed from MPE/XL to MPE/iX just for this added Posix feature. But enough history; Posix is still on the 3000 and remains a powerful interface tool, an alternative to the CI interface that HP created for the system. You can even call Posix commands from the CI, a nifty piece of engineering when it can be done.

That's not always possible, though. A customer wanted to know how to "expand wildcard shells" using Posix. He tried from the CI and had this story to relate.

:LL /BACKUPS/HARTLYNE/S*
ls: File or directory is not found

So how do I do this? I need to be able to tell tar to archive all of the reels of a STD STORE set via a regexp. It does not work in tar, and it apparently does not in ls—so I speculate that there is something special about the innovation of Posix utilities from the CI that I am not aware of. What is it?

Jeff Vance, the 3000 CI guru at while at HP, replied "Wildcards on most (all) Unix systems, including Posix implementations, are done by the shell, not the individual programs or in-lined shell commands, like ls in your example. A solution is to run the shell and execute ll from within.

The magic Posix shell command to do the expansion:

Screen Shot 2017-02-27 at 11.22.22 PMAn interesting footnote if you've read this far: The Posix shell for the 3000 is one part of the operating system that was not built by HP. The shell was licensed by HP from MKS, and Hewlett-Packard paid royalties to MKS so Posix could work inside of MPE/iX. That was an issue that posed a potential snag for source code licensing from HP. But the outside license issues never ended up blocking emulation or source-license arrangements. Managers have used Posix on the 3000 as a way to get familiar with commands in Unix systems. In the great majority of instances, these commands are the same.

Posted by Ron Seybold at 11:10 PM in Hidden Value, Homesteading | Permalink | Comments (0)

February 03, 2017

Fine-tune Friday: Care and feeding of UDCs

Screen Shot 2017-02-06 at 1.53.00 PMMercury Insurance is a long-time HP 3000 shop still running a server in production. Last week Reggie Monroe reached out for a refresher on administration of HP 3000 User Defined Commands (UDCs). These are the HP3000's equivalent of scripting in Unix environments. UDCs are a better version of Command Files, according to Jon Diercks and his MPE/iX System Administration Handbook. UDCs are catalogued, Diercks says, so they can be loaded for individual user accounts.

UDC definition
Click for details

There's a superior PowerPoint slide deck online at the 3K Associates website that covers how to create and use UDCs. But the Diercks book (no longer in print, but available online) is more concise on the use of UDCs. It's also only available as an $80 book today on the used market; put yours in a safe place. Monroe's question asked about "a command to list all users, and the logon UDC associated with them, if one is set."

The initial answer was the command HELP SHOWCATALOG,ALL. This brings an administrator to

SHOWCATALOG [listfile][;USER=username[.accountname]]

But Alan Yeo pointed out that the MPE/iX command only locates system-level UDCs. 

You don't actually get what you think you asked for, so whilst :showcatalog ;user=@.@ sounds very hopeful, in fact it only shows the system level UDCs not account ones. As far as I'm aware the only place you can find them all is in the BULDJOB2 file in PUB.SYS. You do have a BULDJOB2 file don't you? And it's up to date?

And here's where Vesoft's utility does a job the 3000's OS cannot. VEAUDIT LISTUDC @.@ finds UDCs of all kinds.

We have chronicled much of MPEX during the 21 years of the NewsWire's publication. The utility was even the sole subject of the Inside Vesoft column back in the era when HP was starting to lock down 3000 futures. In 2002 Steve Hammond illustrated the distinction of UDC administration under VEAUDIT. It becomes important because security on a 3000 includes management of the UDC catalogs. And yes, there's a tool for the security, too.

VEsoft’s Vladimir Volokh told me he had been asked to find out if any users on a system had the VEsoft utility GOD in one of their UDCs and if it had the lockword embedded in the UDC. He gave me a series of two commands that did the trick and they had some added value to boot. Once I saw the commands, I was impressed with the simple elegance, but like a good programmer, I had to deconstruct it, break it down and reassemble the whole thing. If you’d like to play along, you need: MPEX, VEAUDIT (both available from, who else, VEsoft) and a healthy programmer’s curiosity (you’re going to have to provide that yourself).

The details of the exercise show "GOD.PUB.VESOFT’ [is found using VEAUDIT] and we have accomplished our mission. But wait. What are those CIERR907 files? Those are files in the list that don’t exist! But they are UDCs that have been set! Looks like you can do some housecleaning and those UDCs can be un-set. How about that — you got some value added, you killed two birds with one stone, (insert your favorite cliche here). Time to play system manager again."

When you add a third party tool to your administrator's box, you can make a purge of such files foolproof. MPE/iX cannot select to show a complete set files by attributes such as program capability. Or for that matter, by last accessed time, or file size, or file security. It's a long list of things that MPE makes an administrator do on their own. Missing something might be the path to looking foolish.

VEAudit and MPEX will root out UDCs and do a foolproof purge, including file names. VEAudit will list all of the UDCs on a server, regardless of user -- not just the ones associated with the user who's logged in and looking for UDCs. The list VEAudit creates can be inverted so the filename is the first item on each line. Then MPEX will go to work to do a PURGE. Not MPE's, but a user-defined purge that looks for attributes, then warns you about which ones you want to delete, or would rather not.

Posted by Ron Seybold at 11:49 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 27, 2017

Friday Fine-Tune: Memory and disk behavior

By Jeff Kubler
Kubler Consulting

Hard-disk-headAlong with the relationship between your CPU measurements and overall performance, memory and disk make up the other two components of your HP 3000 performance picture. Main memory is the scratch pad for all the work that the CPU performs. Every item of data that the CPU needs to perform calculations on or updating to must be brought into main memory.

The CPU must manage memory. It must cycle through the memory pages, marking some as Overlay Candidates (this means that new data from disk may be placed here), noting that some are in continued use, and swapping others out to virtual or what is called transient storage. Swapping to disk occurs when data is in continued use but a higher priority process needs room for its data.

To accommodate this higher priority process and its need for memory space, the Memory Manager will swap the memory for the lower priority process out to disk. The more activity the Memory Manager performs, the more CPU it takes to do this. Therefore it is the percentage of CPU used to manage memory that we use as a measurement.

Page Faults per Second

A Page Fault occurs each time a memory object is not found in memory. The threshold for the number of Page Faults per second that can be incurred before a memory problem is indicated varies with the size and the power of the CPU. Larger machines can handle more Page Faults per second while a smaller box will encounter problems with far fewer. We have found that the number of Page Faults per second a system can endure without problem rises with the relative performance rating of the machine.

An exceptional number of Page Faults should never be used as the sole indicator of memory problems but when observed should be tested with the memory manager percentage. If both agree, you have a memory shortage. There are some strange things that I have observed with Page Faults, so it does not stand alone as an indicator of memory shortage.

The number of Page Faults per second and the amount of CPU needed to manage Memory are always evaluated in conjunction with each other. That is to say the high Page Fault Rate will not be considered a problem if the Memory Manager Percentage is not above 4 percent.

Disk Environment

The Disk Environment is referred to as Secondary Storage. This is where all the data needed for system use is stored. Since Main Memory is not large enough to store all of the data that will be needed by all the processes, there must be a location for this larger pool of data. Even though the Disk Environment does not have the significance it once had, this area can still be a bottleneck. As the CPU speeds increase, bottlenecks become more significant.

Several different factors can affect the Disk Environment. One of these is data locality. Data locality pertains to two different types. There is data locality within Image datasets and data locality across the disk itself.

Data locality across disk: This refers to the location of separate pieces of files (called extents). When files are placed on the disk, they can be placed in contiguous sectors or sections of files, or they can be placed in non-contiguous locations or even on many different disks. When files are not in contiguous locations they are said to be fragmented. The advantage of contiguous location is that greater efficiencies are allowed in retrieving data. When files need to be read, the head movement of the disk drive is minimal if files are in contiguous locations. The head moves to the location and the retrieval begins.

As the disk fills up the system cannot find one contiguous location to build any new file. Therefore, the system breaks the file up into extents and places the file wherever it can. A system reload will put files back into contiguous location (usually back on the location of the files file label) or products such as Lund Performance Solutions De-Frag/X can be used to put the files back into contiguous location.

Operating systems allocate disk space in chunks as they create and expand files and transient disk space (swap areas, etc.). When files are purged, these chunks are released for reuse. Over time the disc space may end up fragmented into many small pieces, which can slow the performance and the reliability of the system.

To observe and correct MPE fragmentation, you can use the De-Frag/X product from Lund Performance Software or use the Contigvol command of MPE/iX's Volutil program. The latter creates contiguous free disk space on a volume. Contigvol work about as well as VINIT CONDense did -- that is, it's stable and reliable, but requires multiple passes to get the best results.

Data locality within IMAGE datasets is the other area of major concern. There there are two different types of datasets to be concerned with, detail datasets and automatic or master sets.

The Detail Datasets

This type of set holds the day to day data input. Detail sets begin with nothing in them. When records are added 1 is added to something called the high-water-mark, a number that tells how many records have been in the set, and the record is placed in the set.

The problem is that IMAGE automatically reuses space that is given up when a record is deleted. This space is often called the delete chain. New records are placed in the most recent location available on the "delete chain." This means that new records are not in the same physical locality as the rest of the records and may be far removed from the other records.

The ideal state for a detail database is one where the detail entries are sorted by the key field. This allows the data to be retrieved in the smallest amount of IOs making efficient use of the MPE systems pre-fetching of data. When this is not the case we can measure the dataset lack of efficiency with something called the Elongation factor. This is simply a measure of how many more IOs the user must perform to retrieve desired data.

The Master Datasets

These have unique identifiers (field names). There are two types of master sets, a manual master and an automatic master set. Manual masters have user-entered master entries while automatic masters have automatic entries placed in them to accommodate access to detail records. The issue of importance to performance here is something called the hashing algorithm. This is the method used by the database to calculate the location of the next record placed in the database. The intent is to cause the master set to be as equally distributed as possible.

The hashing algorithm uses the size of the set in its calculation. A poor size or a size that is not large enough will result in an unequally distributed database. A poor size is most easily described as one that does not consist of a prime number. This means that when the hashing algorithm calculates a location there is a higher potential that a record will already exist in that location. When this happens a secondary position must be calculated. When secondaries are placed in another block within the database, another IO must occur to retrieve needed data. Since IO to disk is the slowest type of access, we want to avoid this at all costs.

Posted by Ron Seybold at 07:14 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 06, 2017

Friday Fine-Tune: Logging, IP logins, SNMP

Due to a disk crash, I had to reload my HP 3000 system recently. I’ve just discovered that system logging has been suspended. How do I resume system logging?

Paul Christidis replies:

The reason for the suspension of logging was most likely due to a duplicate log file name. When the SLT was created the then-current log number was recorded, and when you restarted the system from your most recent SLT it tried to open the sequentially next log file. Said file already existed.

  • MOVE the existing log files to a hold area
  • Determine what logfile the system resumed on
  • Perform a series of SWITCHLOG commands until the logfile number advances to one more than the highest number in the hold area
  • Then move the held logfiles back to the pub.sys group — replacing the ones created by the series of ‘switchlog’ commands.

Is there a way to see the IP address associated with a particular login?

Any user with SM can do the following, for example:

:SHOWVAR HPSTDIN_NETWORK_ADDR;JOB=#S495
HPSTDIN_NETWORK_ADDR = 172.16.0.30

The command :listf ci.pub.sys,8 will list all sessions and will show their associated IP address.

I’ve got an older model HP 3000 and I'd like to start monitoring it with SNMP for things like CPU utilized, jobs running or whatever other cool stat I can SNMP-grab. The problem I have is I can’t find the MIBs for it anywhere.

Andreas Schmidt replies:

First of all, I do not recommend the use of SNMP on the 3000, for performance but also security reasons. SNMP is not the securest protocol, as you know. Nevertheless, here are some hints:

• In the group NET.SYS you will find the SNMPUDC. This should be set in any case for MANAGER.SYS or on system level.
• Having set this, a SNMPCONTROL STATUS will show you the status of the SNMP subsystem.
• SNMPCONTROL START / STOP are self-explaining.
• The MIBs specific for MPE can be found in the document HP SNMP/XL User’s Guide

Posted by Ron Seybold at 06:53 PM in Hidden Value, Homesteading | Permalink | Comments (0)

December 16, 2016

Friday Fine-Tune: Login recovery strategies

We have changed passwords on MANAGER.SYS—and now we cannot locate the changed passwords, thanks to some staff reductions that made the new passwords unavailable. Any ideas on how to recover them?

John Stevens says

If you're logged on as OPERATOR.SYS, do LISTACCT on all the accounts that may have SM capability, then logon as the MGR/MANAGER of those accounts: TELESUP, and SUPPORT, and there are others; LISTACCT will find them. Login as those users (unless you don't have those passwords either) and LISTUSER MANAGER.SYS. Vesoft's MPEX might help ease some of this as well.

Duane Percox of QSS adds a simpler approach:

If you can log onto operator.sys:

file xt=mytape;dev=disc
file syslist=$stdlist
store command.pub;*xt;directory;show

Using your favorite editor or other utility search for the string: "ALTUSER MANAGER SYS"
You will notice: PAS=<the pwd> which is your clue.

Steve Ritenour suggests that a logon to the TELSUP account will unlock the passwords.

Some 3000 managers believe the subject itself should be filed in a place not easily found. "These responses are all well and good," said Bruce Collins of Softvoyage, "but shouldn't we be thinking twice about posting this kind of information (i.e. how to hack an HP 3000) to the 3000 newsgroup?"

Bill Lancaster disagreed. Secrecy about password recovery is not really a secret, he said.

A Google search with the right words will yield far more dangerous information about the 3000 that anything in this thread. The genesis of this information being on public networks came through BBS’s in the 1980s. I’m afraid the barn door is already open.

Can the password for MANAGER.SYS be reset?

Gilles Schipper says

Not easily. If you can log on as operator.sys, you should be able to store off the system directory to tape, as follows:

:file t;dev=tape
:store command.pub;*t;directory

Now that you have the directory on tape, you should be able to look around with fcopy (and the ;char and ;hex options) to find passwords for manager.sys.

John Stevens says

If you're logged on as OPERATOR.SYS, do LISTACCT on all the accounts that may have SM capability, then logon as the MGR/MANAGER of those accounts: TELESUP, SUPPORT, (I can't think of others right now, but LISTACCT will find them). Login as those users (unless you don't have those passwords either), and LISTUSER MANAGER.SYS MPEX might help ease some of this as well.

Posted by Ron Seybold at 08:40 PM in Hidden Value, Homesteading | Permalink | Comments (0)

December 09, 2016

Friday Fine-Tune: Driving Filesystem Checks

Space_odysseyIn the middle of a full backup, the HP 3000 at James Byrne's shop came to a system halt at 3 AM. It was the kind of halt that puts up those puzzling abort messages not even HP has fully documented. For example, about SA 1458, Robelle's Neil Armstrong said, "My experience with SA 1458 is that it is a catch-all abort. You need to look at the subsystem information and the only way to truly know the root cause would be to get a dump and analyze it." He referenced a webpage that breaks down the process of doing 3000 system failure analysis, too.

When a halt occurs during a backup, there's always the chance the 3000's filesystem has been injured. "I'd say run FSCHECK.MPEXL.TELESUP and check your filesystem," said Keven Miller of 3k Ranger. He added that a former HP support expert, Lars Appel, "instructed me that System Abort messages are in subsystem 98. From the MPE Error Messages Volume 2, Chapter 4, System Aborts, 1458 MESSAGE means A critical process is being terminated due to a trap."

Sure enough, power interruptions at Byrne's shop introduced damage to an Image database. 

We reached the point last summer were we were toying with going off-grid simply to avoid the repeated power interruptions. If this sort of thing is causing damage then we will have to consider it. And it seems to; we now have a broken backward chain in one of our Image databases. A thing that I cannot ever recall. Coincidence? We are doing a backup, and then I will be using Adager to go in and take a look.

FSCHECK is an included tool on the 3000. It's simply there to validate extents and scan the table cache for missing files. Better tools include not only the legendary Adager, but independent support suppliers for the 3000 owner. People who know the fast commands of tools like CSTM. "What does your support provider say?" asked one support vendor. Self-support can be backed up by 3000-L questions. Some of the advice about the halt even ran to looking at memory issues. A provider can help eliminate these possibilities.

Capricious power service from his utility — a government service that's been privatized — has extracted a price on Byrne's Series 918. 

I am simply trying to find out if there is any way of examining whether or not we actually have a failing drive. We have spares but if there really is no need then I would rather not take the system down again after such a short interval. It has been a bad fall for our poor old 918. The system HDD was toasted by a whipsaw set of power outages on October 11; now our data disc is suspected of being ready to let go as well.

While running FSCHECK, Byrne was advised to use the commands 

Check all Dev=all
Syncaccounting
FSCOUNT / 10000

He also received instruction from Mark Ranft on how to scan logs using CSTM to find disk errors.

Sign on as Manager.SYS. Do a LISTF LOG####,2 to find the start and ending Log files. Alter the log file number range in the commands and enter the commands in LOGTOOL.

list log=3404/3477 type=111 "device class"="hard disc",da,ca,"bus converter" out=LogOut1

list log=3404/3477 TYPE=111;'MGR CODE'= 241,242,900,901,951 out=LogOut2

The results are written to two files LOGOUT1 and LOGOUT2.  I had this set up to run weekly on my systems.  And if the files had errors in them, the job would email the results to me for review. You will see errors due to SCSI or FC resets on every boot, so check the timestamps to tell if the boot caused the error.

At Byrne's shop the Series 918 was recovered after Adager did its repairs. "During the Adager repair an infrequently occurring high-pitched but low volume sound was noted," he said. "This appeared to emanate from the 3000. We have not heard it since the Adager repair completed. A suspicion arose that we might have a disk about to lose a bearing."

Once the recovery was complete and the disk replaced, the trademark wisecracks of long-time 3000 vets began to arise. The sound during Adager's repairs "would have been the sound of the chains being dragged around the disk to put them back straight," said Alan Yeo, "and possibly the sound of a very tiny virtual Alfredo welding a few broken links back together."

Byrne noted that the problems with the Series 918 disks "started I was working on installing the 9.6 version of PostgreSQL on a new FreeBSD host. I wonder if the HP 3000 is throwing a temper tantrum? Naaah. Cannot be."

Posted by Ron Seybold at 09:20 PM in Hidden Value, Homesteading | Permalink | Comments (0)

December 02, 2016

Friday Fine-Tune: Print classes, STORE, FTP

I need to add classes to a printer on my HP 3000. I keep getting a message that the LDEV doesn't exist from SYSGEN's I/O section when I issue the ACLASS command. I can see the device exists, so what's the problem?

Letterpress-2Devices on your system that are configured through NMMGR must have their classes changed through NMMGR, which is probably why you're getting the error message. Run NMMGR and go to the PROFILE for the device. You should be able to add classes to the device at that point. After making changes in NMMGR, remember to VALIDATE these changes to NMCONFIG, and then cross-validate using the RDCC command from SYSGEN's SYSFILE section.

NMMGR changes take effect after the system has been reSTARTed and the power has been cycled on the DTCs. You should cut a System Load Tape (SLT) making these changes so you'll have them on tape for your next UPDATE.

If your device was configured through SYSGEN, the device should have its classes modified through SYSGEN. You're right, you do use the ACLASS command in the IO section. Changes to the IO section require a START to make them take effect.

In HP's MPE/iX manuals I am are directed to backup the system as follows:

STORE @.@.SYS,@.@.@-@.@.SYS;*T;DIRECTORY;SHOW

What is the practical benefit over

STORE @.@.@;*T;DIRECTORY;SHOW 

People trot out the reason from the distant past: if you have to do a re-install it is good to have PUB.SYS (and maybe the TELESUP account) on the front of the tape so you can restore quicker.

Gilles Schipper woke us up with his response. “The reason one would perform the former over the latter is to ensure that all of the files residing in the SYS account be placed at the beginning of the backup. Whether that is a good thing is, in my opinion, dubious."

Gilles said, “Perhaps, in the days of half-inch, 1600-BPI magnetic tapes, it could provide for a quicker restore of files in the SYS account than, say, files in the YAHOO account. In any case, with tape technology, where retrieval of any group of files from any spot on DDS or DLT tape is quick, the raison-d’être for such file placement strategy disappears - assuming there ever really was a good reason.”

If you have user volume sets you MUST explicitly list them after the DIRECTORY keyword or else the directories for the user volume sets will not be stored.

Is there a way to use FTP to move an IMAGE database between two HP 3000s?

FTP by itself won't work, as it doesn't support user-labels, which are included with the IMAGE root file. DSCOPY, while not being as trendy as FTP, does know about user labels. The Web has freeware LZW compressor programs which know how to archive IMAGE datasets, so you can use FTP without having to change the file codes to non-priv values beforehand. Just archive the sets into one non-priv package, FTP it as a binary file, and then use LZW to re-expand the data-sets back to their original form. LZW programs, along with many other open source gems, are available at SourceForge

Posted by Ron Seybold at 04:22 PM in Hidden Value | Permalink | Comments (0)

November 25, 2016

Friday Fine-tune: Adding disks and IP blocks

Is it possible to add a disk drive "on the fly" without doing a reload?

Jeff Kell replied:

You generally have to shut the system down to install and cable the disk to avoid electrical/interface problems. The usual approach is to use SYSGEN to configure the new device on the path where it will reside, keep the new configuration, shutdown the system, install the disk and do a START NORECOVERY.

Once the disk is recognized by the system, you can add it to your running configuration as follows (assuming the new drive will be LDEV 5 in the system volume set):

:volutil
> newvol mpexl_system_volume_set:member5 5 90 90 (DISC,SPOOL)

[For details, see "Volume Management", HP Part No. 32650-90045 or "Performing System Management Tasks" HP Part No. 32650-90004.]

This will add the volume to the system volume set, but it also has some side effects. Since the new volume is "empty" and the disk space allocation routines attempt to "balance" loads across drives, all of your new files and transient space will be allocated on the new drive until it's capacity approaches that of the other volumes. This will create an I/O bottleneck on that drive, at least initially.

You could selectively :RESTORE certain accounts (or the whole system) to try and balance the allocation. You could also perform an INSTALL and a :RESTORE for better efficiency, but at the cost of a great deal of time. There are also certain third-party utilities that will balance disk utilization across members of a volume set. These utilities work online on a running system and don't require any downtime.

The network configuration of our HP 3000 was originally set up with one block of IP addresses. Now I need to add another block of addresses. Where do I add these in NMMGR?

You can add an IP address using NMMGR the following way:

  1. After typing NMMGR, select "Open Directory" .
  2. Then select "Update Dir."
  3. Now select the "Add" option (F5.)
  4. You are placed in a screen where you can enter the IP Address of the machine. The type is generally set to 1( IP).
  5. Now press the "Save Data" (F6) option, back out of NMMGR, and you are done.

Posted by Ron Seybold at 09:00 AM in Hidden Value, Homesteading | Permalink | Comments (0)

November 23, 2016

Mailing news from the HP 3000: an old skill

Blue mailboxInternal mail hosts remain a crucial tool in datacenters, even some running MPE/iX. "You still host your own email?" is not a question you'd only pose to a crazy manager. An organization's security standards can be so high that no outside mail server will be trusted. In the earliest days of email, 3k Associates built and sold a beautiful native MPE mailing system, Netmail/3000. It's a smart mailserver, meaning it doesn't require that an organization's e-mail be piped through an Internet provider's mail server for final delivery. Then in the late 1990s, HP's lab started the long process of porting sendmail to MPE/iX.

Now some 3000 sites are looking at how to replace their 3000-based mailing software as they migrate. One of them contacted us this week to ask about an alternative to sendmail. Linux is their migration target, after a history using the 3000 that goes back to the days of HP Deskmanager. Tim O'Neill shared a story while asking about an alternative to sendmail.

I saw that FreeBSD Unix has its version of sendmail. Seeing reference to FreeBSD made me recall a story about FreeBSD running on an old HP 3000, maybe a Series 70 or an early Spectrum system. I think I have read that FreeBSD is at some sites still running in production mode, as MPE and MPE/iX are. It also made me wonder what the installed base of FreeBSD might be — and how that compares to the installed base of MPE and MPE/iX on old hardware and on Charon hardware.

FreeBSD, like MPE/iX, has some surprisingly large companies using it. You might have heard about one of them called Netflix. Of course the Charon HPA emulator from Stromasys makes every remaining product and archival 3000 a candidate for the kind of longevity we see in FreeBSD.

Sendmail has a colorful history. The Unix Hater's Handbook devoted a full chapter to the software's vulnerabilities; sendmail comes from the Unix heritage, after all. By 2003, HP was still patching sendmail to shut down security breaches, although the breaching wasn't nearly as serious on MPE/iX as on Unix variants including Linux. Sendmail's open source capabilities are now under the banner of ProofPoint, the company that purchased the sendmail resources in 2013.

Sendmail's worldwide release was last updated in 2014. HP announced it was testing sendmail to place in the Fundamental Operating System in November, 2001—a month that's famous in the 3000's history for other reasons. But the software moved along to an 8.13.1 release in FOS. It's only one major release behind the worldwide open source version, now advanced to an 8.14 release. Sendmail also includes encryption.

Sendmail has included encryption facilities since 8.11. That's where security capabilities descend onto the requirements. Encrypting mail is a common feature in commercial hosting solutions. Sendmail/iX sends mail created by and triggered from HP 3000 applications, given enough technical know-how.

There's a robust webpage about the 3000 mail solution that was started by Mark Bixby. He's the engineer responsible for lighting the fire of open source flames at HP. Keven Miller of 3K Ranger has updated and maintained the page and its knowledge about Sendmail/iX. The software itself is in your 3000's SENDMAIL account in a version-specific group named vuuff.

Posted by Ron Seybold at 06:22 PM in Hidden Value, Homesteading, Migration | Permalink | Comments (0)

November 18, 2016

Friday Fine-Tune: Tricks with command files

I'm working on a command file on my HP 3000. Is there any way to have it copy part of itself into a separate (temporary) file?

Jeff Vance replies:

MPE does not support the Unix concept of ‘here’ files, where input data for the command can reside in the same file as the command, except in the case of jobs. But even in a job, you may not include inline data for a script or UDC invoked by that job.

The SPOOKHELP script may be of some use. This single script contains the help text for all of the SPOOK commands plus the code to search for and display that text once HELP xyzzy is entered.

How can we execute a command after a user enters the :bye command in MPE?

Olav Kappert replies:

It is possible to execute many commands after the bye has been entered.  Simply create a UDC (maybe a cmd file) called bye.

The contents of the UDC for the command bye is up to you.  This would be useful if you want to do statistics before the session terminates.

John Pittman adds:

Don’t let them do a bye. We don’t allow any users access to OP system prompt at all. They get a logon no break UDC that runs a menu, and when they end the menu, they get logged off.

Inside that UDC at exit time, we build a string giving user, connection point (LDEV or IP of their PC) connect time, CPU date etc and append it to a log file. Then we know when anybody last used the system, how many users are using different connections, or when different user names are using the same connection point.

Posted by Ron Seybold at 11:10 PM in Hidden Value, Homesteading | Permalink | Comments (0)

November 11, 2016

Friday FineTune: Internal disk plus VA array

Human_pyramidFiberChannel Storage Area Networks and shared tape libraries became popular in the years after HP stopped making its MPE/iX hardware. The HP 3000 supports SAN from the XP series of RAID devices to the VA7100 disk arrays. But how much should you rely on a RAID or SAN device? Internal storage devices might seem to be yesterday's tools, but the modest drive inside an HP 3000 can still be very useful — even if a company has invested in the FiberChannel storage solution of the VA7100.

Moving to the VA solution has great benefits, as reported in a story about using a VA7100 array with the 3000s. But booting directly from a VA array — well, you'll need an N-Class server (native FiberChannel installed) or a very expensive HP A5814A-003 Fiber/SCSI router (if you can find such a thing) to employ VA in the 9x9s.

The Crossroads SA-40 Fiber/SCSI switch will link a VA array to 9xx 3000s. It just won't let you boot your MPE/iX system from any of its drives. Craig Lalley of EchoTech recommends the affordable Mod 20 arrays for boot capability.

Internal drives remain as important as the VA arrays for a Series 9xx HP 3000, or even to the XP line of HP arrays. Even important enough to even duplicate them.

A second bootable disk inside your 3000 can take some forethought, but can be essential to smooth recovery of an LDEV 1 failure. James Killam of HP once reported to the 3000 mailing list, "Keeping a bootable image of MPE on one of the internal drives... saved me once at 2 AM when we lost total connectivity to the XP array the system was attached to and we had some serious troubleshooting to do."

Donna Hofmeister, former OpenMPE board member, has noted:

I did one internal disc and it was expressly for memory dumps. A second internal disc with a bootable image would be wonderful insurance. It would take some planning to be able to manage it all, but there’s no reason why you can’t have two bootable discs. I’ll point out the obvious: if LDEV 1 is internal, and you have a multi-disc system volume, and the remaining system storage is on a disc array — uh, what’s the point? If LDEV 1 fails, you’re toast.

Hofmeister said she's had a drive fail on a VA array. "That array worked perfectly and switched over to the spare without a blink. Given that I had two systems sharing this array, I was more than pleased with how well it worked."

Drive failures are among the most likely of hardware problems using HP's 3000 devices. A second internal drive in a system can make a big difference in recovery time. The other way around drive failures is to make your way onto emulated PA-RISC systems from Stromasys.

Posted by Ron Seybold at 08:47 PM in Hidden Value, Homesteading | Permalink | Comments (0)

October 17, 2016

Rebuilding Systems Faster and Better

I'm looking at how to save as much time as possible in rebuilding an HP 3000's software and directories. My options seem to be using STORE, versus the sysgen tape command "tape store=@.@.@". What's the best way to go here?

Donna Hofmeister of Allegro replies

Construction-loaderUnless your system is small (like a 918 with 8-12GB of disc), you don't want to try to do a full backup via sysgen. If you really do a full backup then I prefer this syntax “store /;...” as it is self-documenting and you know that the Posix files will be backed up as well. (On older releases of MPE, @.@.@ did not back up Posix files <eek>)

You want to make sure that you run 'buldacct' periodically (and routinely). You also want to make sure that you are somehow backing up your directory (store /;*t;directory, for example). Between the two, you have belts and suspenders (for recovering your accounting structure).

On older releases of MPE, you want to make sure that the network is shut down prior to making your SLT tape. And it's still a good idea to have the system quiesced when making an SLT, since everything in the sys account (and .pub.sys in particular) will be locked while the tape is being made. Nothing quite like grumpy users to make your day.

Just as a matter of preference, you should normally do a BULDACCT @ at the beginning of the weekly full backup, then the DIRECTORY option along with ;ONVS=MPEXL_SYSTEM_VOLUME_SET, PRODUCTION_SET, etc as a belt and suspenders approach for that day we all hope never comes.

Mark Ranft of Pro 3K outlined the use of sysgen

Here are instructions for a complete backup of basic MPE/iX system via SYSGEN's TAPE option.  It is best if everything fits on a single DDS (or DDS-2 or DDS-3 or DDS-4) tape cartridge, but it will ask for a second (additional) tape(s) as needed.

Note:  I included a 'second_volume_set' which can be changed or removed

Note 2:  The line below is >80 characters, so you have to know how to create a file so that this does not wrap (or make other adjustments.) 

Step One - Create an indirect file containing the following as a single line... 

@.@.@;onvs=mpexl_system_volume_set, second_volume_set

;progress;maxtapebuf;compress=high;online=start;partialdb;directory

Step Two - Create this job

:job jsltall,manager.sys;outclass=,1          
:sysgen                                      
tape store=^fsltall.job.sys                  
exit                                          
:tellop END OF JSLTALL --------------  EOJ    
:eoj                                          

Feel free to add a BULDACCT to this.

Posted by Ron Seybold at 09:42 PM in Hidden Value, Homesteading | Permalink | Comments (0)

October 07, 2016

How to Make a Windows to 3000 FTP Move

Open-windowI need to move a file to our 3000 from a Windows server with FTP and Windows doing the put. The Windows file has longish variable length records, but I would like them converted to fixed length on the 3000. When I tried, this was the result:

PUT C:\Dev\MViewFTP\transdata\AP_HEADER_GW.CSV LMAPGW.IVD;rec=-1024,1,f,ascii,disc=1000;move

200 PORT command ok.

550 The FILE EQUATION STRING option (item# 52) is not in a valid file equation form. (FILE OPEN ERROR -449)

Keven Miller replies

Item #52 refers to HPFOPEN. From the intrinsic manual

52 File equation string:

Passes a character string that matches the file equation specification syntax exactly. (Refer to the FILE command in the MPE/iX Command Reference Manual.) This option allows the specification of options available in the FILE command.

I don't like the trailing ";move" in your command string.  I'd remove that. Also, you have comma after "ascii" and it should be a semicolon, like this

;rec=-1024,1,f,ascii;disc=1000

Tony Summers adds

I'm pretty sure ";MOVE"  is only valid if you're using the HP 3000 FTP client. Have a look at page 20 of the link below at TeamNA Consulting, another place where the MPE and HP 3000 manuals live

FTP/iX Manual, circa 2000

If you need to move the file during the FTP transfer,  maybe you need to use "RENAME".  

Posted by Ron Seybold at 06:36 PM in Hidden Value, Web Resources | Permalink | Comments (0)

September 26, 2016

3000-L connects again after a silence

Tin-can-telephoneAs if on cue after our report about its silence, the 3000-L mailing list and newsgroup sprang back to life over the weekend. One problem solved by the 415 readers was how to identify if a store to disk backup is a LZW compressed backup file. A Tracy Johnson report also confirmed that a LISTF,2 can report the time of each LISTF, by writing a specialized job.

Meanwhile, a 15-year-old HP 3000 with network connection troubles got advice from the newsgroup's readers. A Series 969 running MPE/iX 6.0 would not be the first thing you'd choose for interfacing to an internal website. But when a 3000 has data that a user needs over the Web, the server is the place to go.

Trouble started to surface when clients access a webpage which then opens a telnet session with the 3000, grabs the info, and then returns the data to the webpage.

We’ve been getting more and more errors over the last year, culminating in non-stop Could not initialize data in path with TCP, which then blocked anyone accessing us through our webserver. We’ve tried many changes but cannot seem to get past this.

When it locks up, the HP 3000 keeps running but won’t accept any new sessions. Which means our clients can’t run searches.  Which is very bad for us. Sometimes we can stopnet and startnet and it will work for a while, but then the errors start again. Eventually, we have to coolstart to be able to have clients log in.

Mark Ranft suggested "If they are already running in the CS queue, here is the likely cause.  Is there some new monitoring in place? If so, it may not be behaving well on the network.

What happens is someone uses a telnet or ftp connection to monitor whether the network is up on the 3000.  They send the SYN, the hp3000 answers with the SYN ACK, and then the 3000 receives a reset before the connection is complete.  This handshake sequence causes this exact error.

Also inetd and other HP3000 networking improved greatly in later versions of MPE/iX. If they must stay on 6.0, they should at least be sure to install the latest patches.

A third party support company served this troubled user until the support vendor folded, "and the only options we found weren't affordable." Getting the 3000 back up will trigger a revisit of that situation. Any server with critical customer data on it—and doesn't have a support vendor—relies on the largesse of the 3000 volunteers of 3000-L. That mailing list did go without a new message for more than a month, a troublesome response time for a critical server.

Posted by Ron Seybold at 07:44 PM in Hidden Value, Homesteading | Permalink | Comments (0)

September 14, 2016

Dancing the Samba services tune, MPE/iX-style

Samba DancersTen years ago this week we were promoting instructions on how to use Samba better on HP 3000s. Samba is "a group of programs that allows a Unix host to act as a fileserver for Windows platforms," according the MPE/iX documentation rolled out in 1999. The file-sharing and printer sharing software which has been a part of MPE/iX since the 6.0 release "allows Unix-like machines to be integrated into a Windows network without installing any additional software on the Windows machines. Many different platforms run Samba successfully; and there are nearly 40 different operating systems which support Samba." And many more now, a decade later.

HP brought features of Samba to the 3000 in a port called Samba/iX. "It is a solution for those wishing to access HP 3000 disk storage and printers (both networked and spooled from MPE/iX) from common PC client operating systems like Windows." Samba/iX allows access to disk and printer resources of MPE/iX by providing standard SMB file and printer services that are accessible from PC clients and their applications. An administration tool called SWAT makes Samba so much easier to use.

Samba 3.0.22 is distributed by the following MPE/iX base patches. Your independent support provider should be able to help you round one of these up. They've got the latest functionality.

  • SMBMXY6D (BT) for MPE/iX 6.5
  • SMBMXY6E (BT) for MPE/iX 7.0
  • SMBMXY6F (BT) for MPE/iX 7.5

The (BT) stands for Beta Test. HP never cut the 3.0.22 version loose as a general release (GR) version. For reference, the following are GR versions with less functionality.

  • SMBMXG3A (GR) for MPE/iX 6.5
  • SMBMXG3B (GR) for MPE/iX 7.0
  • SMBMXG3C (GR) for MPE/iX 7.5

Even a total 3000 network newbie can get Samba up and running. Samba must be running before you can run SWAT. Here's some useful info when getting SWAT going.

In SERVICES.NET you'll want a line that reads:
swat   901/tcp   # Samba/iX Web Admin Tool

In INETDCNF.NET you'll want:
swat stream tcp nowait MANAGER.SYS /usr/local/samba/SWAT swat
(adjust the path to your SWAT NMPRG)

 If you’re running an older version of Samba, you’ll need to modify ‘/usr/local....’ to point to where SWAT actually lives (and case is important).  The user needs to match the user in your samba daemon jobs. (For many, it’s MANAGER.SYS, for you it may be MGR.SAMBA) When you're connected to your MPE/Samba server through a browser to access SWAT, you'll be asked for a logon and password. This is a good thing.

After changing your services and inetdcnf files, all that you should have to do is give inetd a swift kick (e,g,  :inetd.net -c  ) Check inetd’s $stdlist after doing that and you should see that it brought in the new configuration.

In your browser point to http://xxx.xxx.xxx.xxx:901/ (where xxx.xxx.xxx.xxx is the IP of your 3000). Or you can use the name of your 3000 too.

When updating Samba and Apache config files, some are picky about how their records are terminated. Robelle's Qedit can make the needed adjustments. Be sure to know what version of MPE/iX you have installed, including patches.

Posted by Ron Seybold at 08:55 PM in Hidden Value, Homesteading | Permalink | Comments (0)

August 12, 2016

How to purge UDCs on the HP 3000 safely

Cheshire_catThe software vendors most likely to sell products for a flat rate -- with no license upgrade fees -- have been the system utility and administration providers. Products such as VEsoft's MPEX, Robelle's Suprtool, Adager's product of the same name -- came in one, or perhaps two versions, at most. The software was sold as the start of a relationship, and so the relationship focused on the understanding the product provided for people responsible for HP 3000s.

That kind of understanding might reveal a Lewis Carroll Cheshire Cat's smile inside many an HP 3000. The smile is possible if the 3000 uses UDC files and the manager uses only MPE to do a file PURGE. Of course, PURGE ships on all MPE systems. Using that means you'll have to rebuild the UDC catalog. But even that's not enough.

Stan Sieler of Allegro shared a story about this recently. "We recently encountered a site where—somehow—an HFS filename had gotten into COMMAND.PUB.SYS. You can't delete UDC entries with HFS filenames, nor can you add them. I had to edit the file with Debug to change the name into something delete-able." Then there's the rebuilding of the catalog. Keven Miller has contributed a program that sorts and reorganizes UDC files.

There is a more complete way to remove such things from a 3000's storage. You're careful about this because eliminating UDCs with only MPE might leave a user unable to use the server. That grin that lingers is the UDC's filename. 

User Defined Commands are a powerful timesaver for 3000 users, but they have administrative overhead that can become foolproof using the right tools. These UDCs need to be maintained, and as users drop off and come on the 3000, their UDCs come and go. There's always a chance that a UDC file could be deleted, but that file's name could remain in the filesystem's UDC master catalog. When that happens, any other UDCs associated with the user will fail, too. It might include some crucial commands; you can put a wide range of operations into a UDC.

When you add a third party tool to your administrator's box, you can make a purge of such files foolproof. You can erase the Cheshire Cat's grin as well as the cat. It's important because that grin of a filename, noted above, can keep valid users from getting work done on the server with UDCs. This is not the reputation anybody expects from a 3000.

First you have to find all of your UDCs on a system, and MPE doesn't make that as straightforward as you might think. Using SHOWCATALOG is the standard, included tool for this. But it has its limitations. It can display the system-level UDC files of all users in all accounts. But that's not all the UDCs on a 3000.

MPE, after all, cannot select to show a complete set files by attributes such as program capability. Or for that matter, by last accessed time, or file size, or file security. It's a long list of things that MPE makes an administrator do on their own. Missing something might be the path to looking foolish.

Employing a couple of third party tools from VEsoft, VEAudit and MPEX, lets you root out UDCs and do a foolproof purge, including file names. VEAudit will list all of the UDCs on a server, regardless of user -- not just the ones associated with the user who's logged in and looking for UDCs. The list VEAudit creates can be inverted so the filename is the first item on each line. Then MPEX will go to work to do a PURGE. Not MPE's, but a user-defined purge that looks for attributes, then warns you about which ones you want to delete, or would rather not.

By using MPEX -- the X stands for extended functionality -- you can groom your own PURGE command to look out for files that have been recently used, not just recently created. MPE doesn't check if a purged file is a UDC file. 

Such 3000 utilities provided the server and its managers with abilities that went far beyond what HP had built into MPE and its IMAGE database. Now that MPE is moving on, beyond HP's hardware, knowing these third party tools will transfer without extra upgrade fees is like ensuring that a foolproof MPE will be running on any virtualized HP 3000.

They're an extra-cost item, but how much they're worth depends on a manager's desire to maintain a good reputation.

In the earliest days of the sale of these tools, vendors were known for selling them for the price of the support contract alone. That's usually about 20 percent annually of the purchase price. If a $4,000 package got sold that way, the vendor billed for just $800 at first. It made the purchases easier to pass through a budget, since support at the manager-tool level was an easier sell. Think about it. Such third parties passed up $3,200 per sale in revenues in the earliest days. They also established relationships that were ongoing and growing. They were selling understanding of MPE, not just software.

This kind of practice would be useful for the community's remaining software vendors. This is not the time to be raising prices to sustain MPE computing, simply because there's a way to extend the life of the hardware that runs MPE. As the number of MPE experts declines, vendors will be expected to fill in the gaps in understanding. Those who can do this via support fees stand the best chance of moving into the virtualized future of 3000 computing.

Posted by Ron Seybold at 01:17 PM in Hidden Value, Homesteading | Permalink | Comments (0)

August 10, 2016

Measure 3000 performance for datacenters

Hp-1715a-oscilloscopeMeasuring the performance of an HP 3000 used to be a leverage point for increasing investments. By now the numbers help justify continuing to use the server in a datacenter with newer boxes. "We think of our HP 3000s as stable, and even reducing in usage over time," says one systems manager, "though actually as the company grows, the data requirements and load on the 3000s increases."

One way to measure a 3000's footprint is the amount of memory it requires. Memory upgrades cost nothing like what they did even 15 years ago. But any spending at all makes that 15-year-old server suspect. HP's Steve Macsisak recommended sessions x 4, plus jobs x 16, plus 64 MB as the criteria for memory usage.

An HP 3000 uses as much of its memory as possible to make processing efficient. The design of the PA-RISC architecture makes memory the most important element of performance, after IO speed. It's not that unusual to see a 3000 using 100 percent of its memory, according to field reports. There's also CPU usage to measure. 

CPU percentages can come via the REPORT command. Count up the CPU seconds used in the week, and divide by the total number of seconds available (604,800). But for all of this, it doesn't feel like a graphic report the rest of the datacenter gets from its Unix and Linux systems using SAR. There may be a program inside a 3000 that can help, even if the company never purchased performance tools from Lund. HP's Glance gives away its reporting power in its name, one manager has joked.

PloticusThere's freeware available to create handsome graphs like the one at left, suitable for showing in a meeting about datacenter resources. Ploticus/iX was written by Andreas Schmidt. It uses data from SCOPE.SYS. Ploticus even works with SAR's data.

Since there's no port of SAR for MPE/iX, something else must stand in. Some systems have HP Scope, the software that dives in deep enough to produce report-ready numbers. It's not the smooth path a 3000 gets from Lund's Clearview Performance Manager. Scope is the HP Performance Collection Software sold by the vendor while it still had an active 3000 business.

Scope includes a program, EXTRACT.SCOPE.SYS, that permits the software to EXPORT its results to a text file. The manual for the software says it has three components.

SCOPEXL, if you are using an MPE/iX system), UTILITY, and EXTRACT. SCOPEXL is the performance data collector for MPE Systems. It continuously collects and summarizes performance data. UTILITY and EXTRACT are the host programs that let you interact with SCOPEXL and manage the data that it collects.

HP's Scope documentation describes how to use the collection and management software: how the host components interact; detailed command descriptions for each program; and suggestions on how to use the programs to analyze and archive data efficiently.

Posted by Ron Seybold at 08:45 PM in Hidden Value, Homesteading | Permalink | Comments (0)

July 11, 2016

How to Use Input to Create Output Files

Intrinsics are a wonderful thing to power HP 3000 development and enhancement. There was a time when file information was hard to procure on a 3000. It was a long time ago, as I was reminded by Olav Kappert in a call about his HP 3000 history. "The high point in MPE software was the JOBINFO intrinsic," he said. Kappert started with the 3000 in 1979.

Fast-forward 37 years later and you'll find questions from a different programmer still working on a 3000, adding features to a system. The Obtaining File Information section of a KSAM manual on MPE/iX holds an answer to what seems like an advanced problem.

I'm still using our old HP 3000, and I have access to the HP COBOL compiler. We haven't migrated and aren't intending to. My problem is how to use the characteristics of an input file as HPFOPEN parameters to create an output file. I want that output file to be essentially an exact replica of the input file (give or take some of the data). I want to do this without knowing anything about the input file until it is opened by the COBOL program. 

I'm using FFILEINFO and FLABELINFO to capture the characteristics of the input file, after I have opened it. After I get the opens/reads/writes working, I want to be able to alter the capacity of the output file.

Francois Desrochers replies

How about calling FFILEINFO on the input file to retrieve all the attributes you may need? Then apply them to the output file HPFOPEN call.

Donna Hofmeister adds 

You might want to get a copy of the "Using KSAM XL and KSAM 64" manual. Chapters 3 and 4 seem to cover the areas you have questions about. Listfile,5 seems to be a rightly nifty thing.

But rather than beat yourself silly trying to get devise a pure COBOL solution, you might be well advised to augment what you're doing with some CI scripts that you call from your program.

In a lively tech discussion on the 3000-L list, Olav Kappert added, 

Since you want to do this without knowing anything about the input file until it is opened by the COBOL program, the only way is to use one of the MPE intrinsics to determine all the characteristics of the file in question. Then do a command build after parsing that information.

Michael Anderson added details on how the 3000's CI scripting can build upon the fundamentals of file information and COBOL.

I like Donna's plan.This is a strategy that will also help whenever you want similar functionality on a NON-MPE platform. Also, although COBOL is very capable, an external script might be a better tool. You don't always need a hammer.

This is hypothetical, to try to make a point. From your MPE CI prompt, type HELP FINFO. You should be able to set some variables (SETVAR FILEA "XXX"), and using FINFO add some more variables. Then from COBOL using HPCIGETVAR, string together a BUILD command (with a bigger LIMIT maybe), and call "HPCICOMMAND". You could string the build command from a command, into a single variable, then COBOL only needs to HPCIGETVAR once.

You can also write a script to do everything you want, and call HPCICOMMAND to run the script, pass it parms. It's pretty cool, and it makes your COBOL application more portable. (Same program, different script).

For example: On MPE I once wrote (using COBOL) a small utility to CALL DBINFO, extract all the meta-data from any IMAGE database, and then create, and write to the NEW KSAM COPYLIB, ending up with all the COBOL copylib modules needed for all datasets for any database, including call statements and working storage. My point to all this: I used CI scripting to create and write to the copylib. I actually used ECHO to write the copylib ksam file from a CI script. Now, seeing how I work more on HP-UX and Linux, plus OpenCOBOL and Eloquence, I should be able to compile this same program on Linux with minimal modifications, only changing the external script.

I use this method to access SQL databases, and much more, using OpenCOBOL and the Tcl/Tk developer exchange. This way I can run the same program, same script almost anywhere, no matter, Windows, Mac, or Unix.

Eric Sand, another veteran of the 3000, commented that this kind of challenge really shows off the range of possibility for solving development problems. "You can create almost any cause and effect in MPE that you can imagine," he said. "Reading about your concern gave me a little rush, as I mentally organized what I wanted to do to address your concern."

Posted by Ron Seybold at 09:12 AM in Hidden Value | Permalink | Comments (0)

June 29, 2016

Date format variable help for MPE/iX

What would be the easiest way to get a variable date in the format "06/29/16" on a MPE/iX 7.0 and 7.5?

Michael Anderson replies

First echo:

echo ![str("!hpyyyymmdd",5,2)]/![str("!hpyyyymmdd",7,2)]/![str("!hpyyyymmdd",3,2)]

Then setvar:

setvar mydate '![str("!hpyyyymmdd",5,2)]/![str("!hpyyyymmdd",7,2)]/![str("!hpyyyymmdd",3,2)]'

(Note the usage of single-quote and double-quote in the setvar command.)

Barry Lake adds

Please note that the HPYYYYMMDD variable is already a string variable:

Frodo: calc typeof(HPYYYYMMDD)
2, $2, %2

So you don't have to dereference it with ! inside double quotes. In other words, the following works just as well, is easier to read, and might even execute a bit faster:

Frodo: echo ![str(hpyyyymmdd,5,2)]/![str(hpyyyymmdd,7,2)]/![str(hpyyyymmdd,3,2)]
06/29/16

Also note that since you're asking for the value of HPYYYYMMDD three different times, there's the tiniest chance that if you executed this right at last part of the second at 23:59:59, then the value could roll over to a new date in the 2nd or 3rd call, and you could possibly get the wrong date, or even year. Unlikely, I know. But to avoid that, you'd want to capture the current date once, then operate on that, as in:

Frodo: setvar my_date hpyyyymmdd
Frodo: echo ![str(my_date,5,2)]/![str(my_date,7,2)]/![str(my_date,3,2)]
06/29/16

Stan Sieler adds

Kudos to Barry and Michael for apparently correctly guessing that our manager was interested in a CI-based solution. Next time, you might want to make that clear. I was thinking along the lines of SPL/Pascal/COBOL and the CALENDAR intrinsic, as well as whether I should ask the obvious questions or not. Of course, the easiest method is: ECHO 06/29/16

Any code that wants a coherent date and time using separate sources (e.g., CLOCK and CALENDAR intrinsics, or HPTIMEF (#1)) needs to be aware of midnight crossing. Thus, I use a helper function in Pascal like:

  procedure get_date_time (var cal : calendar_type; val clk : clock_type);
     begin
     repeat
        begin
        cal := calendar;
        clk := clock;
        end
     until cal = calendar; 
{avoid problems if ’clock’ called just after midnight} end;

Posted by Ron Seybold at 08:11 PM in Hidden Value, Homesteading | Permalink | Comments (0)

June 06, 2016

SLEEPER, Awakened for the Masses

Sleep-1389978_1280The contributed program SLEEPER is probably the software cited most often as proof of the riches of the Contributed Software Library. First created outside of the IT shops of Boeing (according to its first writer Ray Legault) the program was among the most classic of solutions for straightforward jobsteam management. You wouldn't mistake SLEEPER for something professional like Maestro, or even the free MasterOp. But SLEEPER was contributed to the 3000 community, not cast into the free wilds like MasterOp was after its commercial career ended. It was meant to be shared.

The trouble was, SLEEPER disappeared from the community's shelves when Interex died. The CSL tapes (and eventually CDs) went off the grid, another skid-mark left when the user group careened into the void in 2005. It's been 11 years, though, and it's finally time to at least make SLEEPER ready for a wake-up call. We've got the two simple source files to share.

Nobody has liability anymore for HP 3000 contributed software. SLEEPER was never released with support or a license; it was simply part of being an Interex member at a certain level. And let us take a minute to recall that Interex folded owing millions of dollars to members and vendors. After 11 years, it's time to make this software a community resource once more. "Hey, I know a guy who can get you that" has been the means to share the utility over the last decade.

Surprisingly, it's just a well-packed 184K of SPL and FORTRAN code. MPE magic never took up much space. That's one of the reasons it was magic. SLEEPER is also a fine example of how 3000 managers helped one another.

We've thrown caution to the winds (not exactly a new event here) and loaded those two files onto our website servers for distribution. It comes with no warranty, of course. Like always with the CSL gems, using it carries the same risk that any contributed utility or application did.

The FORTRAN file is available. And the SPL code is at hand, too. One well-honored MPE/iX engineer looked over the software programs and said they were extraordinary.

The SPL is some of the better looking old-style SPL that I've seen. The FORTRAN is very nice looking.

The SPL program runs in the batch job. The FORTRAN program maintains a configuration file that is presumably used by the SPL program.

So let the jobs of MPE/iX, including those running on standard HP hardware as well as those on virtualized 3000s, be managed a bit more easily. I await your comments on rousing the talents of this classic tool.

Posted by Ron Seybold at 10:06 PM in Hidden Value, Homesteading | Permalink | Comments (0)

June 01, 2016

Recovering Your Lost Logins

BWManIn the world of 2016, losing a password can be a time-bending experience. Apple's ID logins for its iCloud services, iTunes, and App Store only bear five attempts at most before they close out access to the phone. From there, you're on to a lengthy call to Apple. The smartphone is a tiny computer. The larger ones are not as strident about refusing repeated attempts, including the HP 3000.

They are secure, though. In these times when the servers can go dark for awhile as cold-start archive systems, though, a login can get misplaced. We're not supposed to write these things down, after all. What do you do when you have a question like the one below?

I restarted a 3000 9x7 after a few years in mothballs to run an old in-house app. I was able to boot up and login as OPERATOR.SYS, but cannot remember or find the password for MANAGER.SYS. Is there anyway to reset, clear, or overwrite the password file? I know the old machine is a very secure one, but now I am hoping there is a way around it.

What follows are a couple of suggestions to get back into the manager's driver's seat. However, you'll need to at least have a ticket to ride in OPERATOR.SYS.

Gilles Schipper shares what he knows.

There's not an easy way around the security. But since you can log on as OPERATOR.SYS, you should be able to store off the system directory to tape, as follows:

:file t;dev=tape
:store command.pub;*t;directory

Now that you have the directory on tape, you should be able to move around with FCOPY (and its ;char;hex options) to find the password for MANAGER.SYS. And that's all assuming you haven't implemented the directory encryption feature available with HP's security product Security Monitor/iX.

There's one other way. It's a good thing, because the alternatives do not include a rescue call to HP. There's nothing like Apple's password recovery support left for the HP 3000, as the vendor has long ago left the field. However, a good support contract for an MPE/iX server can be just the thing, and we know where you can get one.

Posted by Ron Seybold at 05:11 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 18, 2016

Tape drive changer a powerhouse for MPE

Autochanging HensAutochanging tape drives used to be the stuff of science fiction among 3000 managers, but those days passed by before HP cut off making Classic 3000 MPE V systems. Just because an autochanger is a 3000 storage option does not make it automatic to program, however.

A question posed to the community by Ideal Computer Services Ryan Melander reached for help on programmatically controlling such autochangers -- to select a slot, and load the tape and come to ready. "I am trying to configure an old DDS3 auto-changer, one that I don't believe will unload and load the next tape," he said.

Gilles Schipper noted that the command ad ldevno id=hpc1557a path=?? Mode=autoreply configures the device, and to advance tape after use, employ the command (from Devctrl.mpexl.telesup) ldev eject=enable load=online 

DAT tapesDenys Beauchemin mentioned HP's pass-through SCSI driver as a tool to drive the device's robot. The software was built by HP's labs and labeled as "not for the faint of heart" by engineers, but can assert a programmatic control over autochangers. Some third party programs such as Orbit's Software Backup+/iX can also do this work.

If ever there was a theme song for an autochanger at work, it would be a tune called Powerhouse. Children of the Fifties and Sixties will know it as soon as they hear it, if they've ever watched a Warner Brothers cartoon.

Some programming ideas came from Beauchemin, the engineer who developed at HiComp for the HiBack 3000 solution. "For the next tape to be brought online automatically, I seem to remember there had to be a special setting with the dip switches."

As for being able to control the robot itself, you definitely need to have the [HP] SCSI pass-through driver configured and loaded, and then you need a program to actually issue the IOCTL calls to the robot with the properly formatted SCSI commands. There was such a program a long time ago from a vendor, but that's all gone now.

Perhaps the high-test flutes and heavy octane horns of Powerhouse -- used in Duck Dodgers and the 24th and a Half Century -- can be put up on the MP3 player while fitting the driver to MPE. ("Oh drat these computers -- they're so naughty and so complex," says Marvin the Martian in one installment. "I could just pinch them.")

From our archives we found this advice from the late Jack Connor of Abtech, pointing to similar complex answers about controlling DDS changers.

Typically, there's a second SCSI port/address assigned for the transport control which allows the selection of specific tape. For MPE, stacker mode is typically selected, which tells the drive to just mount the next tape in line when requested. I don't know if the DDS autoloaders have a network connection available like the C7145NA DLT autoloaders do; with that device's web interface you can reload any tape, bypass a bad tape, and so on.

Back in 2011, John Pitman checked in to report that a much simpler solution to his changer's control needs popped up. "On re-examining my code for HPDEVCONTROL, I found I had catered for 1- and 2-digit device numbers in the string passed, but I had configured the drive as dev 777. This produced a string dev number of 77, which doesn't exist as a tape drive. Once I fixed this, it works like a treat."

Posted by Ron Seybold at 09:16 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 06, 2016

Options for STORE You May Not Know

StorehouseSTORE is the default backup tool for every HP 3000, but this bedrock of backup has options which might exceed expectations for a subsystem utility. Gilles Schipper, the support guru of three-plus decades on the 3000, told us about a few STORE sweet spots.

To start, using the :MAXTAPEBUF option can cut a four-hour backup to three hours or less. Schipper says that increasing the buffer size default to 32K, from the usual 16K, speeds up the backup when STORE sees MAXTAPEBUF. "That's a pretty good payback for one option."

Backups don't need to be specified with an @.@.@ command to be complete. "People should really be using the forward slash," he says, "because it's easy to accidentally omit the Posix file structure if you're not careful constructing your fileset backup." The slash is so much better that a backup specified by HP's TurboStore will replace any @.@.@ operation with "./" Combining @.@.@ with exclusions can lead to omitting files which should have been in a backup.

Schipper says that including a directory on a backup is smart, but private volumes in use on a 3000 need more than :directory as an option.

"If you're using private volumes and the directory option, you'll only get the system volume directory — and you will not get the private volume directories," he says. The backup must explicitly specify the volumes through the ONVS= command, using the long name of the private volume.

The partdb option on a STORE command ensure that any 3000 databases which are incomplete will get backed up. Without partdb, if a root file "doesn't have its corresponding data sets, the root file won't get stored. It's silly, but partdb ensures they get stored." A privileged file can also look like a partial database, so partdb brings those files into a STORE backup.

A 3000 with HP's TurboStore, rather than just the default STORE, can take advantage of the :online command. "It will give you zero downtime, if you have that version from HP," Schipper says. But that begins to drift away from the no-cost STORE options available to any HP 3000 administrator or owner.

Posted by Ron Seybold at 05:47 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 04, 2016

CPR for a Non-Responsive Console

On my HP 3000, after a short power blip, the console is now non-responsive. I can connect to the system's GSP port and the session is connected, but nothing is displayed. Neither <ctrl> A or <ctrl> B works. I type away, but get no response. I can then connect via VT-MGR and take the console :console !hpldevin and I receive all the console messages.

So, the messages are being sent (since I see them on the VT connection), but neither the physical console or the GSP gets any console messages. What can I try?

Gilles Schipper says

I believe a START NORECOVERY reboot is in order here. Since <ctrl> A <ctrl> B do not work, you will need to power-recycle the machine to effect a reboot. Presumably you would want to do this after gracefully stopping all jobs and asking online users to log off, if possible.

Depending upon which patch level your level of MPE is on, the :SHUTDOWN RESTART MPE command may also work from a logged-on session with at least OP capability.

Mark Ranft adds

If you haven't rebooted, I've seen similar issues. From the VT console can you try to do 'abortio 20' until it says no I/O to abort. A WHILE loop may make this easier. I've had luck with this in the past. But since Ctrl-B doesn't work, you may be out of luck.

Robert Thwaites notes

These are the simplest things to try first

<ctrl>Q (x-on)

<return>

Among the commonest issues: forgetting to do an x-on after a <ctrl>S (x-off) to stop output, so you can look at the line you are interested in. One time I saw another issue where someone had pressed <space> on the console and hadn't pressed <return>.

Posted by Ron Seybold at 10:37 PM in Hidden Value, Homesteading, User Reports, Web Resources | Permalink | Comments (0)

April 22, 2016

How to Transform MPE Spoolfiles to PDFs

HP 3000 data becomes more useful if it can be e-mailed as industry standard report documents. After more than two decades of pushing at it, Adobe has made its PDF the de-facto way to exchange documents, even complex ones.

Which might have prompted this question from HP consultant and Suprtool trainer Jeff Kubler:

Does anyone have a lead on a tool that converts spoolfiles to PDF files? Are there any Contributed Library tools?

It won't be in the Interex Contributed Library (because the programs have gone underground; ask your colleagues if they have a swap tape) but the txt2pdf product works nicely to make this conversion. Even in its most advanced version it's about $1,000, at last glance. Bob McGregor reports as much.

Jeff, txt2pdf does this. We have a job that runs that:

1. Checks a pseudo device we have setup for any spoolfiles that are on the device with an PRI >0

2. If it finds a spoolfile, we convert it to PDF and move it to a server

3  Sends an e-mail to the streamedby variable telling them the PDF doc is ready on the server.

4. Alters the priority to 0 to mark it processed

We've been using it for a couple years, and it works great — of course, once we got the bugs worked out. What's cool is if someone delete the file, we just adjust the priority to something greater than 0 and it gets reprocessed.

Posted by Ron Seybold at 08:52 PM in Hidden Value, Migration | Permalink | Comments (0)

April 15, 2016

Use GSP and More for 3000 ID, control

GSP boardThere's been plenty of change in the 3000 manager's life over the last 15 years. Some of the change is a little easier by exploring the use of the Guardian Service Processor. It can help in changing the location of HP 3000 consoles from one part of a shop to another. There's even a story out there that says the identity of the HP A- and N-Class hardware resides in the GSP.

Kent Wallace, while a 3000 manager for Idaho-Oregon healthcare delivery system Primary Health, needed to move his 3000 console.

I must move the 3000 console another 10 feet farther from the rack (it's an N-Class HP 3000/N4000-100-22). What are the 3-pin positions on the wire that I need to extend its RS-232 cable?

Our blog contributing editor Gilles Schipper chipped in with a solution offering even farther movement:

If you want to extend the range of the console to anywhere on the planet (at least where there’s Internet access) you could consider the HP Secure Web Console to replace the physical console. Depending upon the condition of your physical console, this solution may also save a bit of wear and tear on your eyeballs, instead of adjusting those pins.

Schipper wrote us a great article on setting up such a web console. The GSP is quite the tool.

Former HP support engineer Lars Appel offered another take on Schipper's strategy:

While Gilles is right about the possibility of using the web console, it would probably be easier to use the already built-in dedicated LAN port of the N-Class systems that gives access to the GSP by telnet.

I prefer the “telnet console” over the “web console” because it gives more freedom in the choice of terminal emulator — whereas the web console typically lacks features like “easy cut and paste” or special key mappings (e.g. German language ;-) or something similar.

This prompted Schipper to clarify his suggestion:

Lars is absolutely right about the built-in “secure-web-console” that comes with all N-Class and all but the earliest A-Class e3000s.

And, yes, the built-in is definitely more functional, allowing cut-and-paste as well as telnet access, whereas the external variety has only Java access to it via a web browser and no cut-and-paste.

So, if one has a choice, the built-in is definitely superior and available with only proper configuration. However, the external secure web console is available for all HP 3000s, and would still be most useful where is internal secure web console is not an option.

The GSP is a powerful part of an HP 3000 that runs whenever the server is plugged in. It's the maintenance control console commanding the ultimate class of the server to reboot, do memory dumps and even fully power down the 3000. Consultant Craig Lalley has noted the GSP has one fewer feature than its Unix counterpart, though.

"On HP-UX it is possible to reset the GSP from the host OS," he said. "I have not found a way from MPE."

From time to time a reset may be required for diagnostics services on A-Class and N-Class servers. If your 3000 gets loving care from a consultant or service provider outside your computer room, you may need a paper clip to keep up service levels.

The GSP can also reveal the 3000's speedometer, as profiled near the bottom of an Allegro webpage.

The gap between controlling the 3000 and HP's HP-UX Integrity GSPs is a common shortfall of HP designs. Even though the 3000's MPE/iX includes a Posix interface, HP didn't engineer enough Unix into the 3000 to enable some of the administration that HP-UX users enjoy. (That lack of Unix can sometimes be a good thing, however, when a security breach opens up in the Unix world.)

But when a 3000 needs a GSP reset, pressing a recessed button on the 3000's back will do the trick if a telnet command doesn't work. You can telnet to the IP address of the GSP, log in and do the reset. But you can also get someone to press the physical reset button at the back of the machine. It's recessed into the cabinet so you may need a magic paper clip bent just-so.

Lalley calls the GSP, which HP introduced with its final generation of 3000s, one of the most useful things in the A-Class and N-Class boxes.

The GSP is a small computer that is always powered on when the plug has power. With it, it is possible to telnet to and BE the console. While multiple admins can telnet in and watch, only one has the keyboard.

It is possible to reboot, memory dumps and even fully power down the HP 3000 from the GSP. Use the command PC OFF.

It is probably the best feature of the N-Class and A-class boxes. The problem is sometimes it needs to be reset, usually with a paper clip. Since the GSP is a different CPU, this reset can be done during business hours.

 

Posted by Ron Seybold at 08:35 PM in Hidden Value, Homesteading | Permalink | Comments (0)

March 18, 2016

Big files get zipped, moved on HP 3000s

ZipperA computer manager who's new to the HP 3000 is looking for CSL files this week. The Contributed Software Library is just an oasis to this IT veteran, something shimmering in his future that holds a highly useful thicket of utilities and more.

Someone in the 3000 community is bound to connect our new user with this CSL, for one reason: he's looking for MPE V programs to supplement his discovery of the emulated Classic HP 3000, simh. That's the MPE V-ready version of a virtual HP 3000: what amounts to a CISC skin for a 3000 on top of the simh code. Whenever the newbie connects with a CSL resource, if they've got their files on a 3000 they're bound to need to send about 24MB to him. That's going to require zipping them.

The act of zipping to compress for a transfer is an essential in 3000 management. Although the code for compressing files on HP 3000s is more than a decade old, like a lot of things on the system, it continues to work as expected.

Tracy Johnson, who manages the Invent3K server operated by OpenMPE, noted he's using the MPE/iX Posix shell's compress and uncompress. "It creates a file that ends in capital Z. Seems the compressed format is compatible with both GNU-zip and Winzip programs or any other Unix/Linux machine."

Lars Appel, who ported the Samba file sharing tool to MPE, offers a comprehensive answer. He points to an HP 3000 Web starter software kit that resides on a development server, open to the public.

Lars explained in a post a few years ago:

You can pick up the InfoIP zip/unzip programs (in a tar file) at my area in the Infocorp server. The link in that webpage that contains the zip/unzip programs is

E:\WebKit2\upload\infozip.tar.Z

Transfer it to the 3000 in bytestream or (fixed) binary format and then unpack with :/bin/tar "-xvzopf FILENAME". Place the two programs where you like; I typically have them in /usr/local/bin or (with uppercase filename) in a group or directory that is part of my HPPATH settings.

The webpage also contains a tar.Z file with /usr/local/bin/gzip

E:\WebKit2\upload\gnuzip.tar.Z

(gzip -d decompresses; creating a symbolic link gunzip is also useful)

Posted by Ron Seybold at 06:04 PM in Hidden Value, Homesteading | Permalink | Comments (1)

March 04, 2016

3000-L breaks silence with DTC primer

DTC NetworkOn the verge of four weeks without a new message, the 3000-L mailing list and newsgroup delivered a primer on the control of DTCs on MPE/iX networks. A longtime contributor to the community set up a question about using these venerable devices that connected HP's terminals and other devices to a 3000.

I have multiple HP 3000s sharing one DTC.  My problem is, which one controls the DTC?  In the event of  a power cycle, there is a race between them for which 3000 will download the new configuration.  I need system A, my A-Class, to be in charge. However, systems B or C are most likely to download first, leaving me with the manual step of unplugging the network from B and C, power-cycling the DTC, and then waiting for the A-Class to download the configuration. Is there anything that can be done to just leave the A-Class in charge?

Tracy Johnson replied, "It's always been a crap shoot. I'm of the opinion the first HP 3000 to notice the DTC needs downloading will do the job. Which usually means the less busy machine." As it was just the first answer on the newsgroup, there was still a need to do unplugging.

"For example," he added, "if you have a test machine on the side nobody's using, I would simply unplug the test machine's network cable until the DTC was reset. Since there will be no other machine in control, the regular machine will do the download.  When complete, I'd plug the network cable back into the test machine. However if both machines are in production and you have users active on both, you'll have to make a decision."

Lalley came back with an answer of his own, at least to eliminate the travel from chair to server.

"On the DTC's I guess it could be possible to do it without leaving your chair, if you TELNET to the console on B and CDTCCNTRL (has to be run from the physical console)  to stop the DTC subsystem, then restart it."

Mark Ranft at Pro3k had another idea.

Back in the good old days, we used DTC Switching - a feature that allowed a DTC prompt to appear at a terminal.  The user entered 'C HPA' to connect to host HPA.  I believe you could set a default system.  Initially DTC Switching was only available if you configured and downloaded the DTC using OpenView DTC Manager software running on a PC.  Later HP set up NMMGR to allow DTC Switching.

The Communicator for that release may have more details, but I only entered the MAC of the DTC on the one system that was supposed to download the DTC.  The other systems would have the DTC name, but not the MAC.  This allowed the system to have printers and other devices accessible on the DTC without concerns about which one downloaded the configuration.

Of course, the location of that MPE Communicator took all of 41 minutes to dig up. Barry Lake pointed at HP documentation (now hosted outside HP's baffling website) that covers Enhanced Host-Based DTC Management Functionality in Chapter 3. Plus, a manual on Configuring and Managing Host-Based X.25 Links.

Not too shabby for a mailing list that's more than 20 years old, but still manned by dozens of experts. It's the kind of expertise a good third party support provider offers—as usual, better than HP's today.

Posted by Ron Seybold at 06:33 PM in Hidden Value, Homesteading | Permalink | Comments (0)

February 26, 2016

21 days of radio silence on the 3000-L

Right_WirelessTelegraphThe slowing current of 3000 communication showed a fresh signal by the end of this month. As we write it's been 21 days since a message of any kind on the 3000-L MPE newsgroup. The resource that carried 45 messages during last February has 10 for the current month. All of this month's traffic was wrapped around finding resources: Brian Edminster of Applied Technologies and Vesoft support. Both were located.

However, the three weeks without a new message is new territory for the community's log of technical help and outreach by cohorts. Among those who were posting during 2015, several told us they're on the mailing list-newsgroup out of habit — rather than needing details for their datacenter's 3000s.

"I’m still on the list out of inertia, nostalgia and mild interest," said Dave Heasman, a UK IT manager. "My employer got rid of their 3000s and me in 2008. Bought a series of packages to replace a big bespoke brokerage/investment system."

Robert Mills said he "remained a member of the list, mainly as a lurker, to keep appraised of what was happening in the 3000 community. Except for three requests in September 2012, December 2014, and February 2015, I've only posted to the list when I felt that the 3000 knowledge I had would help somebody solve a problem." Mills said he retired when his company went insolvent in 2009, but he's kept his hand in IT.

"I have been involved with the GnuCOBOL (formerly OpenCOBOL) Project on SourceForge since October 2014, and decided to write a macro preprocessor that emulated the functionality available on the 3000," he said. "The preprocessor, CobolMac, is now in its 5th version (B.04) and has received good reviews by its users."

Others who contacted us said they haven't worked on the 3000 since the days that HP sold support for MPE/iX. "I have been a BizTalk developer full time since 2008," said Kent Wallace. "I needed to work, and this was the direction the world was going." The 3000-L still has more than 500 subscribers on its mailing list rolls, but much of the messaging comes from consultants and vendor experts, supplying answers to questions and tips. A total of 45 messages have passed through the list since the start of 2016. The IT pros like Wallace have taken the path to other platforms, first to HP-UX, then to Windows.

"I left my previous employer in Boise and I moved to a Microsoft shop, whose mainframe was HP-UX," Wallace said. "However, in 2015 we migrated off HP-UX and onto SQL Server on Microsoft Server 2008. We do health insurance and the purchased software, Trizzeto, was moved to MS SQL servers."

Another registered user keeps up with the community, but he can imagine a future where he'd be back on the MPE/iX front lines. "We're totally out of the 3000 business," said Ted Johnson of Wake Forest University, adding a sad-face emoji. "But I love seeing the 3000-L posts and keeping up. Who knows — maybe they'll get rid of me one of these days, and I'll end up back on a 3000."

The 3000-L, hosted at the University of Tennessee at Chattanooga where the late Jeff Kell launched it the early 1990s, holds more than two decades of traffic. 10 years ago the list was big enough to measure a signal-to noise-ratio, but by now it's almost entirely signal. When John Burke was a monthly columnist for the NewsWire who summarized its content in net.digest, he rounded up the following help in just one month's communications. For a 3000 owner managing a homesteading shop, the 3000-L's tips still carry some value.

Quick Cuts

• Do you want to know when a particular account or group was created? LISTACCT and LISTGROUP are no help. But “listfile /ACCOUNTNAME,3” for the account or “listfile /ACCOUNTNAME /GROUPNAME,3” for the group tell all. And then some.

• The number of sectors reported by the REPORT command for a group or groups is sometimes inaccurate, sometimes very inaccurate. Running the program FSCHECK.MPEXL.TELESUP and issuing the SYNCACCOUNTING command will fix this problem.

• In case you were wondering, despite many requests for the enhancement, TurboStore will NOT append store sets to tape. Well, it might if you use the proper incantations, but it is unsupported and highly dangerous because under certain circumstances you could overwrite a previous backup without knowing.

• Speaking of things you cannot do that you might like to do, the ALLOW command is not persistent across sign-ons unless you use the extremely dangerous “ALLOW @.@; commands” version. This is another example of an enhancement that has been requested for years, but now will never happen. Fortunately, there are a number of options, for sale and free (MPEX, CSL, etc.).

• CI integer variables are signed 32-bit entities. So be careful if you are doing some wild arithmetic in your CI scripts.

• Here is a little trick when using Apache’s indexing (for example to keep track of documentation) to index file displays. You can override the default ascending sort by name by appending “?N=D” to the url. Instructions on changing Apache’s default behavior are available on the Web.

• If you are trying to program VPlus applications and are interested in working examples programmed in your favorite language, look in the group HP32209.HPPL89 (which should be on every FOS tape). This group contains source code for the ENTRY program in a variety of languages including COBOL, Fortran, Basic and Pascal.

• To see the firmware (aka PDC) Revision of a system (CPU): Run cstm, and at the cstm > prompt, type ‘map’ and note the Dev Num of a CPU and then type ‘sel dev DEV_NUM’ (e.g., ‘sel dev 41’) and then type ‘info’ and then type ‘il’ and look at the output for the ‘PDC Firmware Revision’. Easy, huh? Thanks to Guy Paul of HP for this tip.

• SPFXFER will allow you to write to disk (undocumented “feature/bug”). But don’t do it, because SPFXFER cannot read the disk file it creates! Doing this could lead to a big oops.

• While it would certainly be a nice to have, MPE/iX CI scripts have no provision for inline comments. Sorry, don’t even bother trying.

Posted by Ron Seybold at 05:01 PM in Hidden Value, Homesteading, Migration, User Reports | Permalink | Comments (1)

February 19, 2016

How hot plug disks can replace DDS offsite

300 GB Ultra SCSII need to find an alternative to DLT and DDS tapes for offsite storage. Sure, there's DS2100 and Jamaica drives. But a few $35 300GB Ultra SCSI drives would hold a lot more data with less points of failure. I'll set up a BACKUP_VOLUME_SET and use the internal disks to do store-to-disk backups of the system. 

I've always used my A-Class and N-Class systems with fiber-attached disk. Are the internal disk drives hot-pluggable? 

Jim Hawkins, IO maven for HP 3000 systems at HP, replies with details.

There are multiple layers of changes for actual hot plugs or swaps to work.  

  • You need the disk HDD to handle this electrically.
  • You need HDD physical carrier and physical interface to comply.
  • You need the system physical interface and receptacle to comply. 
  • You need your Host System Bus Adapter (HBA) to electrically support this.
  • You need the OS to be aware enough of the HBA to not get flustered by absence of the device and deal with any notifications from the HBA of the activity.   

Given that the N-Class disk cage has a screw-based cover and the HDD carriers have no quick release levers (as compared with HASS/Jamaica or VA7400) I would state definitively that there is no hot-plug intention.  

At the same time, the SCSI bus is pretty low power and low voltage, so it would be generally not-too-unsafe to experiment. But you're also close to AC inputs and they are not low power.

Hawkins explored the not-too-unsafe scenario with theoretical possibilities.

Might you be able to pull/push a drive where you've closed the volume?  Likely it would work, but there may be all kinds of noise and stress on the SCSI bus which may not be well handled. However, I think each disk is on its own HBA channel which isn't shared with anything else, and so unlikely to abort someone else's IO.  

This takes us to the last issue: mechanical wear.  

These connectors were likely intended for more or less permanent mating of two components. Very likely they have a limited number of cycles that they are specified to hold-up. I've seen connectors that are specified for fewer than 25 cycles before you lose gold contact material.   This is okay for normal HDD where one might replace one or two per slot in a system lifetime, but not sufficient if you're doing nightly back-ups and swaps.  Connectors, where there is an expectation of a high number of pull/replace cycles, have special designs.  

Now a little good news here is that the N-Class was still pretty much old-school HP design, so likely they didn't pick up something cheap that saved them .2 cents per unit on gold plating. No idea though if the HDD connector is a 10-, 100-, 1000-cycle part. Your system, your risk. 

Consultant Mark Ranft points out that the HP design for 3000s seems to make the servers and components good candidates for exceptional mechanical wear tolerances.

It is especially helpful to understand the concept of mechanical wear on the connectors. HP always had excellent and innovative hardware engineering on their HP 3000 servers. Remember, you can drop them off a building and still self-test them

The Unix N-Class appears to allow hot-pluggable drives.   

The actual power supply and the fans are in the front of the N-Class.  The power receptacles in the back have internal cords that lead to the front.

Posted by Ron Seybold at 08:59 PM in Hidden Value, Homesteading | Permalink | Comments (0)

February 12, 2016

How to get specific about IP access for PCs

I want to give a 3000 a static IP, so I can permit a user to access the HP 3000 from that PC with that static IP. Is there a way to force a particular user ID to use a specific IP address?

Tracy Johnson replies:

A simple logon UDC should suffice:

IF HPREMIPADDR = "aaa.bbb.ccc.ddd" then
  ECHO Welcome.
ELSE
  ECHO Evil message here.
BYE
ENDIF

Bob Schlosser adds:

You can set up a logon UDC that checks that the var HPLOCIPADDR is equal to the device (PC) that you want them to use. Something like this:

LOGON
OPTION NOBREAK,LOGON
IF "!HPLOCIPADDR" <> "123.456.789.321"        change "123.456.789.321" to
your IP address
  BYE
ENDIF

Using this, we verify that the user is on the correct (assigned) IP address, and log them off if not.

Chris Bartram, who's created e-mail solutions for the 3000 at 3K Associates, and hosted Web servers since early in the 1990s, adds:

The following is an excerpt from system UDCs I use on my HP 3000s that might give you some ideas.

The "VALIDATEIPADDR" call in this UDC calls another command file that actually does a validation of the logging-on user based on data in a control file to determine if he/she is allowed to log onto the system from the specific host/IP address they are coming from.

The variables the UDC sets will work whether the logging on user is coming in via Telnet or NSVT (or hardwired or via a modem).

The TELLOPs also leave a nice log on the system console (and log file) of the login, including where they came from, and what protocol was used to access the system.

***
LOGON
OPTION LOGON,NOBREAK,NOHELP

setvar _network_node ''
if bound(hpstdin_network_node) then
  setvar _network_node '!hpstdin_network_node'
endif

setvar _na ''
setvar _at 'HARDWIRED'
if bound(hpstdin_network_addr) then
  setvar _na '!hpstdin_network_addr'
elseif bound(hpremipaddr) then
  setvar _na '!hpremipaddr'
endif

if bound(hplocport) then
  if !hplocport=23 then
    setvar _at 'TELNET'
  endif
endif
  IF BOUND(HPSTDIN_ACCESS_TYPE) THEN
    SETVAR _AT "!HPSTDIN_ACCESS_TYPE"
  ENDIF

IF BOUND(HPSTDIN_TRANSPORT_TYPE) THEN
  SETVAR _TP "!HPSTDIN_TRANSPORT_TYPE"
ELSE
  IF "!_AT"="TELNET" THEN
    SETVAR _TP "TCP/IP"
   ELSE
    SETVAR _TP "SERIAL"
  ENDIF
ENDIF

IF BOUND(HPVT_CLIENT_VENDOR) THEN
  SETVAR _VND " (!HPVT_CLIENT_VENDOR)"
ELSE
  SETVAR _VND " "
ENDIF

TELLOP LOGON VIA !_AT USING !_TP !_VND

setvar _node ups(ltrim(rtrim("!_network_node")))
setvar _addr ups(ltrim(rtrim("!_na")))
if '!_node'<>'' then
  tellop !_at, IP: "!_addr" Node: "!_node"
else
  tellop !_at, IP: "!_addr"
endif

setjcw cierror=0
continue
VALIDATEIPADDR
if !cierror<>0 then
  echo
  echo ************************************
  echo **  NODE/IP CONTROL FILE CORRUPT  **
  echo ************************************
  echo
  bye
endif

Posted by Ron Seybold at 03:56 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 18, 2016

The GSP makes the A and N worthwhile

GSP boardIt's a powerful part of an HP 3000 that runs whenever the server is plugged in. The Guardian Service Processor (GSP) is the maintenance control console commanding the ultimate class of the server to reboot, do memory dumps and even fully power down the 3000. Consultant Craig Lalley of EchoTech has noted the GSP has one fewer feature than its Unix counterpart, though.

"On HP-UX it is possible to reset the GSP from the host OS," he said. "I have not found a way from MPE."

From time to time a reset may be required for diagnostics services on A-Class and N-Class servers. If your 3000 gets loving care from a consultant or service provider outside your computer room, you may need a paper clip to keep up service levels.

The GSP can also reveal the 3000's speedometer, as profiled near the bottom of a webpage from Allegro Consultants.

The gap between 3000 and HP's HP-UX Integrity GSPs is a common shortfall of HP designs. Even though the 3000's MPE/iX includes a Posix interface, HP didn't engineer enough Unix into the 3000 to enable some administration that HP-UX users enjoy. (That lack of Unix can sometimes be a good thing when a security breach opens up in the Unix world.)

But when a 3000 needs a GSP reset, pressing a recessed button on the 3000's back will do the trick if a telnet command doesn't work. You can telnet to the IP address of the GSP, log in and do the reset. But you can also get someone to press the physical reset button at the back of the machine. It's recessed into the cabinet so you may need a magic paper clip bent just so.

Lalley calls the GSP, which HP introduced with its final generation of 3000s, one of the most useful things in the A-Class and N-Class boxes.

The GSP is a small computer that is always powered on when the plug has power. With it, it is possible to telnet to and BE the console. While multiple admins can telnet in and watch, only one has the keyboard.

It is possible to reboot, memory dumps and even fully power down the HP 3000 from the GSP. Use the command PC OFF.

It is probably the best feature of the N-Class and A-class boxes. The problem is sometimes it needs to be reset, usually with a paper clip. Since the GSP is a different CPU, this reset can be done during business hours.

 

Posted by Ron Seybold at 10:43 AM in Hidden Value, Homesteading | Permalink | Comments (0)

January 13, 2016

Using Store-To-Disk for Backup Preservation

By Brian Edminster

Second of two parts

Yesterday I outlined some of the powers of the Posix program pax, as well as tar, to move MPE/iX backup files offsite. Here’s a warning. There are some file types that cannot be backed up by tar/pax while also storing their attributes:  ;CIR (circular) and ;MSG (message) files (and possibly others. I haven’t tested all possible file types yet.  Also, there is an issue with tar that is a fairly well known and has been discussed on the 3000 newsgroup. Occasionally it does not un-tar correctly.  It is unclear if and when this was fixed, but I’d love to hear from anybody that might be in the know, or which specific situations to avoid.

Regardless of these limitations, I’ve found a simple way around this. Use store-to-disk to make your backup, then tar to wrap it, so as to preserve the store-to-disk files’ characteristics, before shipping the files off-system. Later, when you retrieve your tar backups and un-tar them, you’ll get your original store-to-disk files back without having to specify the proper ‘;REC= , CODE= , and DISC=’ options on an FTP ‘GET’. I’ve been doing this for several months now on several systems, and I have not had any failures.

If you have a version of STORE that has compression, use it to reduce the size of backup.  If not, use the ‘z’ option in the tar/pax archive you create from your store-to-disk backup.  Do not use both.  They don’t play well together, and you may end up with a larger tar file.

But what about the tar archive size limit of 2GB?  There’s an easy way around this as well, as this limit is common on early Unix and Linux systems. Just pipe the output through ‘split’ to create chunks of whatever size you want. Below, there's simple examples for both directions.

Piping TarFigure 1, just below, is an example of the ‘cksum’ file produced.

Checksum 1 Grey

Below, Figure 2 is an example of a ‘cksum’ created of the files as they’re stored on the NAS. 

Checksum 2 GreyAs both the hashes and #bytes shown in each file are the same as on the MPE/iX server — we know the backups are transferred correctly.  The same technique can be used ‘in reverse’ to verify that when FTP’d back to the FTP server, they’re still intact.

When un-taring this backup, ‘cat’ the pieces together and pipe it through tar.   At least, that’s the way it’s supposed to work.  Yes, there is a known issue with the MPE/iX Posix shell’s built-in cat command. But I’ve so far been unable to successfully use the external cat command to successfully cat either.  Here’s how this should work for a 2-chunk tar backup:

sh>/bin/cat ./CS1STD1.ustar.aa ./CS1STD1.ustar.ab | tar -xfv - *

Unfortunately, for me at least, it always throws an error indicating bad format for the tar files.There is a work-around, however.  Note that while ‘cat’ing the tar ‘chunks’ didn’t work using the internal or external cat command, untar with multi-file option does work.  Even though it gives a minor error messages, files were returned to proper store-to-disk format, and the recovered store-to-disk backup is intact and has been used to recover the desired files. To do this, use tar like this: 

sh>tar -xfv ./CS1STD1.ustar.aa *  

Also note that when using tar in this way, it will ask for the name of the 2nd-nth component tar files, as it finishes reading each prior piece.  You must give the filename and press return to continue for each.  I believe that it should be possible to script this so that it’s fed the filenames, but I haven’t gotten around to doing that yet.  

Brian Edminster is president of Applied Technologies, a 3000 consultancy serving MPE/iX sites in contract and ongoing engagements.

Posted by Ron Seybold at 05:44 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 12, 2016

Backing Up Your 3000 Backup Files

By Brian Edminster
Applied Technologies 

Once store-to-disk backups on the 3000 are regularly being processed, it’s highly desirable to move them offsite — for the same reasons that it’s desirable to rotate tape media to offsite storage. You want to protect against site-wide catastrophic failures. It could be something as simple as fire, flood, or a disgruntled employee, or as unusual as earthquake or act of war.

Regardless of the most pressing reason, it really is important to keep at least some of your backups offsite, so as to facilitate rebuilding / recovering from scratch, either at your own facility, or at a backup/recovery site.

The problem comes in that the MPE/iX file system is far more structured than Unix, Windows, or any other non-MPE/iX file system-based storage mechanisms. While transferring a file off MPE/iX is easy via FTP, sftp/scp, or rsync, retrieving it is problematic, at least if you wish the retrieved files and the original store-to-disk files to be identical (i.e., with the same file characteristics: filecode, recsize, blockfactor, type, and so forth).

What would be optimal is automatic preservation of these attributes, so that a file could be moved to any offsite storage that could communicate with the MPE/iX system. Posix on MPE/iX comes to the rescue.

For FTP transfers between late-model MPE/iX systems this retrieval is automatic, because the FTP client and server recognize themselves as MPE/iX systems.  For retrieving files from other systems, HP has made that somewhat easier by making its FTP client able to specify ‘;REC= , CODE= , and ;DISC=’ on a ‘GET’:

Figure 1If you do not specify the ‘buildparms’ for a file being retrieved, it will default to the file-type implied by the FTP transfer mode: ASCII (the default), binary, or byte-stream (often called ‘tenex’ on Unix systems).  The respective defaults used are shown below:

Figure 2 GreyWhat follows is an example of automatic preservation of these attributes, so that a file could be moved to any offsite storage that could communicate with the MPE/iX system.  And this is yet again where Posix comes to the rescue, via the venerable ‘tar’ (Tape ARchiver), or ‘pax’ archiving utilities.

‘pax’ is a newer backup tool, designed to be able to read/write with tar format archives, newer ‘ustar’ format (that includes Extended Attributes of files). At the same time it has a more ‘normal/consistent’ command syntax (as Unix/Posix stuff goes, anyway), plus a number of other improvements. Think of it as tar’s younger (and supposedly more handsome) brother.

A little known feature of most ‘late-model’ tar and all pax commands is the ability for it to recognize and utilize Extended Attributes.  These will vary with the target implementation platform, but for the tar and pax commands included with releases after v5.5 of MPE/iX this capability is not only present — but contrary to the man command’s output and HP’s Posix Command Line manual, it’s the default! You use the -A switch to turn it off, returning tar to a bytestream-only tool.

While not externally documented, via a little experimentation I’ve determined that the following series of Extended Attributes value-pairs are in the MPE/iX Posix implementation of a tar or pax ‘file header’ for each non-Posix file archived:

MPE.RECORDSIZE= value in bytes
MPE.BLOCKFACTOR= integer value
MPE.RECORDFORMAT= integer value (0=unstructured?)
MPE.CCTL= integer value (0=nocctl)
MPE.ASCII= integer value (0=binary, 1=ascii)
MPE.FILECODE= integer value, absent for ‘0’
MPE.FILELIMIT= value in bytes
MPE.NUMEXTENTS= integer value, may be absent
MPE.NUMUSERLABELS= integer value (0=no user labels), and
MPE.USERLABELS=[binary content of user labels]

Brian Edminster is president of Applied Technologies, a 3000 consultancy serving MPE/iX sites in contract and ongoing engagements.

Posted by Ron Seybold at 02:38 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 05, 2016

Migrating 3000 Data from Spoolfiles to Excel

I need assistance with putting an output spool file from MPE/iX 7.5 into Excel or other readable format. The file is generated by Query, then processed by Editor, then sent to the printer. Instead of printing it, I want to put it into a readable format.

MigratingI do not have QEdit or any smart tools on MPE, so my approach thus far has been to move the file to a PC before doing anything.  However, that carries with it the initialization sequence for the printer to which the job is spooled. The job is set up to print on a PCL 5 laser, which means it has hundreds of lines of control before the data starts.

Tom Moore replies

I would put commas in between my columns (in the query, or using Editor). I FCOPY from the file to a new file with NOCCTL to get rid of carriage control byte. You could also remove the PCL 5 lines by subset in the FCOPY command. Depending on the data, I would use EDIT3000 to change all " ," to "," and all ", ","," to compress the file, removing the spaces before and after the commas inserted above, then save the file for download to the PC.

I would also consider using ODBC to directly extract from the IMAGE database, rather than Query and all the subsequent steps. The HP free ODBC driver would do the job very well.

Birket Foster of MB Foster notes

Not only did we make that free ODBCLink/SE as HP's lab resource from 1998 to 2006, but we have continued to develop the ability to work with data in all kinds of file formats. We do supply 32- and 64-bit versions for ODBC to the HP 3000.

UDALink-MPE was designed for the HP 3000. We provide data in several different formats including XLS for Excel, XML, CSV etc. We can have a discussion about what you are trying to do with data; perhaps UDACentral is the right product for your challenge and we can organize a demonstration for you.

Charles Finley adds

There seem to be at least three steps to what you are trying to do.

  • Remove the headers, footers and perhaps page numbers from the report.
  • Remove the ff or CNTL characters from the text file.
  • Import a space-delimited file to Excel.

There are any number of different scripting tools that can do this including various Unix tools. Here's a reference to an Excel solution that might get you started. In fact, if it were my problem to solve, I would likely do it all with Excel scripting.

John Hohn replies

  1. Output to a delimited file (tabs, pipes, etc).
  2. Download to your laptop or PC or wherever Excel is running
  3. Start macro recording in Excel
  4. Import/format the delimited file, save as .xls
  5. Turn recording off, save macro

Set the Excel file to auto-execute the macro every time the Excel file it's opened, i.e., re-input/format the delimited file. Then you can, for example, schedule delivery of a new version of this delimited file whenever you'd like, to your server. When people open it they would automatically get the formatted version of the new data.

Connie Sellitto of Hillary Software suggests

Hillary Software has a product, byREQUEST, which does just this.

It has the ability to suppress headings on pages after the first, and define the type of data in the columns (text, numeric, dates in various formats). It can remove blank pages and leading and trailing blank lines. It can even call an Excel macro to make the headings a different font, background color, etc — anything you'd want to do with a macro. In addition to Excel, byREQUEST can create a PDF file, Word, csv or Text.

Posted by Ron Seybold at 10:25 PM in Hidden Value, Homesteading, User Reports | Permalink | Comments (0)