June 06, 2016

SLEEPER, Awakened for the Masses

Sleep-1389978_1280The contributed program SLEEPER is probably the software cited most often as proof of the riches of the Contributed Software Library. First created outside of the IT shops of Boeing (according to its first writer Ray Legault) the program was among the most classic of solutions for straightforward jobsteam management. You wouldn't mistake SLEEPER for something professional like Maestro, or even the free MasterOp. But SLEEPER was contributed to the 3000 community, not cast into the free wilds like MasterOp was after its commercial career ended. It was meant to be shared.

The trouble was, SLEEPER disappeared from the community's shelves when Interex died. The CSL tapes (and eventually CDs) went off the grid, another skid-mark left when the user group careened into the void in 2005. It's been 11 years, though, and it's finally time to at least make SLEEPER ready for a wake-up call. We've got the two simple source files to share.

Nobody has liability anymore for HP 3000 contributed software. SLEEPER was never released with support or a license; it was simply part of being an Interex member at a certain level. And let us take a minute to recall that Interex folded owing millions of dollars to members and vendors. After 11 years, it's time to make this software a community resource once more. "Hey, I know a guy who can get you that" has been the means to share the utility over the last decade.

Surprisingly, it's just a well-packed 184K of SPL and FORTRAN code. MPE magic never took up much space. That's one of the reasons it was magic. SLEEPER is also a fine example of how 3000 managers helped one another.

We've thrown caution to the winds (not exactly a new event here) and loaded those two files onto our website servers for distribution. It comes with no warranty, of course. Like always with the CSL gems, using it carries the same risk that any contributed utility or application did.

The FORTRAN file is available. And the SPL code is at hand, too. One well-honored MPE/iX engineer looked over the software programs and said they were extraordinary.

The SPL is some of the better looking old-style SPL that I've seen. The FORTRAN is very nice looking.

The SPL program runs in the batch job. The FORTRAN program maintains a configuration file that is presumably used by the SPL program.

So let the jobs of MPE/iX, including those running on standard HP hardware as well as those on virtualized 3000s, be managed a bit more easily. I await your comments on rousing the talents of this classic tool.

Posted by Ron Seybold at 10:06 PM in Hidden Value, Homesteading | Permalink | Comments (0)

Get e-mail notice when the NewsWire blog gets a new entry. Just say "Blog Me" in a message to editor@3000newswire.com.

June 01, 2016

Recovering Your Lost Logins

BWManIn the world of 2016, losing a password can be a time-bending experience. Apple's ID logins for its iCloud services, iTunes, and App Store only bear five attempts at most before they close out access to the phone. From there, you're on to a lengthy call to Apple. The smartphone is a tiny computer. The larger ones are not as strident about refusing repeated attempts, including the HP 3000.

They are secure, though. In these times when the servers can go dark for awhile as cold-start archive systems, though, a login can get misplaced. We're not supposed to write these things down, after all. What do you do when you have a question like the one below?

I restarted a 3000 9x7 after a few years in mothballs to run an old in-house app. I was able to boot up and login as OPERATOR.SYS, but cannot remember or find the password for MANAGER.SYS. Is there anyway to reset, clear, or overwrite the password file? I know the old machine is a very secure one, but now I am hoping there is a way around it.

What follows are a couple of suggestions to get back into the manager's driver's seat. However, you'll need to at least have a ticket to ride in OPERATOR.SYS.

Gilles Schipper shares what he knows.

There's not an easy way around the security. But since you can log on as OPERATOR.SYS, you should be able to store off the system directory to tape, as follows:

:file t;dev=tape
:store command.pub;*t;directory

Now that you have the directory on tape, you should be able to move around with FCOPY (and its ;char;hex options) to find the password for MANAGER.SYS. And that's all assuming you haven't implemented the directory encryption feature available with HP's security product Security Monitor/iX.

There's one other way. It's a good thing, because the alternatives do not include a rescue call to HP. There's nothing like Apple's password recovery support left for the HP 3000, as the vendor has long ago left the field. However, a good support contract for an MPE/iX server can be just the thing, and we know where you can get one.

Posted by Ron Seybold at 05:11 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 18, 2016

Tape drive changer a powerhouse for MPE

Autochanging HensAutochanging tape drives used to be the stuff of science fiction among 3000 managers, but those days passed by before HP cut off making Classic 3000 MPE V systems. Just because an autochanger is a 3000 storage option does not make it automatic to program, however.

A question posed to the community by Ideal Computer Services Ryan Melander reached for help on programmatically controlling such autochangers -- to select a slot, and load the tape and come to ready. "I am trying to configure an old DDS3 auto-changer, one that I don't believe will unload and load the next tape," he said.

Gilles Schipper noted that the command ad ldevno id=hpc1557a path=?? Mode=autoreply configures the device, and to advance tape after use, employ the command (from Devctrl.mpexl.telesup) ldev eject=enable load=online 

DAT tapesDenys Beauchemin mentioned HP's pass-through SCSI driver as a tool to drive the device's robot. The software was built by HP's labs and labeled as "not for the faint of heart" by engineers, but can assert a programmatic control over autochangers. Some third party programs such as Orbit's Software Backup+/iX can also do this work.

If ever there was a theme song for an autochanger at work, it would be a tune called Powerhouse. Children of the Fifties and Sixties will know it as soon as they hear it, if they've ever watched a Warner Brothers cartoon.

Some programming ideas came from Beauchemin, the engineer who developed at HiComp for the HiBack 3000 solution. "For the next tape to be brought online automatically, I seem to remember there had to be a special setting with the dip switches."

As for being able to control the robot itself, you definitely need to have the [HP] SCSI pass-through driver configured and loaded, and then you need a program to actually issue the IOCTL calls to the robot with the properly formatted SCSI commands. There was such a program a long time ago from a vendor, but that's all gone now.

Perhaps the high-test flutes and heavy octane horns of Powerhouse -- used in Duck Dodgers and the 24th and a Half Century -- can be put up on the MP3 player while fitting the driver to MPE. ("Oh drat these computers -- they're so naughty and so complex," says Marvin the Martian in one installment. "I could just pinch them.")

From our archives we found this advice from the late Jack Connor of Abtech, pointing to similar complex answers about controlling DDS changers.

Typically, there's a second SCSI port/address assigned for the transport control which allows the selection of specific tape. For MPE, stacker mode is typically selected, which tells the drive to just mount the next tape in line when requested. I don't know if the DDS autoloaders have a network connection available like the C7145NA DLT autoloaders do; with that device's web interface you can reload any tape, bypass a bad tape, and so on.

Back in 2011, John Pitman checked in to report that a much simpler solution to his changer's control needs popped up. "On re-examining my code for HPDEVCONTROL, I found I had catered for 1- and 2-digit device numbers in the string passed, but I had configured the drive as dev 777. This produced a string dev number of 77, which doesn't exist as a tape drive. Once I fixed this, it works like a treat."

Posted by Ron Seybold at 09:16 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 06, 2016

Options for STORE You May Not Know

StorehouseSTORE is the default backup tool for every HP 3000, but this bedrock of backup has options which might exceed expectations for a subsystem utility. Gilles Schipper, the support guru of three-plus decades on the 3000, told us about a few STORE sweet spots.

To start, using the :MAXTAPEBUF option can cut a four-hour backup to three hours or less. Schipper says that increasing the buffer size default to 32K, from the usual 16K, speeds up the backup when STORE sees MAXTAPEBUF. "That's a pretty good payback for one option."

Backups don't need to be specified with an @.@.@ command to be complete. "People should really be using the forward slash," he says, "because it's easy to accidentally omit the Posix file structure if you're not careful constructing your fileset backup." The slash is so much better that a backup specified by HP's TurboStore will replace any @.@.@ operation with "./" Combining @.@.@ with exclusions can lead to omitting files which should have been in a backup.

Schipper says that including a directory on a backup is smart, but private volumes in use on a 3000 need more than :directory as an option.

"If you're using private volumes and the directory option, you'll only get the system volume directory — and you will not get the private volume directories," he says. The backup must explicitly specify the volumes through the ONVS= command, using the long name of the private volume.

The partdb option on a STORE command ensure that any 3000 databases which are incomplete will get backed up. Without partdb, if a root file "doesn't have its corresponding data sets, the root file won't get stored. It's silly, but partdb ensures they get stored." A privileged file can also look like a partial database, so partdb brings those files into a STORE backup.

A 3000 with HP's TurboStore, rather than just the default STORE, can take advantage of the :online command. "It will give you zero downtime, if you have that version from HP," Schipper says. But that begins to drift away from the no-cost STORE options available to any HP 3000 administrator or owner.

Posted by Ron Seybold at 05:47 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 04, 2016

CPR for a Non-Responsive Console

On my HP 3000, after a short power blip, the console is now non-responsive. I can connect to the system's GSP port and the session is connected, but nothing is displayed. Neither <ctrl> A or <ctrl> B works. I type away, but get no response. I can then connect via VT-MGR and take the console :console !hpldevin and I receive all the console messages.

So, the messages are being sent (since I see them on the VT connection), but neither the physical console or the GSP gets any console messages. What can I try?

Gilles Schipper says

I believe a START NORECOVERY reboot is in order here. Since <ctrl> A <ctrl> B do not work, you will need to power-recycle the machine to effect a reboot. Presumably you would want to do this after gracefully stopping all jobs and asking online users to log off, if possible.

Depending upon which patch level your level of MPE is on, the :SHUTDOWN RESTART MPE command may also work from a logged-on session with at least OP capability.

Mark Ranft adds

If you haven't rebooted, I've seen similar issues. From the VT console can you try to do 'abortio 20' until it says no I/O to abort. A WHILE loop may make this easier. I've had luck with this in the past. But since Ctrl-B doesn't work, you may be out of luck.

Robert Thwaites notes

These are the simplest things to try first

<ctrl>Q (x-on)

<return>

Among the commonest issues: forgetting to do an x-on after a <ctrl>S (x-off) to stop output, so you can look at the line you are interested in. One time I saw another issue where someone had pressed <space> on the console and hadn't pressed <return>.

Posted by Ron Seybold at 10:37 PM in Hidden Value, Homesteading, User Reports, Web Resources | Permalink | Comments (0)

April 22, 2016

How to Transform MPE Spoolfiles to PDFs

HP 3000 data becomes more useful if it can be e-mailed as industry standard report documents. After more than two decades of pushing at it, Adobe has made its PDF the de-facto way to exchange documents, even complex ones.

Which might have prompted this question from HP consultant and Suprtool trainer Jeff Kubler:

Does anyone have a lead on a tool that converts spoolfiles to PDF files? Are there any Contributed Library tools?

It won't be in the Interex Contributed Library (because the programs have gone underground; ask your colleagues if they have a swap tape) but the txt2pdf product works nicely to make this conversion. Even in its most advanced version it's about $1,000, at last glance. Bob McGregor reports as much.

Jeff, txt2pdf does this. We have a job that runs that:

1. Checks a pseudo device we have setup for any spoolfiles that are on the device with an PRI >0

2. If it finds a spoolfile, we convert it to PDF and move it to a server

3  Sends an e-mail to the streamedby variable telling them the PDF doc is ready on the server.

4. Alters the priority to 0 to mark it processed

We've been using it for a couple years, and it works great — of course, once we got the bugs worked out. What's cool is if someone delete the file, we just adjust the priority to something greater than 0 and it gets reprocessed.

Posted by Ron Seybold at 08:52 PM in Hidden Value, Migration | Permalink | Comments (0)

April 15, 2016

Use GSP and More for 3000 ID, control

GSP boardThere's been plenty of change in the 3000 manager's life over the last 15 years. Some of the change is a little easier by exploring the use of the Guardian Service Processor. It can help in changing the location of HP 3000 consoles from one part of a shop to another. There's even a story out there that says the identity of the HP A- and N-Class hardware resides in the GSP.

Kent Wallace, while a 3000 manager for Idaho-Oregon healthcare delivery system Primary Health, needed to move his 3000 console.

I must move the 3000 console another 10 feet farther from the rack (it's an N-Class HP 3000/N4000-100-22). What are the 3-pin positions on the wire that I need to extend its RS-232 cable?

Our blog contributing editor Gilles Schipper chipped in with a solution offering even farther movement:

If you want to extend the range of the console to anywhere on the planet (at least where there’s Internet access) you could consider the HP Secure Web Console to replace the physical console. Depending upon the condition of your physical console, this solution may also save a bit of wear and tear on your eyeballs, instead of adjusting those pins.

Schipper wrote us a great article on setting up such a web console. The GSP is quite the tool.

Former HP support engineer Lars Appel offered another take on Schipper's strategy:

While Gilles is right about the possibility of using the web console, it would probably be easier to use the already built-in dedicated LAN port of the N-Class systems that gives access to the GSP by telnet.

I prefer the “telnet console” over the “web console” because it gives more freedom in the choice of terminal emulator — whereas the web console typically lacks features like “easy cut and paste” or special key mappings (e.g. German language ;-) or something similar.

This prompted Schipper to clarify his suggestion:

Lars is absolutely right about the built-in “secure-web-console” that comes with all N-Class and all but the earliest A-Class e3000s.

And, yes, the built-in is definitely more functional, allowing cut-and-paste as well as telnet access, whereas the external variety has only Java access to it via a web browser and no cut-and-paste.

So, if one has a choice, the built-in is definitely superior and available with only proper configuration. However, the external secure web console is available for all HP 3000s, and would still be most useful where is internal secure web console is not an option.

The GSP is a powerful part of an HP 3000 that runs whenever the server is plugged in. It's the maintenance control console commanding the ultimate class of the server to reboot, do memory dumps and even fully power down the 3000. Consultant Craig Lalley has noted the GSP has one fewer feature than its Unix counterpart, though.

"On HP-UX it is possible to reset the GSP from the host OS," he said. "I have not found a way from MPE."

From time to time a reset may be required for diagnostics services on A-Class and N-Class servers. If your 3000 gets loving care from a consultant or service provider outside your computer room, you may need a paper clip to keep up service levels.

The GSP can also reveal the 3000's speedometer, as profiled near the bottom of an Allegro webpage.

The gap between controlling the 3000 and HP's HP-UX Integrity GSPs is a common shortfall of HP designs. Even though the 3000's MPE/iX includes a Posix interface, HP didn't engineer enough Unix into the 3000 to enable some of the administration that HP-UX users enjoy. (That lack of Unix can sometimes be a good thing, however, when a security breach opens up in the Unix world.)

But when a 3000 needs a GSP reset, pressing a recessed button on the 3000's back will do the trick if a telnet command doesn't work. You can telnet to the IP address of the GSP, log in and do the reset. But you can also get someone to press the physical reset button at the back of the machine. It's recessed into the cabinet so you may need a magic paper clip bent just-so.

Lalley calls the GSP, which HP introduced with its final generation of 3000s, one of the most useful things in the A-Class and N-Class boxes.

The GSP is a small computer that is always powered on when the plug has power. With it, it is possible to telnet to and BE the console. While multiple admins can telnet in and watch, only one has the keyboard.

It is possible to reboot, memory dumps and even fully power down the HP 3000 from the GSP. Use the command PC OFF.

It is probably the best feature of the N-Class and A-class boxes. The problem is sometimes it needs to be reset, usually with a paper clip. Since the GSP is a different CPU, this reset can be done during business hours.

 

Posted by Ron Seybold at 08:35 PM in Hidden Value, Homesteading | Permalink | Comments (0)

March 18, 2016

Big files get zipped, moved on HP 3000s

ZipperA computer manager who's new to the HP 3000 is looking for CSL files this week. The Contributed Software Library is just an oasis to this IT veteran, something shimmering in his future that holds a highly useful thicket of utilities and more.

Someone in the 3000 community is bound to connect our new user with this CSL, for one reason: he's looking for MPE V programs to supplement his discovery of the emulated Classic HP 3000, simh. That's the MPE V-ready version of a virtual HP 3000: what amounts to a CISC skin for a 3000 on top of the simh code. Whenever the newbie connects with a CSL resource, if they've got their files on a 3000 they're bound to need to send about 24MB to him. That's going to require zipping them.

The act of zipping to compress for a transfer is an essential in 3000 management. Although the code for compressing files on HP 3000s is more than a decade old, like a lot of things on the system, it continues to work as expected.

Tracy Johnson, who manages the Invent3K server operated by OpenMPE, noted he's using the MPE/iX Posix shell's compress and uncompress. "It creates a file that ends in capital Z. Seems the compressed format is compatible with both GNU-zip and Winzip programs or any other Unix/Linux machine."

Lars Appel, who ported the Samba file sharing tool to MPE, offers a comprehensive answer. He points to an HP 3000 Web starter software kit that resides on a development server, open to the public.

Lars explained in a post a few years ago:

You can pick up the InfoIP zip/unzip programs (in a tar file) at my area in the Infocorp server. The link in that webpage that contains the zip/unzip programs is

E:\WebKit2\upload\infozip.tar.Z

Transfer it to the 3000 in bytestream or (fixed) binary format and then unpack with :/bin/tar "-xvzopf FILENAME". Place the two programs where you like; I typically have them in /usr/local/bin or (with uppercase filename) in a group or directory that is part of my HPPATH settings.

The webpage also contains a tar.Z file with /usr/local/bin/gzip

E:\WebKit2\upload\gnuzip.tar.Z

(gzip -d decompresses; creating a symbolic link gunzip is also useful)

Posted by Ron Seybold at 06:04 PM in Hidden Value, Homesteading | Permalink | Comments (1)

March 04, 2016

3000-L breaks silence with DTC primer

DTC NetworkOn the verge of four weeks without a new message, the 3000-L mailing list and newsgroup delivered a primer on the control of DTCs on MPE/iX networks. A longtime contributor to the community set up a question about using these venerable devices that connected HP's terminals and other devices to a 3000.

I have multiple HP 3000s sharing one DTC.  My problem is, which one controls the DTC?  In the event of  a power cycle, there is a race between them for which 3000 will download the new configuration.  I need system A, my A-Class, to be in charge. However, systems B or C are most likely to download first, leaving me with the manual step of unplugging the network from B and C, power-cycling the DTC, and then waiting for the A-Class to download the configuration. Is there anything that can be done to just leave the A-Class in charge?

Tracy Johnson replied, "It's always been a crap shoot. I'm of the opinion the first HP 3000 to notice the DTC needs downloading will do the job. Which usually means the less busy machine." As it was just the first answer on the newsgroup, there was still a need to do unplugging.

"For example," he added, "if you have a test machine on the side nobody's using, I would simply unplug the test machine's network cable until the DTC was reset. Since there will be no other machine in control, the regular machine will do the download.  When complete, I'd plug the network cable back into the test machine. However if both machines are in production and you have users active on both, you'll have to make a decision."

Lalley came back with an answer of his own, at least to eliminate the travel from chair to server.

"On the DTC's I guess it could be possible to do it without leaving your chair, if you TELNET to the console on B and CDTCCNTRL (has to be run from the physical console)  to stop the DTC subsystem, then restart it."

Mark Ranft at Pro3k had another idea.

Back in the good old days, we used DTC Switching - a feature that allowed a DTC prompt to appear at a terminal.  The user entered 'C HPA' to connect to host HPA.  I believe you could set a default system.  Initially DTC Switching was only available if you configured and downloaded the DTC using OpenView DTC Manager software running on a PC.  Later HP set up NMMGR to allow DTC Switching.

The Communicator for that release may have more details, but I only entered the MAC of the DTC on the one system that was supposed to download the DTC.  The other systems would have the DTC name, but not the MAC.  This allowed the system to have printers and other devices accessible on the DTC without concerns about which one downloaded the configuration.

Of course, the location of that MPE Communicator took all of 41 minutes to dig up. Barry Lake pointed at HP documentation (now hosted outside HP's baffling website) that covers Enhanced Host-Based DTC Management Functionality in Chapter 3. Plus, a manual on Configuring and Managing Host-Based X.25 Links.

Not too shabby for a mailing list that's more than 20 years old, but still manned by dozens of experts. It's the kind of expertise a good third party support provider offers—as usual, better than HP's today.

Posted by Ron Seybold at 06:33 PM in Hidden Value, Homesteading | Permalink | Comments (0)

February 26, 2016

21 days of radio silence on the 3000-L

Right_WirelessTelegraphThe slowing current of 3000 communication showed a fresh signal by the end of this month. As we write it's been 21 days since a message of any kind on the 3000-L MPE newsgroup. The resource that carried 45 messages during last February has 10 for the current month. All of this month's traffic was wrapped around finding resources: Brian Edminster of Applied Technologies and Vesoft support. Both were located.

However, the three weeks without a new message is new territory for the community's log of technical help and outreach by cohorts. Among those who were posting during 2015, several told us they're on the mailing list-newsgroup out of habit — rather than needing details for their datacenter's 3000s.

"I’m still on the list out of inertia, nostalgia and mild interest," said Dave Heasman, a UK IT manager. "My employer got rid of their 3000s and me in 2008. Bought a series of packages to replace a big bespoke brokerage/investment system."

Robert Mills said he "remained a member of the list, mainly as a lurker, to keep appraised of what was happening in the 3000 community. Except for three requests in September 2012, December 2014, and February 2015, I've only posted to the list when I felt that the 3000 knowledge I had would help somebody solve a problem." Mills said he retired when his company went insolvent in 2009, but he's kept his hand in IT.

"I have been involved with the GnuCOBOL (formerly OpenCOBOL) Project on SourceForge since October 2014, and decided to write a macro preprocessor that emulated the functionality available on the 3000," he said. "The preprocessor, CobolMac, is now in its 5th version (B.04) and has received good reviews by its users."

Others who contacted us said they haven't worked on the 3000 since the days that HP sold support for MPE/iX. "I have been a BizTalk developer full time since 2008," said Kent Wallace. "I needed to work, and this was the direction the world was going." The 3000-L still has more than 500 subscribers on its mailing list rolls, but much of the messaging comes from consultants and vendor experts, supplying answers to questions and tips. A total of 45 messages have passed through the list since the start of 2016. The IT pros like Wallace have taken the path to other platforms, first to HP-UX, then to Windows.

"I left my previous employer in Boise and I moved to a Microsoft shop, whose mainframe was HP-UX," Wallace said. "However, in 2015 we migrated off HP-UX and onto SQL Server on Microsoft Server 2008. We do health insurance and the purchased software, Trizzeto, was moved to MS SQL servers."

Another registered user keeps up with the community, but he can imagine a future where he'd be back on the MPE/iX front lines. "We're totally out of the 3000 business," said Ted Johnson of Wake Forest University, adding a sad-face emoji. "But I love seeing the 3000-L posts and keeping up. Who knows — maybe they'll get rid of me one of these days, and I'll end up back on a 3000."

The 3000-L, hosted at the University of Tennessee at Chattanooga where the late Jeff Kell launched it the early 1990s, holds more than two decades of traffic. 10 years ago the list was big enough to measure a signal-to noise-ratio, but by now it's almost entirely signal. When John Burke was a monthly columnist for the NewsWire who summarized its content in net.digest, he rounded up the following help in just one month's communications. For a 3000 owner managing a homesteading shop, the 3000-L's tips still carry some value.

Quick Cuts

• Do you want to know when a particular account or group was created? LISTACCT and LISTGROUP are no help. But “listfile /ACCOUNTNAME,3” for the account or “listfile /ACCOUNTNAME /GROUPNAME,3” for the group tell all. And then some.

• The number of sectors reported by the REPORT command for a group or groups is sometimes inaccurate, sometimes very inaccurate. Running the program FSCHECK.MPEXL.TELESUP and issuing the SYNCACCOUNTING command will fix this problem.

• In case you were wondering, despite many requests for the enhancement, TurboStore will NOT append store sets to tape. Well, it might if you use the proper incantations, but it is unsupported and highly dangerous because under certain circumstances you could overwrite a previous backup without knowing.

• Speaking of things you cannot do that you might like to do, the ALLOW command is not persistent across sign-ons unless you use the extremely dangerous “ALLOW @.@; commands” version. This is another example of an enhancement that has been requested for years, but now will never happen. Fortunately, there are a number of options, for sale and free (MPEX, CSL, etc.).

• CI integer variables are signed 32-bit entities. So be careful if you are doing some wild arithmetic in your CI scripts.

• Here is a little trick when using Apache’s indexing (for example to keep track of documentation) to index file displays. You can override the default ascending sort by name by appending “?N=D” to the url. Instructions on changing Apache’s default behavior are available on the Web.

• If you are trying to program VPlus applications and are interested in working examples programmed in your favorite language, look in the group HP32209.HPPL89 (which should be on every FOS tape). This group contains source code for the ENTRY program in a variety of languages including COBOL, Fortran, Basic and Pascal.

• To see the firmware (aka PDC) Revision of a system (CPU): Run cstm, and at the cstm > prompt, type ‘map’ and note the Dev Num of a CPU and then type ‘sel dev DEV_NUM’ (e.g., ‘sel dev 41’) and then type ‘info’ and then type ‘il’ and look at the output for the ‘PDC Firmware Revision’. Easy, huh? Thanks to Guy Paul of HP for this tip.

• SPFXFER will allow you to write to disk (undocumented “feature/bug”). But don’t do it, because SPFXFER cannot read the disk file it creates! Doing this could lead to a big oops.

• While it would certainly be a nice to have, MPE/iX CI scripts have no provision for inline comments. Sorry, don’t even bother trying.

Posted by Ron Seybold at 05:01 PM in Hidden Value, Homesteading, Migration, User Reports | Permalink | Comments (1)

February 19, 2016

How hot plug disks can replace DDS offsite

300 GB Ultra SCSII need to find an alternative to DLT and DDS tapes for offsite storage. Sure, there's DS2100 and Jamaica drives. But a few $35 300GB Ultra SCSI drives would hold a lot more data with less points of failure. I'll set up a BACKUP_VOLUME_SET and use the internal disks to do store-to-disk backups of the system. 

I've always used my A-Class and N-Class systems with fiber-attached disk. Are the internal disk drives hot-pluggable? 

Jim Hawkins, IO maven for HP 3000 systems at HP, replies with details.

There are multiple layers of changes for actual hot plugs or swaps to work.  

  • You need the disk HDD to handle this electrically.
  • You need HDD physical carrier and physical interface to comply.
  • You need the system physical interface and receptacle to comply. 
  • You need your Host System Bus Adapter (HBA) to electrically support this.
  • You need the OS to be aware enough of the HBA to not get flustered by absence of the device and deal with any notifications from the HBA of the activity.   

Given that the N-Class disk cage has a screw-based cover and the HDD carriers have no quick release levers (as compared with HASS/Jamaica or VA7400) I would state definitively that there is no hot-plug intention.  

At the same time, the SCSI bus is pretty low power and low voltage, so it would be generally not-too-unsafe to experiment. But you're also close to AC inputs and they are not low power.

Hawkins explored the not-too-unsafe scenario with theoretical possibilities.

Might you be able to pull/push a drive where you've closed the volume?  Likely it would work, but there may be all kinds of noise and stress on the SCSI bus which may not be well handled. However, I think each disk is on its own HBA channel which isn't shared with anything else, and so unlikely to abort someone else's IO.  

This takes us to the last issue: mechanical wear.  

These connectors were likely intended for more or less permanent mating of two components. Very likely they have a limited number of cycles that they are specified to hold-up. I've seen connectors that are specified for fewer than 25 cycles before you lose gold contact material.   This is okay for normal HDD where one might replace one or two per slot in a system lifetime, but not sufficient if you're doing nightly back-ups and swaps.  Connectors, where there is an expectation of a high number of pull/replace cycles, have special designs.  

Now a little good news here is that the N-Class was still pretty much old-school HP design, so likely they didn't pick up something cheap that saved them .2 cents per unit on gold plating. No idea though if the HDD connector is a 10-, 100-, 1000-cycle part. Your system, your risk. 

Consultant Mark Ranft points out that the HP design for 3000s seems to make the servers and components good candidates for exceptional mechanical wear tolerances.

It is especially helpful to understand the concept of mechanical wear on the connectors. HP always had excellent and innovative hardware engineering on their HP 3000 servers. Remember, you can drop them off a building and still self-test them

The Unix N-Class appears to allow hot-pluggable drives.   

The actual power supply and the fans are in the front of the N-Class.  The power receptacles in the back have internal cords that lead to the front.

Posted by Ron Seybold at 08:59 PM in Hidden Value, Homesteading | Permalink | Comments (0)

February 12, 2016

How to get specific about IP access for PCs

I want to give a 3000 a static IP, so I can permit a user to access the HP 3000 from that PC with that static IP. Is there a way to force a particular user ID to use a specific IP address?

Tracy Johnson replies:

A simple logon UDC should suffice:

IF HPREMIPADDR = "aaa.bbb.ccc.ddd" then
  ECHO Welcome.
ELSE
  ECHO Evil message here.
BYE
ENDIF

Bob Schlosser adds:

You can set up a logon UDC that checks that the var HPLOCIPADDR is equal to the device (PC) that you want them to use. Something like this:

LOGON
OPTION NOBREAK,LOGON
IF "!HPLOCIPADDR" <> "123.456.789.321"        change "123.456.789.321" to
your IP address
  BYE
ENDIF

Using this, we verify that the user is on the correct (assigned) IP address, and log them off if not.

Chris Bartram, who's created e-mail solutions for the 3000 at 3K Associates, and hosted Web servers since early in the 1990s, adds:

The following is an excerpt from system UDCs I use on my HP 3000s that might give you some ideas.

The "VALIDATEIPADDR" call in this UDC calls another command file that actually does a validation of the logging-on user based on data in a control file to determine if he/she is allowed to log onto the system from the specific host/IP address they are coming from.

The variables the UDC sets will work whether the logging on user is coming in via Telnet or NSVT (or hardwired or via a modem).

The TELLOPs also leave a nice log on the system console (and log file) of the login, including where they came from, and what protocol was used to access the system.

***
LOGON
OPTION LOGON,NOBREAK,NOHELP

setvar _network_node ''
if bound(hpstdin_network_node) then
  setvar _network_node '!hpstdin_network_node'
endif

setvar _na ''
setvar _at 'HARDWIRED'
if bound(hpstdin_network_addr) then
  setvar _na '!hpstdin_network_addr'
elseif bound(hpremipaddr) then
  setvar _na '!hpremipaddr'
endif

if bound(hplocport) then
  if !hplocport=23 then
    setvar _at 'TELNET'
  endif
endif
  IF BOUND(HPSTDIN_ACCESS_TYPE) THEN
    SETVAR _AT "!HPSTDIN_ACCESS_TYPE"
  ENDIF

IF BOUND(HPSTDIN_TRANSPORT_TYPE) THEN
  SETVAR _TP "!HPSTDIN_TRANSPORT_TYPE"
ELSE
  IF "!_AT"="TELNET" THEN
    SETVAR _TP "TCP/IP"
   ELSE
    SETVAR _TP "SERIAL"
  ENDIF
ENDIF

IF BOUND(HPVT_CLIENT_VENDOR) THEN
  SETVAR _VND " (!HPVT_CLIENT_VENDOR)"
ELSE
  SETVAR _VND " "
ENDIF

TELLOP LOGON VIA !_AT USING !_TP !_VND

setvar _node ups(ltrim(rtrim("!_network_node")))
setvar _addr ups(ltrim(rtrim("!_na")))
if '!_node'<>'' then
  tellop !_at, IP: "!_addr" Node: "!_node"
else
  tellop !_at, IP: "!_addr"
endif

setjcw cierror=0
continue
VALIDATEIPADDR
if !cierror<>0 then
  echo
  echo ************************************
  echo **  NODE/IP CONTROL FILE CORRUPT  **
  echo ************************************
  echo
  bye
endif

Posted by Ron Seybold at 03:56 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 18, 2016

The GSP makes the A and N worthwhile

GSP boardIt's a powerful part of an HP 3000 that runs whenever the server is plugged in. The Guardian Service Processor (GSP) is the maintenance control console commanding the ultimate class of the server to reboot, do memory dumps and even fully power down the 3000. Consultant Craig Lalley of EchoTech has noted the GSP has one fewer feature than its Unix counterpart, though.

"On HP-UX it is possible to reset the GSP from the host OS," he said. "I have not found a way from MPE."

From time to time a reset may be required for diagnostics services on A-Class and N-Class servers. If your 3000 gets loving care from a consultant or service provider outside your computer room, you may need a paper clip to keep up service levels.

The GSP can also reveal the 3000's speedometer, as profiled near the bottom of a webpage from Allegro Consultants.

The gap between 3000 and HP's HP-UX Integrity GSPs is a common shortfall of HP designs. Even though the 3000's MPE/iX includes a Posix interface, HP didn't engineer enough Unix into the 3000 to enable some administration that HP-UX users enjoy. (That lack of Unix can sometimes be a good thing when a security breach opens up in the Unix world.)

But when a 3000 needs a GSP reset, pressing a recessed button on the 3000's back will do the trick if a telnet command doesn't work. You can telnet to the IP address of the GSP, log in and do the reset. But you can also get someone to press the physical reset button at the back of the machine. It's recessed into the cabinet so you may need a magic paper clip bent just so.

Lalley calls the GSP, which HP introduced with its final generation of 3000s, one of the most useful things in the A-Class and N-Class boxes.

The GSP is a small computer that is always powered on when the plug has power. With it, it is possible to telnet to and BE the console. While multiple admins can telnet in and watch, only one has the keyboard.

It is possible to reboot, memory dumps and even fully power down the HP 3000 from the GSP. Use the command PC OFF.

It is probably the best feature of the N-Class and A-class boxes. The problem is sometimes it needs to be reset, usually with a paper clip. Since the GSP is a different CPU, this reset can be done during business hours.

 

Posted by Ron Seybold at 10:43 AM in Hidden Value, Homesteading | Permalink | Comments (0)

January 13, 2016

Using Store-To-Disk for Backup Preservation

By Brian Edminster

Second of two parts

Yesterday I outlined some of the powers of the Posix program pax, as well as tar, to move MPE/iX backup files offsite. Here’s a warning. There are some file types that cannot be backed up by tar/pax while also storing their attributes:  ;CIR (circular) and ;MSG (message) files (and possibly others. I haven’t tested all possible file types yet.  Also, there is an issue with tar that is a fairly well known and has been discussed on the 3000 newsgroup. Occasionally it does not un-tar correctly.  It is unclear if and when this was fixed, but I’d love to hear from anybody that might be in the know, or which specific situations to avoid.

Regardless of these limitations, I’ve found a simple way around this. Use store-to-disk to make your backup, then tar to wrap it, so as to preserve the store-to-disk files’ characteristics, before shipping the files off-system. Later, when you retrieve your tar backups and un-tar them, you’ll get your original store-to-disk files back without having to specify the proper ‘;REC= , CODE= , and DISC=’ options on an FTP ‘GET’. I’ve been doing this for several months now on several systems, and I have not had any failures.

If you have a version of STORE that has compression, use it to reduce the size of backup.  If not, use the ‘z’ option in the tar/pax archive you create from your store-to-disk backup.  Do not use both.  They don’t play well together, and you may end up with a larger tar file.

But what about the tar archive size limit of 2GB?  There’s an easy way around this as well, as this limit is common on early Unix and Linux systems. Just pipe the output through ‘split’ to create chunks of whatever size you want. Below, there's simple examples for both directions.

Piping TarFigure 1, just below, is an example of the ‘cksum’ file produced.

Checksum 1 Grey

Below, Figure 2 is an example of a ‘cksum’ created of the files as they’re stored on the NAS. 

Checksum 2 GreyAs both the hashes and #bytes shown in each file are the same as on the MPE/iX server — we know the backups are transferred correctly.  The same technique can be used ‘in reverse’ to verify that when FTP’d back to the FTP server, they’re still intact.

When un-taring this backup, ‘cat’ the pieces together and pipe it through tar.   At least, that’s the way it’s supposed to work.  Yes, there is a known issue with the MPE/iX Posix shell’s built-in cat command. But I’ve so far been unable to successfully use the external cat command to successfully cat either.  Here’s how this should work for a 2-chunk tar backup:

sh>/bin/cat ./CS1STD1.ustar.aa ./CS1STD1.ustar.ab | tar -xfv - *

Unfortunately, for me at least, it always throws an error indicating bad format for the tar files.There is a work-around, however.  Note that while ‘cat’ing the tar ‘chunks’ didn’t work using the internal or external cat command, untar with multi-file option does work.  Even though it gives a minor error messages, files were returned to proper store-to-disk format, and the recovered store-to-disk backup is intact and has been used to recover the desired files. To do this, use tar like this: 

sh>tar -xfv ./CS1STD1.ustar.aa *  

Also note that when using tar in this way, it will ask for the name of the 2nd-nth component tar files, as it finishes reading each prior piece.  You must give the filename and press return to continue for each.  I believe that it should be possible to script this so that it’s fed the filenames, but I haven’t gotten around to doing that yet.  

Brian Edminster is president of Applied Technologies, a 3000 consultancy serving MPE/iX sites in contract and ongoing engagements.

Posted by Ron Seybold at 05:44 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 12, 2016

Backing Up Your 3000 Backup Files

By Brian Edminster
Applied Technologies 

Once store-to-disk backups on the 3000 are regularly being processed, it’s highly desirable to move them offsite — for the same reasons that it’s desirable to rotate tape media to offsite storage. You want to protect against site-wide catastrophic failures. It could be something as simple as fire, flood, or a disgruntled employee, or as unusual as earthquake or act of war.

Regardless of the most pressing reason, it really is important to keep at least some of your backups offsite, so as to facilitate rebuilding / recovering from scratch, either at your own facility, or at a backup/recovery site.

The problem comes in that the MPE/iX file system is far more structured than Unix, Windows, or any other non-MPE/iX file system-based storage mechanisms. While transferring a file off MPE/iX is easy via FTP, sftp/scp, or rsync, retrieving it is problematic, at least if you wish the retrieved files and the original store-to-disk files to be identical (i.e., with the same file characteristics: filecode, recsize, blockfactor, type, and so forth).

What would be optimal is automatic preservation of these attributes, so that a file could be moved to any offsite storage that could communicate with the MPE/iX system. Posix on MPE/iX comes to the rescue.

For FTP transfers between late-model MPE/iX systems this retrieval is automatic, because the FTP client and server recognize themselves as MPE/iX systems.  For retrieving files from other systems, HP has made that somewhat easier by making its FTP client able to specify ‘;REC= , CODE= , and ;DISC=’ on a ‘GET’:

Figure 1If you do not specify the ‘buildparms’ for a file being retrieved, it will default to the file-type implied by the FTP transfer mode: ASCII (the default), binary, or byte-stream (often called ‘tenex’ on Unix systems).  The respective defaults used are shown below:

Figure 2 GreyWhat follows is an example of automatic preservation of these attributes, so that a file could be moved to any offsite storage that could communicate with the MPE/iX system.  And this is yet again where Posix comes to the rescue, via the venerable ‘tar’ (Tape ARchiver), or ‘pax’ archiving utilities.

‘pax’ is a newer backup tool, designed to be able to read/write with tar format archives, newer ‘ustar’ format (that includes Extended Attributes of files). At the same time it has a more ‘normal/consistent’ command syntax (as Unix/Posix stuff goes, anyway), plus a number of other improvements. Think of it as tar’s younger (and supposedly more handsome) brother.

A little known feature of most ‘late-model’ tar and all pax commands is the ability for it to recognize and utilize Extended Attributes.  These will vary with the target implementation platform, but for the tar and pax commands included with releases after v5.5 of MPE/iX this capability is not only present — but contrary to the man command’s output and HP’s Posix Command Line manual, it’s the default! You use the -A switch to turn it off, returning tar to a bytestream-only tool.

While not externally documented, via a little experimentation I’ve determined that the following series of Extended Attributes value-pairs are in the MPE/iX Posix implementation of a tar or pax ‘file header’ for each non-Posix file archived:

MPE.RECORDSIZE= value in bytes
MPE.BLOCKFACTOR= integer value
MPE.RECORDFORMAT= integer value (0=unstructured?)
MPE.CCTL= integer value (0=nocctl)
MPE.ASCII= integer value (0=binary, 1=ascii)
MPE.FILECODE= integer value, absent for ‘0’
MPE.FILELIMIT= value in bytes
MPE.NUMEXTENTS= integer value, may be absent
MPE.NUMUSERLABELS= integer value (0=no user labels), and
MPE.USERLABELS=[binary content of user labels]

Brian Edminster is president of Applied Technologies, a 3000 consultancy serving MPE/iX sites in contract and ongoing engagements.

Posted by Ron Seybold at 02:38 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 05, 2016

Migrating 3000 Data from Spoolfiles to Excel

I need assistance with putting an output spool file from MPE/iX 7.5 into Excel or other readable format. The file is generated by Query, then processed by Editor, then sent to the printer. Instead of printing it, I want to put it into a readable format.

MigratingI do not have QEdit or any smart tools on MPE, so my approach thus far has been to move the file to a PC before doing anything.  However, that carries with it the initialization sequence for the printer to which the job is spooled. The job is set up to print on a PCL 5 laser, which means it has hundreds of lines of control before the data starts.

Tom Moore replies

I would put commas in between my columns (in the query, or using Editor). I FCOPY from the file to a new file with NOCCTL to get rid of carriage control byte. You could also remove the PCL 5 lines by subset in the FCOPY command. Depending on the data, I would use EDIT3000 to change all " ," to "," and all ", ","," to compress the file, removing the spaces before and after the commas inserted above, then save the file for download to the PC.

I would also consider using ODBC to directly extract from the IMAGE database, rather than Query and all the subsequent steps. The HP free ODBC driver would do the job very well.

Birket Foster of MB Foster notes

Not only did we make that free ODBCLink/SE as HP's lab resource from 1998 to 2006, but we have continued to develop the ability to work with data in all kinds of file formats. We do supply 32- and 64-bit versions for ODBC to the HP 3000.

UDALink-MPE was designed for the HP 3000. We provide data in several different formats including XLS for Excel, XML, CSV etc. We can have a discussion about what you are trying to do with data; perhaps UDACentral is the right product for your challenge and we can organize a demonstration for you.

Charles Finley adds

There seem to be at least three steps to what you are trying to do.

  • Remove the headers, footers and perhaps page numbers from the report.
  • Remove the ff or CNTL characters from the text file.
  • Import a space-delimited file to Excel.

There are any number of different scripting tools that can do this including various Unix tools. Here's a reference to an Excel solution that might get you started. In fact, if it were my problem to solve, I would likely do it all with Excel scripting.

John Hohn replies

  1. Output to a delimited file (tabs, pipes, etc).
  2. Download to your laptop or PC or wherever Excel is running
  3. Start macro recording in Excel
  4. Import/format the delimited file, save as .xls
  5. Turn recording off, save macro

Set the Excel file to auto-execute the macro every time the Excel file it's opened, i.e., re-input/format the delimited file. Then you can, for example, schedule delivery of a new version of this delimited file whenever you'd like, to your server. When people open it they would automatically get the formatted version of the new data.

Connie Sellitto of Hillary Software suggests

Hillary Software has a product, byREQUEST, which does just this.

It has the ability to suppress headings on pages after the first, and define the type of data in the columns (text, numeric, dates in various formats). It can remove blank pages and leading and trailing blank lines. It can even call an Excel macro to make the headings a different font, background color, etc — anything you'd want to do with a macro. In addition to Excel, byREQUEST can create a PDF file, Word, csv or Text.

Posted by Ron Seybold at 10:25 PM in Hidden Value, Homesteading, User Reports | Permalink | Comments (0)

December 22, 2015

Studying the Scripts for HP 3000s

A recent question on the 3000-L mailing list and newsgroup asked for help on scripting. The question aimed at the automation prospects through terminal emulation programs. Does Minisoft/92 script, a manager asked. Of course, and Tracy Johnson replied to give details as well as an example.

Minisoft scripts are plain text files with a file extension of .s92 and can be assigned to function keys f1 through f12. We use the keyboard mapping config menu to map it to Type "Script".  Once you choose "Script" a blank box appears below where you put the magic words DO SCRIPT followed by a path including the file name.

Joe FridayLast week, the use of scripts also surfaced while talking to Birket Foster about data migrations. A client of Fosters runs five scripts to clean up phone numbers during a transfer of data. Being able to reach for the scripts improved the quality of data, and the work was automatic. the power of scripting reminded me of a fine column written for us by Ken Robertson. Its subject was an introduction for Unix administrators to the use of shell scripts. But the writing was the kind of operational lore that can make a 3000 look more powerful to an admin new to the 3000.  Robertson wrote about it for the Newswire.

The marvels of scripting lie deep in the roots of MPE. When HP expanded the OS to MPE/XL in the 1990s, it added the Posix shell, which extended the 3000's scripting potential. The MPE/iX command interpreter has a generous command set, pushing the shell into the realm of a true programming tool. Its ability to evaluate expressions and to perform IO on files allows the end-user to perform simple data-processing functions. The Command Interpreter can be used to solve complex problems. Its code, however, is interpreted, which may cause a CI solution to execute too slowly for practical purposes.

For the average task, the MPE scripting language is easier to read and understand than most Unix scripts. For example, command line parameters in MPE have names, just like in regular programming languages. Of course, there are several script languages on Unix and only one on MPE. On Unix you can write shell scripts for any of the many shells provided (C shell, Bourne shell, ksh, bash, etc). Although there is also a Posix shell on MPE, most scripts are written for the CI. 

A command file can be as simple as a single command, such as a Showjob command with the option to only show interactive sessions (and ignore batch jobs):

:qedit
/add
1      showjob job=@s
2      //
/keep ss
/e
:

You have created a command file called SS — when you type SS you will execute showjob job=@s

On MPE, the user needs read (r) or execute access (x) to SS. On Unix you normally must have x access, not just r access, so you do a chmod +x on the script. This is not necessary in MPE, although, if don’t want users to be see the script, you may remove read access and enable execute access.

Structure of a Command File (aka CI script)

A script is an ASCII file with maximum 511 byte records. Unlike Unix, the records may contain an ASCII sequence number in the last 8 columns of each line. The command file consists of 3 optional parts:

1. Parameter line with a maximum of 255 arguments:
parm sessionnumber
parm filename, length=”80”

2. Option lines:
option nohelp,nobreak
option list

3. The body (i.e., the actual commands)”
showjob job=!sessionnumber
build !filename;rec=-!length,,ascii
In MPE scripts, there is no inline data, unlike Unix ‘hereis’ files.

Parameters

Notice in the example above that parameters are used with an exclamation (!), as opposed to the $ in Unix. The same is true for variables. Parameters are separated by a space, comma or semicolon. All parameter values are un-typed, regardless of quoting.

In a typical Unix script, the parameters are referenced by position only ($1, $2, $3, …). In an MPE script, the parameters have names, as in the function of a regular programming language, and can also have default values. In Unix you use $@ for all of the parameters as a single string; in MPE you use an ANYPARM parameter to reference the remainder of the command line (it must be the last parameter).

Here is a script to translate “subsys” and “err” numbers from MPE intrinsics into error messages. The subsys and error numbers are passed in as parameters:

parm p_subsys=108,p_error=63
setvar subsys hex(!p_subsys)
setvar error hex(!p_error)
comment the hex conversion allows for negative numbers
comment the #32765 is magic according to Stan!
setvar cmd “wl errmsg(#32765,!subsys);wl errmsg(!error,!subsys);exit”
debug !cmd

As you can see above, the Setvar command assigns a value to parameter or to a new variable. But there are also system pre-defined variables. To see them all do Showvar @;hp. To get information on variables, do help variable and to get help on a specific variable, say hpcmdtrace, do help hpcmdtrace (set TRUE for some debugging help).
In most MPE commands, you must use an explicit exclam ! to identify a variable: build !filename

However, some MPE commands expect variables, and thus do not require the explicit !. For example, Setvar, If, ElseIf, Calc, While, and for all function arguments, and inside ![expressions].

Warning: variables are “session global” in MPE. This means that if a child process, or scripts, changes a variable, it remains changed when that child process terminates. In Unix you are used to the idea that the child can do whatever it likes with its copy of the variables and not worry about any external consequences.

Of course having global variables also means that it is much easier to pass back results from a script! And this is quite common in MPE scripts.

Options

Options allow you to list the commands as they are execute (option list), disable the Break key (option nobreak), enable recursion (option recursion), and disable help about the script (option nohelp).

The script body below shows active process information. This example shows many of the commands commonly used in scripts: If, While, Pause, Setvar, Input and Run. Other commands you will see are Echo, Deletevar, Showvar, Errclear.

WHILE HPCONNSECS > 0
    IF FINFO("SQMSG",0)
       PURGE SQMSG,TEMP
    ENDIF
    BUILD SQMSG;REC=-79,,F,ASCII;TEMP;MSG
    FILE SQMSG=SQMSG,OLDTEMP
    SHOWQ;ACTIVE >*SQMSG
    SETVAR PINLIST ""
    WHILE FINFO("SQMSG",19) <> 0
         INPUT SQLINE < SQMSG
         IF POS("#",SQLINE) <> 0 THEN
           SETVAR PIN RTRIM(STR(SQLINE,47,5))
           SETVAR PINLIST "!PINLIST" + "," + "!PIN"
         ENDIF
    ENDWHILE
    IF FINFO("SPMSG",0)
       PURGE SPMSG,TEMP
    ENDIF
    BUILD SPMSG;REC=-79,,F,ASCII;TEMP;MSG
    FILE SPMSG=SPMSG,OLDTEMP
    SETVAR PROC "SHOWPROC PIN="+"!PINLIST"+";SYSTEM >*SPMSG"
    !PROC
    WHILE FINFO("SPMSG",19) <> 0
         INPUT SPLINE < SPMSG
         IF POS(":",SPLINE) <> 0 THEN
           ECHO !SPLINE
         ENDIF
    ENDWHILE
    PAUSE 30
ENDWHILE


Handling Errors

In most Unix scripts, if a step fails, you check for an error with an If-conditional and then take some action, one of which is ending the script. Without an If, the script continues on, ignoring the error.

In MPE, the default action when a step fails is to abort the script and pass back an error. To override this default, you insert a Continue command before the step that may fail. You then add If logic after the step to print an error message and perhaps Return (back 1 level) or Escape (all the way back to the CI).

     continue
      build newdata
      if cierror<>100 then
         print "unable to build newdata file"
         print !hpcierrmsg
         return
      else
         comment - duplicate file, okay
      endif

You can set HPAUTOCONT to TRUE to continue automatically in case of errors, but this can be dangerous. The default behavior at least lets you know if an unexpected problem occurs.

Posted by Ron Seybold at 07:42 PM in Hidden Value, Homesteading | Permalink | Comments (0)

December 07, 2015

On MPE Chatting, B-Tree Plants and More

How we can chat on the HP 3000 system with the other users who have logged on?

Lars Appel replies:

You can try the TELL and TELLOP commands. For more information see :HELP TELL or :HELP TELLOP.

When I run dbutil.pub.sys, and type the command set MYDB btreemode1=on I get the message “Database root file must be at least “C”4 for SET <db> BTREEMODE1=O." Why can’t I set my btree mode?

Rene Woc replies:

Before you can “set btreemode1,” the database has to have Btrees. You add Btrees with “addindex MYDB for all (or specific master datasets)”. This command will also set root level to C4. To use “addindex” your system needs to be at least on TurboIMAGE version C.07.xx. So how do you find out what version of IMAGE you have? Use the version command in QUERY.

I need to take some groups off of the mirrored drives, and add (move) other groups onto the mirrored drives. Is it as simple to use the altgroup command and specify the volume set?

[Editor’s note: “mirrored drives” is a straw man that has nothing to do with the problem or answer.]

Craig Lalley and John Clogg reply:

It is simple but not that simple. What you need to do is create a temporary group on the target volume set. Copy files in the group you want to move to the temporary group. Delete the source group. Create the new group using

NEWGROUP xxx;homevs=volume_set
NEWGROUP xxx;onvs=volume_set

Note that it is a two-step command. Then rename the files from the temp group to the newgroup. John Clogg also noted that another approach would be to STORE the files, and restore them once the group was relocated. That way you could preserve creation and modification dates, and creator ID.

I have inherited a HP 3000, and I was trying to clean off an old TurboIMAGE database. I did a purge @.data and it purged about 100-plus files. However, it did not purge everything. They can’t be purged, even when logged in as manager.sys. The error that I get is CIERR 45, Privileged File access. I have tried to release it with no luck either. Any suggestions?

Steve Macsisak and Denys Beauchemin reply:

You can purge TurboIMAGE databases using the utility DBUTIL.PUB.SYS and the command PURGE (database name). The database name is the name of the root file, the one without numbers at the end. The command will purge all the datasets of the database. There are a few other ways to do this, but this is the most obvious.

[Editor's Note: Vladimir Volokh reports that if you have several databases, you must run this several times. Using MPEX, you can purge everything, so long as you are a manager of the account. All databases, all non-databases, everything will be purged with purge @.groupname. purge@.data (ispriv) will purge only databases in the group.]

I have two device numbers I don’t want to be in use and assigned to users on the network. LDEVs 5 and 6 are available for use and we don’t want them to be. Is there some place to configure these out of use?

Gilles Schipper replies:

Simply add these devices as network printer devices (id=hptcpjd path=none) in sysgen. This will prevent them from being used by any logon session.

Posted by Ron Seybold at 08:33 PM in Hidden Value, Homesteading | Permalink | Comments (0)

November 24, 2015

The Wide World of Connecting Storage

IO used to be more complex for IT. Sure, the array of choices for disk is vast today. But in the era when 3000s used to think they were lucky to get SCSI plugged into them, configuring disk connections was not simple. HP-IB protocol, built to link HP's instruments, was simple, used for all HP devices, and slow. But it was integrated and seamless compared to the SCSI of single-ended, fast/wide, and Ultra Fast.

Such was the case for one 3000 manager seeking advice from his colleagues. You never think about these things on a 3000 until the hardware breaks. Or backups fail. Or storage media gets rare. Aging hardware is one of several issues that require expertise, even if a 3000 runs the ultimate 7.5 version of MPE/iX. Our manager hunted for his help on the longest-running 3000 classroom in the world, the HP3000-L mailing list.

A single-CPU A-Class was moving away from DDS technology, the DDS-3 that was first launched in the '90s. There are other options for 3000 tape backup. But these options include single-ended, fast/wide, and other cable and termination combinations. DLT technology, introduced more recently but still a 1990s choice, runs with HP 3000s. It helps to get the ends right, though, if DLT is to have a new beginning on an old-school 3000.

"Until now they have done their backup on DDS," a manager talking to the 3000 newsgroup explained. "Lately they had a failure on the DDS drive, and have realized that it is getting difficult to get new tapes. They have decided to move to DLT8000, model C6378A, and have bought two of them. One is supposed to go live on the 3000, and the other to be stored as a spare device."

The DLT is hooked to the Ultra Wide SCSI interface on the A-Class. But ODE/Mapper doesn't recognize the device."

There was an error, and no DLT joy. Soon enough, one veteran consultant said, "You will have trouble connecting a fast wide SCSI device to an ultra-wide SCSI controller." It wasn't a rookie mistake, but the veterans who still prowl 3000-L had a solution and even a link to an inexpensive fix. So it goes, here in the fifth decade of HP 3000 mission-critical service. Answers are everywhere.

This wasn't an inexperienced 3000 pro, it seemed, when reading that he tried to "add the device in IOCONFIG by adding first the path 0/0/1/0.2, and then the device with the command: ad 8;path=0/0/1/0.2.0;ID=dlt;mode=autoreply."

SCSI on the 3000 sure isn't the world of USB, where just 2.0 and 3.0 cover the scope of IO choices. A $59 adapter card connected that DLT to the 3000. The IO challenge also prompted advice even a pro might not know — making a case for having fresher hardware than HP's to run MPE.

There was advice about using Mapper on the 3000 to troubleshoot an IO device from Michalis Melis.

Normally the path and the device should be recognized by running ODE/ Mapper without even loading the operating system. You do not have to go to SYSGEN. If Mapper fails you have a problem before the OS loads.

Craig Lalley made the link between two incompatible kinds of SCSI interfaces.

You are trying to hook up a Fast/Wide SCSI device to an Ultrawide interface. The C6378A can only connect to a HVD Fast Wide SCSI interface (A4800A SCSI card comes to mind). Remember, the A-class does not support Dual-Head cards, so your only option is A4800A. You need either a DLT8000 with a Fast-Wide interface, or you need a cheap A4800A HVD (High Voltage Differential) SCSI card. You can daisy chain devices to the card, but I would only use one tape at a time.

Lalley also tipped his hat to Keven Miller, who supplied the link to that $59 adapter card.

Then Denys Beauchemin, who has been among one of the more prolific contributors to the 3000-L, delivered detailed advice about connecting backup devices. His background reaches back to the first decade of 3000 use, including years spent with Hi-Comp on backup software development.

Fast/Wide SCSI (FWSCSI) is essentially HVD SCSI on SCSI-2 standard. This means that the signal is a differential in the voltage between various wires (HVD is High Voltage Differential) and Ultrawide SCSI is SE (Single Ended) SCSI, on the SCSI-2 standard which makes is wide (16 bits), like the FWSCSI.

So what is needed is a converter to power the signal from the Ultra Wide SCSI interface on your server to the FWSCSI interface on the DLT device. I have a number of those somewhere here, but they were for SE SCSI, not UltraSCSI.  They might work for that, since all they did was provide the powered signal and the cable is the one that converted from wide to narrow.

Another thing to consider is that since HP nicely crippled the A-class, that 3000 system would not be able to keep the DLT8000 streaming. And that device hates not streaming, so much so that it will enter shoeshine mode and perform abysmally. Just a parting gift from HP to the MPE community. You should hear what they're doing to the VMS crowd.

That last comment comes from Beauchemin's current duties as migration manager for the OpenVMS users who are leaving that platform. VMS had a steady Internet community to help Digital users, just as the 3000 has 3000-L. People like Beauchemin, largely working outside the 3000 world, are still providing advice for homesteaders -- even while assisting in migrations. After migration there is much to manage, but simply migrating off Hewlett-Packard's 3000 hardware makes using MPE/iX less complex.

Posted by Ron Seybold at 08:07 PM in Hidden Value, Homesteading, User Reports | Permalink | Comments (0)

November 17, 2015

Putting PDF Into a 3000's Data Flow

PDF logoHP 3000 experts know of a wide array of techniques to create PDF files from the server's data, then move them via FTP to a Windows server. While the simplest answer for getting reports into PDF format and then out via Windows is probably Hillary Software's byRequest, there are other commercial solutions. There's also several bolt-together techniques if you've got very limited budget to homestead.

Bob McGregor reports:

We use txt2pdfPRO by Sanface. We had a job that would run and check a pseudo device for spoolfile output, and if the pri > 0, would run the sf2html process, convert to PDF and then FTP to a Windows server. The process would then delete spoolfiles=0 on the pseudo device the next day. Setup took a bit... but once done, worked well.

Lars Appel, author of the Samba/iX file sharing tool, adds:

I wonder if it might make sense to configure a "dummy" network printer on MPE/iX and have it send spooler output to a little socket listener on the Windows system (similar to the FakeLP example from the 3000-L archive) and then invoke GhostPCL on the Windows side for generating the PDF output.

The "dummy" network printer would let the MPE spooler take care of the PCL conversion and also perform the "file transfer" automagically. The GhostPCL software is probably easier to get (or build / update) on Windows than on MPE (okay, I admit that it did also build on MPE long ago.)

John Pitman employed that concept of Appel's in a combination of an off-the-shelf FTP solution, a freeware PDF converter, along with a good deal of innovative 3000 integration.

Nominate a spooled ldev as always suspended (74 in our case, arbitrary). Users can choose this device as their printer in their menu, and all subsequent reports (until changed to another real printer ldev) will go to this device — and therefore NOT physically print. Some reports that are commonly used to import to Excel have been modified to make headings tightly lined up with the data columns, and only print one page heading, to ease the import process.

Run a job on 3000 that every few minutes scans for spoolfiles for this ldev, then copies them to Posix space specific to 74 (for generality), with the creating user and account in the file name (e.g. mgr_stock_O12345.txt), then delete the original spoolfile.

A scheduled program on the Windows box (every minute) connects via FTP to the 3000. When it finds a spool files as above example, it checks for a Windows destination dir of MGR_STOCK, and copies the file to it as O12345.txt, and deletes the 3000 copy of the file. The account name enables segregation of reports for different applications in our case. If the file is > 1MB (an arbitrary size of your choice), it's zipped. It could as easily be converted to any desired form — for example to PDF via the shareware cutepdf.

It could also readily email the output to a user, given access to a mail server, and a way to develop the email address. Users have a client to access the FTP server and obtain their .txt or .zip files

This has been running for more than 10 years with almost no issues. Occasionally a large file might hang FTP, but canceling and restarting the copy usually fixed it. I have seen report selection errors produce 500 MB TXT files.

You might use several suspended ldevs for different types or groups  of users. We ran this on four 3000s in different locations, each with their own separate windows boxes using BP-FTP server. This means that users in Australia can run a report on the Houston or China system to the local printer 74, pause, connect their FTP client to the relevant FTP server, and download the report without having to print it.

The process also enables soft storage of month end reports, which can be very useful for comparative purposes, auditing, and general historical reference. We now have about eight years of this information stored, with backups and CD copies. Much more compact than paper, and cheaper.

Michael Anderson of consulting firm J3K Solutions added that there is also a open source PDF tool, pdfcreator, a manager can use to set up a network PDF printer. "Some assembly required, and batteries not included," he noted.

Another vote came in for the Advant/X software from Tracy Johnson, the OpenMPE volunteer who manages the Invent3K shared open source server. Johnson notes that this STR Software product "while intended to convert spool files and then e-mail or fax them, can be used short of the transmission process."

Posted by Ron Seybold at 05:44 PM in Hidden Value, Homesteading | Permalink | Comments (1)

November 11, 2015

Protecting a 3000 by Eliminating Its Services

Ron as PrivateHere on this day when we celebrate people who have served in the armed forces, a question emerged about enabling HP 3000 JINETD services. Or disabling them, to make a 3000 more powerful and secure. (Yes, it seems to defy the logic about more services being better, one that we can hear in national defense debates. We didn't have such debates at Signal Corps training for the Second Battalion.) The solution to the 3000 service problem included advice on how to trim back risk as well as performance drains on a 3000.

Grigor Terterian said he was having a Series 979 freeze up, because JINETD was receiving a call "for echo udb." Mark Ranft and Denys Beauchemin said the fastest repair would be to comment out echo in the inetdcnf file. Ranft got specific with an example.

:print inetdcnf.net
# Internet server configuration database
#
#echo  stream tcp nowait MANAGER.SYS internal
#echo  dgram  udp nowait MANAGER.SYS internal
#daytime  stream tcp nowait MANAGER.SYS internal
#daytime  dgram  udp nowait MANAGER.SYS internal
#time  stream tcp nowait MANAGER.SYS internal
#time  dgram  udp nowait MANAGER.SYS internal
#discard  stream tcp nowait MANAGER.SYS internal
#discard  dgram  udp nowait MANAGER.SYS internal
#chargen stream tcp nowait MANAGER.SYS internal
#chargen dgram  udp nowait MANAGER.SYS internal
telnet  stream tcp nowait MANAGER.SYS internal
#bootps  dgram  udp wait   MANAGER.SYS /SYS/NET/BOOTPD bootpd
#tftp  dgram  udp wait   NET.SYS   /SYS/NET/TFTPD tftpd
ftp  stream tcp nowait MANAGER.SYS /SYS/ARPA/FTPSRVR ftpsrvr

In the example above, only telnet and ftp services are enabled, Ranft said. This led Art Bahrs, a Certified Security Professional, to add that the services you leave on are the ones that can cause trouble, if you don't need them enabled.

Bahrs, who's also a retired Marine, celebrated his Veterans Day with this advice.

You should never, ever, no time, (did I mention 'never'?) run services you don't use or have a business or production need for.

Two reasons: First is security minded. If you have a service active, it is just another way to be hacked. Second is that an active, running service uses machine power, which is wasteful of electrons if there’s no need for it.

Ranft added his experience with inetd on MPE/iX:

Note that the command:

inetd.net.sys - c

will have inetd re-read the configuration.

Your success with this may vary. I've had lots of trouble with inetd in the far past. They got a lot better with the latest (final, for MPE/iX) set of patches. But on occasion, a scheduled restart (inetd.net.sys -k) and re-stream will probably help prevent issues.

I run my inetd with the logging feature.

!job jinetd, manager.sys
!....
!run inetd.net.sys;pri=cs;info="-l"
!eoj

This allows one to see the offending IP address in the $STDLIST.

Received call for: telnet tcp
telnet/tcp: Connection from unknown (10.0.1.226) at Fri Nov 6 19:56:28 2015
Received call for: echo tcp
echo/tcp: Connection from unknown (127.0.0.1) at Wed Nov 11 12:56:45 2015
Received call for: echo udp
echo/udp: Connection from unknown (127.0.0.1) at Wed Nov 11 12:57:07 2015
Received call for: echo udp
echo/udp: Connection from unknown (127.0.0.1) at Wed Nov 11 12:57:25 2015

As one veteran to others, I honor the services of all on this day, and thank you for your efforts toward our security. Long may it wave.

Posted by Ron Seybold at 09:59 PM in Hidden Value, Homesteading | Permalink | Comments (0)

October 20, 2015

Patching an HP 3000 with Patch/iX, Stage/iX

Editor's Note: Today a system manager asked for directions on using AUTOPAT on an HP 3000. That's very old school tech even for a classic system like the 3000 (TOH to Alan Yeo for digging out that link). More than 10 years ago, our contributor John Burke detailed the use of the more-modern MPE/iX patch tools.

By John Burke

Patch bicycleEven after HP introduced Patch/iX and Stage/iX to MPE/iX, these HP tools were poorly understood and generally under-used. Both are tremendous productivity tools when compared with previous techniques for applying patches to MPE/iX. There's a good reason for using the newer tools. Prior to the introduction of Patch/iX and Stage/iX, system managers did their patching with AUTOPAT, and you had to allow for at least a half-day of downtime. Plus, in relying on tape, you were relying on a notoriously flaky medium where all sorts of things could go wrong and create “the weekend in Hell.”

Patch/iX moves prep time out of downtime, cutting downtime in half because you create the CSLT (or staging area) during production time. Stage/iX reduces downtime to as little as 15-20 minutes by eliminating tape altogether and, furthermore, makes recovery from a bad patches as simple as a reboot.

This article is based upon the Patch Management sessions I presented at three Interex Solutions Symposiums. The complete set of 142 slides (over 100 screen shots) and 20 pages of handouts are downloadable from my website. The complete presentation takes you step-by-step through the application of a PowerPatch using Patch/iX with a CSLT and the application of a downloaded reactive patch using Patch/iX and Stage/iX. Included is an example of using Stage/iX to recover from a bad patch.

Why should you care about Patch Management? Keeping a system running smoothly includes knowing how to efficiently and successfully apply patches to the system. HP supplyed bug fix patches to MPE/iX and its subsystems through 2008, including two PowerPatches a year through 2006. Some patches add functionality to older 3000s. Patch/iX is a tool for managing these patches.

Patch/iX can be used to apply reactive patches, a PowerPatch, or an add-on SUBSYS tape with a PowerPatch. Patch/iX is actually a bundle including the PATCHIX program and a number of auxiliary files that are OS release dependent.

Patch/iX allows you to:

• Qualify all patches in a set of patches;

• Install reactive patches at the same time as a PowerPatch;

• Selectively apply patches from a PowerPatch tape; and,

• Create the CSLT (or staging area for Stage/iX) while users are still on the system; i.e., when it is convenient for you without incurring downtime.

Stage/iX is an OS facility for applying and managing the application of patches. Stage/iX reduces system downtime for applying patches to the length of time required for a reboot and provides an easy and reliable method for backing out patches. Stage/iX includes an interface to Patch/iX that creates the “staging area” and two utilities:

• STAGEMAN, which allows you to manage all aspects of patch staging, including which version of the OS will be used for the next update; and,

• STAGEISL, an ISL utility available from the ISL prompt whenever the system is down. It contains a subset of STAGEMAN functionality that allows you to recover from most problems.

Steps in staging

The set of all operating system files, for example NL.PUB.SYS, etc., are considered the current Base OS. Stage/iX creates and manages staging areas, which are HFS directories that hold versions of files that are different from the Base. More than one staging area can exist at a time. Each staging area contains the difference, or delta, between the Base OS and a patched version of the OS.

When a staging area is activated on the next boot, the files in the staging area are moved into their natural locations while the Base versions of the files are saved in a Stage/iX archive HFS directory. To back out a patch, the reverse takes place and the system is restored to its original state.

Once you are satisfied with the new and patched OS, you can COMMIT the staging area to the Base, deleting the staging area directory and all archived Base files. Note that Stage/iX and Patch/iX allow new patches to be staged and applied in a cumulative fashion. In other words, if you create a new staging area while another staging area is active, the new staging area will contain all the changes between the Base and the active staging area plus all the new changes.

Whether or not you use Patch/iX and Stage/iX, the key to successful OS patching is preparation. Information is the key to preparation. The System Software Maintenance Manual (S2M2) for your particular release of MPE/iX is the bible for all patch management activities. It contains a checklist for each possible update and patch activity along with detailed sections corresponding to checklist items. A hardcopy version and a PDF version on CD usually ship with each major OS release.

The S2M2 for each OS release is also available and downloadable from docs.hp.com. Information about specific patches is available at the IT Resource Center (itrc.hp.com). A PowerPatch usually comes with some patch specific documentation – make sure you have it available before you start.

Finally, before you ever sit down at the keyboard, create a Patch Book for the specific patch activity you will be attempting. You can do it with the hardcopy manual and a copy machine, but I prefer to use the PDF version, printing out the two-page checklist and each section that makes up the checklist to create my Patch Book.

How to get, apply patches

Suppose you just got the latest Patch Digest and there is a patch you need to install or you’ve read a thread on the 3000-L mailing list that references a patch you think you should install. Let’s look at how you can get and apply the patch using the ITRC, Patch/iX and Stage/iX. Note that the complete reference manual for both Patch/iX and Stage/iX can be found in the appendices of every S2M2 since, and including, the one for MPE/iX 5.5.

Before proceeding too far, check HPSWINFO.PUB.SYS to ensure the patch has not already been applied. [Note that Patch/iX will tell you if a patch, or even a superceding patch, has already been applied, but it only takes a moment to check HPSWINFO and it could save you some time.] Each patch has an eight character ID. For example, consider TIXMXC3B. The first three characters indicate the subsystem; in this case TIX stands for TurboIMAGE. The next four characters are internal to HP’s patch management mechanism. The final character is the version identifier; in this case “B” indicates the second version of this patch.

First off, identify the proper checklist, in this case Checklist B, and create your Patch Book. Next, review the information about the patches at the ITRC; in particular, look for any patch dependencies.

You need to make sure you have the latest version of Patch/iX installed on your target system. This is critical to your success. All sorts of bad things can happen if you use an old or incomplete version of the Patch/iX bundle. To check the version of Patch/iX, sign on as “MANAGER.SYS,INSTALL” and type in PATCHIX VERSION, The program will respond with something like: Patch/iX Version B.01.09

[Ed. note: getting patches from HP is a free process, but you may have some work in front of you to find a place inside HP support that recognizes the 3000 as a server, and knows how to deliver its patches. If you have a maintenance contract with a good third party — Pivital Solutions, for example — they can help lead you to the right HP cubbyhole.]

Once you have the current Patch/iX and your patches, you are ready to run Patch/iX and create your staging area. There are four steps in any run of Patch/iX:

• “Select Activities,” where you define what type of patching activity you want to perform. You have the choice of adding a PowerPatch, adding reactive patches from tape, adding reactive patches from download or adding SUBSYS products.

• “View Patches” (optional): You can actually view information about all the patches that have been applied previously to your system. Note that this can easily number in the hundreds for a system that is kept reasonably up to date.

• “Qualify Patches,” where Patch/iX does a lot of work to determine which patches of the set you supply it with can and/or should be applied.

• Create the stage, the tape, or both that will be used to actually change the OS.

This is all done while normal production continues and places a minimal load on your system. Once you have created your stage with Patch/iX, you run STAGEMAN to activate your staging area with the SET command. The next time you boot your system (and this can be done remotely from your home at 3 AM Sunday morning if you like), your changes will take effect. Total downtime is the time it takes to do a SHUTDOWN followed by a START NORECOVERY.

What if something goes wrong? If you have problems after successfully rebooting your system and you want to back out your patches, simply run STAGEMAN and use the SET command to make the Base the active stage, reboot your system and you are right back where you started. Suppose you cannot even boot the system successfully after setting the stage? Simply boot to the ISL prompt and use STAGEISL to set the active stage to BASE, reboot and, again, you are back to where you started.

Once you are satisfied with your changes after reasonable testing you can again run STAGEMAN and this time use the COMMIT command to make the active stage the Base and free up the disk space occupied by the old Base.

Of course I have only touched on the key points in the Patch/iX and Stage/iX process. There are a number of details I have left out that are covered in your Patch Book. My goal has been to demonstrate the ease with which you can patch an HP 3000 OS using Patch/iX and Stage/iX. Try it once and you will never go back to applying patches via AUTOPAT — nor will you ever again use a CSLT tape to apply patches if you can possibly help it.

Posted by Ron Seybold at 05:54 PM in Hidden Value, Homesteading | Permalink | Comments (0)

October 07, 2015

Reloading, Redux: How To

Loading cementIt used to be the worst thing that could happen to an HP 3000 was a reload of its data. Adager gained its central place in the 3000 manager's toolbox because it would prevent the need for a reload after database restructuring. Although the worst thing to happen to today's 3000 is the loss of an expert to keep it healthy and sustained, reloads are still a significant event.

Not very long ago, a 3000 manager was looking for a refresher on how to do a system reload. Ernie Newton explained why he needed one and shared what he knew. Advice about adding the DIRECTORY parameter, and using BULDJOB1 to set up the accounting structure, followed when Ernie said

We suffered a double drive failure on our Raid 5 disk array yesterday and I'm thinking that I may have to do a reload. It's been almost 15 years since I've done one. If I recall, I do a load when bringing the system up, then do a restore @.@.@.

Our resident management expert Gilles Schipper provided detailed instruction on doing a reload. We hope it's another 15 years for Ernie until he's got to do this again.

Here are the instructions, assuming your backup includes the ;directory option, as well as the SLT:

1. Boot from alternate path and choose INSTALL (assuming alternate path is your tape drive) 
2. After INSTALL completes, boot from primary path and perform START NORECOVERY. 
3. Use VOLUTIL to add ldev 2 to MPEXL_SYSTEM_VOLUME_SET. 
4. Restore directory from backup (:restore *t;;directory) 
5. openq lp
6. Perform a full restore with the following commands
:file t;dev=7(?)
:restore *t;/;keep;show=offline;olddate;create;partdb;progress=5
7.Perform START NORECOVERY

Gilles adds, "I would suggest setting permanent and transient space each equal to 100 percent on LDEV 2. The 75 percent default on LDEV 1 is fine as long as you don’t need the space. And if you did, your solution shouldn’t really be trying to squeeze the little extra you’d get by increasing the default maximum limits."

The reason for limiting LDEV to 75 percent is to minimize the otherwise already heavy traffic on LDEV 1, since the system directory must reside there, as well as many other high traffic “system” files.

You won't want to omit the ;CREATE and ;PARTDB options from the restore command. Doing so will certainly get the job done -- but perhaps not to your satisfaction. If any file that exists on your backup was created by a user that no longer exists, that file (or files) will NOT be restored.

Similarly, if you omit the ;PARTDB option, any file that comprises a TurboIMAGE database whose corresponding root file does not exist, will also not be restored.

I suppose it may be a matter of personal preference, but I would rather have all files that existed on my disks prior to disk crash also exist after the post disk-crash RELOAD. I could then easily choose to re-delete the users that created those files -- as well as the files themselves.

Another reason why the ;SHOW=OFFLINE option is used is so that one can quickly see the users that were re-created as the result of the ;CREATE option. Purging the "orphan" datasets would be slightly more difficult, since they don’t so easily stand out on the stdlist.

Finally, it’s critical that a second START NORECOVERY be performed. Otherwise, you cannot successfully start up your network.

Posted by Ron Seybold at 04:10 PM in Hidden Value, Homesteading | Permalink | Comments (0)

September 23, 2015

Where Do Those DBEs Go In MPE?

Where can I get help with storing and restoring an Allbase DBE?

Gilles Schipper replies

SQLUTIL.PUB.SYS should let you access the DBE. IMAGESQL is to a DBE what QUERY/DBUTIL is to TurboIMAGE.

Denys Beauchemin adds

ISQL.PUB.SYS is used to access the data in a DBE. If a TurboIMAGE database is attached to a DBE, there is a DBTC file for each Turbo database and an ATCINFO file with the DBE.

How can I find all the IMAGE databases on a system?

Michael Anderson replies:

IMAGE database root files all have a unique filecode value, -400. IMAGE datasets all have a value of -401. So if you want to find all IMAGE databases on a MPE system I would use the following command:

LISTFILE @.@.@;SELEQ=[CODE=-400];FORMAT=6

This will give you all the IMAGE root files and only the IMAGE root files. If you use something like “listf @01.@.@” you get the first dataset in the database, but also any file where the filename ends with “01”, and that may or may not be a database file. Also, the MPE file system allows filecode representations to be alpha (“PRIV”) and Numeric. When you see “PRIV” as a filecode it simply means that the numeric value of the filecode is negative, and again this can include non-database file types.

What is the command to kill a session that refuses to die?

Ernie Newton and Sam Knight reply:

If it is a VT or Telnet session, try NSCONTROL KILLSESS=#Snnnn

What is the maximum sized LUN that MPE 7.5 will recognize?

Guy Paul replies:

You should be able to have up to a 146 GB LDEV. Trial and error over the years has taught me to put no more than 70Gb/FWSCSI on a busy system. If you have Hyper-Volume extension, I would suggest slicing the mechs up a little smaller.

I have a couple of configuration questions about an N-4000. Does it make sense that an A6795A (PCI 4x 2Gb/s Single-port Fibre Channel Adapter) would be installed in PCI slots 1 or 2? The system's docs indicate that PCI slots 1 and 2 are 2x PCI slots; so the fibre channel would be limited to 1Gb/s. Does it make sense to have two Fiber Channel cards for a single VA7110?

Craig Lalley and Guy Paul reply:

The VA7110 does have two fiber connections, and both CAN be used. However, in order to be consistent with HP’s maintenance criteria, you should have one connected to a Command View workstation. Everything in Command View can be done through the serial port on the VA7100, except for logging. It is also a lot easier to define LUN’s graphically (Command View) as opposed to the command line (cryptic) language on the serial port.

Also on the VA7110, one of the fiber channels is “passive” and is for the command view workstation. I would not install a A6795A HBA into slots 1 or 2. Slots 3-12 are the twin-turbo (4x) slots you should use.

Posted by Ron Seybold at 09:02 PM in Hidden Value | Permalink | Comments (0)

September 18, 2015

Passing audits: MPE privileges can be keys

Migrated HP 3000 data can become forgotten while making provisions for an audit. Since some HP 3000s work as mission-critical servers, these active, homesteading systems must weather IT and regulatory audits. The 3000 is capable of passing these audits, even in our era of PCI, HIPAA and Sarbanes-Oxley challenges — all more strenuous than audits of the past.

However, establishing and enforcing a database update procedure is a step onto filling the gap in the security of an MPE/iX system. HP 3000 managers should take a hard look at how their users employ System Manager (SM) privileges. (Privileged Mode, PM, and System Supervisor OP should also be watched. Overall, there can be 21 capabilities to each user.) In their most strict definition, those privileges can expose a database. Hundreds of users can be created at Ecometry sites; even seasonal help gets SM users, according to one consultant's report, users which are seldom deleted after the holiday has passed. One site had a script to create new users, and each had PM capability, automatically.

Privileges are often a neglected aspect of 3000 operations, especially when the system's admin experts have moved on to non-3000 duties, or even to other companies. (Then there's the prospect that nobody knew how to use privileges in the first place.) Some SM users have disturbed the integrity of 3000 databases. It's easy to do accidentally. A creator of a database can also update a 3000 database — a capability that can foul up a manager's ability to pass some audits.

VEAudit from VEsoft, using its LISTUSER @.@ (CAP("SM")) filter, can give you a report of all of the SM users on your HP 3000. You can even ask for the SM users where password="". (Now there's a good list to find: SM users who have no passwords.) There is no MPE command that will do such things, we are reminded by VEsoft co-founder Vladimir Volokh. Even after more than three decades of his business as a 3000 software vendor, he also offers consulting on MPE operations and management, and still travels the US to deliver this. 

If you are worried about arbitrary access via QUERY, you can "disable subsystem access" via DBUTIL. This will, of course, only disable the access on QUERY.

Some less-adept auditors can also demand that a database's password be changed every 90 days. It's quite impossible to do, considering the database password is built into every application program.

So a database's security might be compromised through SM privileges, but it depends on the meaning of "update." This term can be construed to be as restrictive as using DBUPDATE to change an entry. It can also refer to UPDATE access DBOPEN MODE 2. 

To get very specific, an update can mean that the modify date has been changed in the file label of one or more IMAGE-related files. In a very general definition, an SM user can update the database simply by way of a restore from tape. (OP privileges permit this, too.)

Auditors sometimes ask broad questions, the sort of inquiry that fits better with the everyday use of HP 3000s in an enterprise. But for MPE/iX experts, "update" means any kind of modification capability.

So you can answer your auditor's question and say "no, SM privileges don't permit any of our users to update a database in another 3000 account." This answer is true, to the extent that the auditor's concern is about changing data — not just making a minor date change or using DBOPEN MODE 2. For auditors without MPE/iX and IMAGE expertise, well, they might not go so far in their examinations.

As for the SM user's ability to muck up an IMAGE database, it’s a mistake that is not difficult to make. An SM user who obtains a database password can corrupt an IMAGE database just by using the restore command. We’ve heard a story that such a user might explain, "Oops, I thought I was signed onto the test  account."

It's important to make a system fool-proof, because as Vladimir says, "fools are us." 

Posted by Ron Seybold at 04:31 PM in Hidden Value, Homesteading | Permalink | Comments (0)

September 15, 2015

COBOL Tools for Comma Separated Values

CSV exampleIt's been a long time since I wrote in COBOL, and I have an elementary question about creating a comma-delimited file. If I want...

"item one", "item two","","","item three"

How do I create the "," between item one and item two?

And the ","",""," would it be the same?  Just put that inside double quotes?

Something tells me that there is an escape sequence, but the mind is not cooperating. 

Walter Murray, who worked in HP’s Language Labs before moving on to other 3000 work, replies

The STRING statement is helpful. I don't advocate using an apostrophe to delimit nonnumeric literals, preferring to stick with standard COBOL. And yes, QUOTE is a figurative constant guaranteed to give you a quotation mark.

Murray offers a COBOL sample program to illustrate.
000100  IDENTIFICATION DIVISION.                            
000200  PROGRAM-ID. COBTEST.                                
000610  DATA DIVISION.                                      
000620  WORKING-STORAGE SECTION.                            
000630  77  ONE-COMMA    PIC X       VALUE ",".             
000640  01  1ST-ITEM     PIC X(6)    VALUE "ITEM 1".        
000650  01  2ND-ITEM     PIC X(6)    VALUE "ITEM 2".        
000660  01  3RD-ITEM     PIC X(6)    VALUE "ITEM 3".        
000670  01  MY-RECORD    PIC X(72).                         
000700  PROCEDURE DIVISION.                                 
000800  1000-START.                                         
000900      INITIALIZE MY-RECORD                            
001000      STRING  QUOTE 1ST-ITEM QUOTE ONE-COMMA          
001010              QUOTE 2ND-ITEM QUOTE ONE-COMMA          
001011              QUOTE          QUOTE ONE-COMMA          
001012              QUOTE          QUOTE ONE-COMMA          
001013              QUOTE 3RD-ITEM QUOTE                    
001014              DELIMITED SIZE                          
001015              INTO MY-RECORD                          
001016      DISPLAY MY-RECORD                               
001020      STOP RUN.       
001100  END PROGRAM COBTEST.

And the output...

"ITEM 1","ITEM 2","","","ITEM 3"

Tony Summers adds

It's worth putting a final comma at the end of the line, otherwise some versions of Excel can make the last column very wide.

Ken Roberston notes

One item worth noting, is that right before the STRING statement, you should move spaces to your output buffer.  STRING will not clear the buffer, so if your current string expression turns out to be shorter than your last, whatever was there before will remain - and you will get bad values in your output.

MOVE SPACES TO OSTRING
STRING XYZ etc. 
  INTO OSTRING

WRITE BIGREC FROM OSTRING

Finally Robert Mills has a macro for COBOL that creates a CSV record (copy and paste to retrieve all the characters in every line of the macro).

*> *************************************************************************
*> %AppendCsv(Field#,CsvRecord#)
*> -------------------------------------------------------------------------
*> Append Field to CsvRecord. If Field contains a comma (,) then Field is
*> quoted (") before being added. Quotes (") within Field will be replaced
*> with single-quotes (').
*> *************************************************************************

01 AppendCsv-macro.
05 AppendCsv-comma-count pic s9(04) comp value zero.
05 AppendCsv-field pic x(512) value spaces.
05 AppendCsv-field-length pic s9(04) comp value zero.
05 AppendCsv-index pic s9(04) comp value zero.
05 AppendCsv-pointer pic s9(04) comp value zero.

$define %AppendCsv=
initialize AppendCsv-macro
move function trim(!1) to AppendCsv-field
move length(function trim(AppendCsv-field)) to AppendCsv-field-length
move length(function trim(!2, trailing)) to AppendCsv-pointer
if AppendCsv-pointer > 1 then
move "," to !2(!3:1)
add 1 to AppendCsv-pointer end-add
end-if
perform
varying AppendCsv-index from 1 by 1
until AppendCsv-index > AppendCsv-field-length
if AppendCsv-field(AppendCsv-index:1) = "," then
add 1 to AppendCsv-comma-count end-add
end-if
if AppendCsv-field(AppendCsv-index:1) = quote then
move "'" to AppendCsv-field(AppendCsv-index:1)
end-if
end-perform
if AppendCsv-field-length > zero then
if AppendCsv-comma-count > zero then
string
quote, function trim(AppendCsv-field), quote delimited by size
into !2 with pointer AppendCsv-pointer
end-string
else
string
function trim(AppendCsv-field) delimited by size
into !2 with pointer AppendCsv-pointer
end-string
end-if
end-if#

Posted by Ron Seybold at 03:58 PM in Hidden Value, Homesteading | Permalink | Comments (0)

August 31, 2015

Posix file movements, using FTP and more

I'm attempting to move files in and out of Posix namespace on my HP 3000. The file I've copied becomes bytestream, and has a REC of 1. But I want to transfer that file from Posix down to my PC, I need to maintain it's structure — but what appears to be happening is it's one long record, with no separators. Is there a way (automated) that I can move files in and out of Posix, maybe FCOPY, and be able to keep the structure?

Donna Hofmeister replies

Simply copying/renaming a 'regular' MPE namespace file into HFS-namespace will not change its structural attributes.

If the MPE file was ';rec=-80,,f,ascii' to begin with, it will still be that afterwards.  And it will retain those attributes (cr/lf in particular) following an FTP transfer from your 3000 to [something else less enlightened].

To have a foreign/non-MPE filed take-on MPE fixed-record length attributes during an FTP transfer, simply add something like the following on your transfer line:

[put|get] non_mpe_file_name MPEFILE;rec=-NN,,f,ascii (making all the proper substitutions)

How do I get my HP 3000 to play well with Web-based FTP clients?

HP's James Hofmeister, who led the effort to keep FTP up to date on the 3000, replies

Lots of work went into an implementation of the FTPSRVR to support web access to the 3000. The "SITE POSIX ON" command can be sent by a FTP client and the 3000 FTPSRVR will emit Posix "standard" FTP output and will react like a Posix host (including file naming conventions).

It also is possible as documented to specify "POSIX=ON" mode in the SETPARMS.arpa.sys file and achieve this functionality system-wide for all non-3000 client to 3000 FTPSRVR connections; again the FTPSRVR will emit Posix "standard" FTP output and will react like a Posix host (including file naming conventions).

Warning:  Before you specify "POSIX=ON" mode in the SETPARMS.arpa.sys file, make sure you read the FTPDOC file closely; as you are warned that MPE file syntax will "no longer" work; The 3000 FTPSRVR is acting in Posix mode.

Posted by Ron Seybold at 06:41 PM in Hidden Value, Homesteading | Permalink | Comments (0)

August 19, 2015

Stripping on the 3000, Carriage-Style

How can I strip out the Carriage Controls from a spool file?

Tony Summers replies:

CarriageWithout dropping into Posix shell, the only other idea that comes to mind would be some third party tool. SPOOLPDF was a program we used from Open Seas to convert spoolfiles into raw PCL. A second program (OPENPDF) subsequently converted the PCL to a PDF document. OPENPDF was simply ported version (to MPE) of a Unix application (pcl2pdf). And we still use pcl2pdf on our HP-UX servers.

Lars Appel pulls a new approach out of his files:

Well, EDITOR.PUB.SYS also can change or replace columns.

/CHANGEQ 1/1 to "" in ALL

This changes column 1 through 1 to "nothing" in all lines (quietly).

If you don't want to strip CCTL but convert it to PCL escape codes, you might try using the network spooler (which typically does this when sending a file to the JetDirect printer). Example programs are in the HP3000-L archives, such as listening for TCP port 9100 to capture such data sent by the network spooler. Look for the FakeLP challenge with examples in Java, Perl, and C.

Tracy Johnson adds a note on a free utility:

I use Beechglen's SF2HTML utility. It's a convenient tool to convert CCTL codes to line and form feeds. Then I run it through an editor to get strip the HTML code. It only generates three codes, "<HTML>", "<BODY>", and "<LISTING>".  Easy enough to convert to blanks or nothings. Then use the Sanface Software program txt2pdf to convert the edited file to the final result.

Beechglen's Doug Werth notes

With enhancements in later versions, SF2HTML can be controlled by several variables, including one to remove the HTML tags from the output. This eliminates the extra step of running the resulting file through an editor before feeding it to txt2pdf.

Dave Powell offers a command file alternative:

Many moons ago I wrote a command file, HP2RTF to convert a cctl file to word-processor-readable rich-text format, converting the carriage control codes to the appropriate number of line-feeds as part of the process. It's happier if you redirect your output to a CCTL disk file, but it can sort-of handle spool files too. It contains its own complete source code, so if it doesn't do exactly what you want you can tweak it.

Posted by Ron Seybold at 08:15 PM in Hidden Value, Homesteading | Permalink | Comments (0)

August 12, 2015

How to Keep Watch on Backup Completions

DAT tapeWe've had a backup hang up on a bad DAT, and we learned about it when the morning jobs couldn't start on the 3000. (We shut everything down we can, back up, then open everything up again.) To find a better way to respond to this, I'm making a procedure to compare the expected backup duration (from a table we've built) to the backup's actual duration so far. The idea is to get an early report if the duration has been exceeded by more than an hour.

I've parsed the JOBLIST output to get what I wanted. But it looks like I'll need help on converting a string variable to a numeric variable as part of this procedure. Does MPE have anything like that?

Francois Desrochers replies with this ![...] construct, an undocumented part of MPE/iX:

While

SETVAR TONY STR(TONYALL,3,2)

would create a string variable, you could do something like:

SETVAR TONY ![STR(TONYALL,3,2)]

to create a numeric variable.

We're trying to restore about half a terabyte of data from an old HP 3000 backup set to a non-3000 machine. Is there a sound way to do this, or any way at all?

Stan Sieler replies:

We've done this for a customer who was without an HP 3000. We decided to use MPE/iX on our system to

  1. Restore their data onto our 3000
  2. Package up the files (either with tar, or LZW)
  3. Move them to an external disk drive on a Unix or Linux or Windows machine
  4. Unpack them
  5. Then send that external drive to the customer.

This morning I came in to find our backup job stalled. Abortjob was ineffective, as was abortio. I ended up rebooting the system. While coming up, I got a “defective sector” message with “FILE.GROUP.ACCOUNT has an extent with unreadable data.” The file is now locked, and I need to use FSCHECK to unlock it. How can I determine which drive this extent is on?

Stan Sieler replies

FSCHECK’s DISPLAYEXTENTS command may help. Note that, if I recall correctly, it displays logical unit numbers, not exactly LDEVs.

I'd like to make sure I get complete backups. Should I be using @.@.@ in my TurboSTORE command?

Gilles Schipper replies that backups don't need to be specified with an @.@.@ command to be complete:

People should really be using the forward slash, because it's easy to accidentally omit the Posix file structure if you're not careful constructing your fileset backup. The slash is so much better — a backup specified by HP's TurboStore will replace any @.@.@ operation with "./" Combining @.@.@ with exclusions can lead to omitting files which should have been in a backup.

Posted by Ron Seybold at 05:56 PM in Hidden Value, Homesteading | Permalink | Comments (0)

August 10, 2015

How to Make a 3000 Act Like It Uses DNS

DomainsI have a script that uses FTP to send files to a site which we open by IP address. We've been asked to change to SFTP (port 22) and use the Domain Name Service name instead of an IP address. Does the 3000 support using DNS names?

Allego's Donna Hofmeister replies:

To start, I'm not sure you want to do SFTP on port 22. That's the SSH port. SFTP is meant to use port 115. Have a look at one of Allegro's white papers on how to enable SFTP on MPE

If you are going to use DNS, you must have your 3000 configured for that.  It's easily done.

However, if you've never done anything on your 3000 make it act like a real computer (oh -- that's right, it is a real computer and fully capable of using DNS), this can turn into a can o' worms.

For 'DNS lite' it's probably simplest to:

1. Copy hostsamp. net to hosts.net

2. Edit hosts.net to make sure it has

127.0.0.1 loopback
1.2.3.4   name    <--- where 1.2.3.4 and name are corrected to the system you want to connect to

3. Copy the NSSWSAMP.net to nsswitch.net

4. Edit nsswitch.net to have this line:

hosts : files[SUCCESS=return NOTFOUND=continue]

With this done, the 3000 sorta kinda acts like it's using DNS — because it's looking the the hosts file for how to translate 'name' into '1.2.3.4'

Tony Summers provides a caveat:

One warning. The upgrade from FTP to sFTP (or SSH FTP etc) can involve more change to your scripts than you expect.  What we do for FTP (originally on the HP 3000, and now on the HP-UX server) is build a text file with the commands (the sample below, edited)

cat FTPT0070
open ftpserver.site.co.uk
user USERNAME PASSWORD
ascii
get /export/002_iccm_extract_1161.csv ICR21161

quit

The file is then presented to the FTP client. On the HP 3000 it was something like....

RUN FTP.ARPA.SYS < FTPT0070 > FTPS0070  

Then both the output file, FTPS0070, and any JCWs set by the FTP program were inspected to test the success of the FTP session.

cat FTPS0070

Connected to xxxxxx.co.uk

220 Welcome to FTP service - xxxx.
331 Please specify the password.
230 Login successful.
200 Switching to ASCII mode.
200 PORT command successful. Consider using PASV. 550 Failed to open file.
221 Goodbye.

In particular,  the 3-digit status codes were analysed,  looking for error codes like "550".If you do something similar in your FTP scripts,  then all I can say is welcome to a very different world.

Karsten Brøndum adds:

Here's a completely different approach. 

Depending on your skills in the Java area, there is a nice LPGL package called ftp4j (which requires Java 1.4 or later) that I have used a couple of times. It's a full-featured Java-based FTP client. (By the way, ftp4j will do both SFTP and FTPS). I've found it way easier than to fiddle with files with text files containing commands, especially when it comes to error handling.

Posted by Ron Seybold at 05:04 PM in Hidden Value, Homesteading | Permalink | Comments (0)

July 17, 2015

Do Secure File Transfers from the 3000

I'm trying to use ftp.arpa.sys to FTP a file to a SFTP server and it just hangs. Is there a way to do a secure FTP from the HP 3000?

Brian Edminster replies:

The reason that using MPE's FTP client (ftp.arpa.sys) fails is because as similar as they sound, FTP and SFTP are very different animals. Fortunately, there is a SFTP client available for the 3000 -- the byproduct of work by Ken Hirsh and others.

It used to be hosted on Ken's account on Invent3K, but when that server was taken out of service, so was Ken's account. As you've no doubt already noticed, it's available from a number of sources (such as Allegro). I'd like to highlight another source: www.MPE-OpenSource.org

Edminster goes on to explain he administers that site, as well as puts together the 'pre-packaged' install available there. It's in a single store-to-disc file in Reflection 'WRQ' format, making it easy for the majority of sites to retrieve and use.

I have a customer that's been using SFTP daily as part of their PCI compliance solution for several years. They push and pull data hourly from dozens of Point-of-Sale systems all over the country, and have moved lots of data this way.  

The biggest caveat from that customer's implementation is that if you're moving data over a WAN, SFTP seems to be more sensitive to jitter and latency issues than conventional FTP.  We ended up having to upgrade a couple of their more anemic 'last mile' circuits to accomodate that.  

In all other respects, it’s quite a robust solution, and can be tightly integrated with existing legacy apps. I know; I've done it.

If you have any questions about how to use the pre-packaged install -- or how to get around any limitations you might run into,-- don't hesitate to contact me. I've used this on dozens of systems over the last decade, and have transferred many, many gigabytes of data with it.

Posted by Ron Seybold at 09:02 PM in Hidden Value, Homesteading | Permalink | Comments (0)

July 08, 2015

Freeing HP's Diagnostics Inside the 3000

DiagnosticsWhen HP officially closed its formal HP 3000 support, the vendor left its diagnostics software open for use by anybody who ran a 3000. Throughout the years when HP sold 3000 support, CSTM needed a password that only HP's engineers could supply. But the CSTM diagnostics tools started to run in 2011 without any HP support-supplied password. 

However, managers need a binary patch to free up the diagnostics. Support providers who've taken over for HP know how to enable CSTM. The community has a former Hewlett-Packard engineer to thank, Gary Robillard, for keeping the door to the diagnostics open. Robillard says he is the engineer who last worked on CSTM for MPE/iX when he was a contractor at HP.

A 3000 site must request a patch to get these expert tools working. HP arranged for 3000 sites to get such patches for free at the end of 2010. We tracked the procedure in a NewsWire story, since the HP link on how to get these patches, once on the old division's webpages, has gone dead.

One such patched version of CSTM needs a binary patch. Robillard created this binary patch fix.

Versions of CSTM [patched] with ODINX19A or ODINX25A allow the expert tools with no licensing, but you still have to issue the HLIC command. 

If you install ODINX25A/B/C (6.5, 7.0, 7.5) you won't need to do anything except issue the HLIC command with any password. The HLIC command might say it was not accepted, but the license is activated anyway

When HP halted all its support of the 3000, Robillard said his patch corrects the problem with ODINX19A -- and gives 3000 managers access to these system diagnostics -- for servers running the 6.5, 7.0, or 7.5 versions of CSTM.

If you have installed ODINX19A, you need to do the following:

Logon as MANAGER.SYS. It's safest to create an input file to sompatch.

1. Run editor.pub.sys

2. Add the following three lines EXACTLY. The sompatch will only work if the instruction at offset 268 matches 86a020c2. The message "Error: Old value does not match" is displayed and no changes are made)

Here are the contents of BINPCHIN file (You will want to copy and paste these): 

~~~~~The 3 lines are below~~~~~~~

; Fix problem in DIAGMOND after 12/19/2010
modify ms_init_manage_sys  + 268,1 86a020c2|08000240 
exit
~~~~The 3 lines are above~~~~~~~~
//
K binpchin,unn 

• Make sure DIAGMOND is not running (run STMSHUT.DIAG.SYS)
• copy /usr/sbin/stm/uut/bin/sys/diagmond,DIAGMOND 
• run sompatch.pub.sys;stdin=BINPCHIN;INFO='DIAGMOND' 
• copy DIAGMOND,/usr/sbin/stm/uut/bin/sys/diagmond;YES 
• Restart DIAGMOND (run STMSTART.DIAG.SYS

After a few minutes, a "SHOWPROC 1;TREE;SYSTEM" should show the DIAGMOND process, and either the mapping processes, or memlogd, diaglogd and maybe cclogd (on A/N Class 3000s only).

Posted by Ron Seybold at 10:13 PM in Hidden Value, Homesteading | Permalink | Comments (0)

June 22, 2015

Fixing Date Problems From The Future

HP 3000 managers have traveled long roads toward the future of their servers, but sometimes the server travels even farther. Into the future, it seems, to apply modification dates to files that couldn't possibly be modified months or years from now.

Back To The FutureThis can cause problems with system maintenance. Craig Lalley experienced some last week. After running the NMVALCK command, he discovered "I have thousands of files with future dates." He was pretty sure there's a way to adjust a date like FRI, SEP 10, 2027, 1:53 AM by using MPEX. (A good bet, since the Vesoft product manages the 3000's files better than MPE/iX itself). But what about other repair options?

There are two, one in the community's freeware resources, and one in its Posix namespace. The freeware comes from Allegro Consultants. FIXFDATE (just do a "find" on the web page to locate the utility's entry) "will sweep through your files and change any creation, modification, access, allocation, or statechange date that is a "future" date to be today."

Another resource comes from within the 3000's Fundamental Operating System. Touch, a common Posix utility, exists on the HP 3000's implementation. 

Touch is chronicled on the Open Group Base Modifications website. While Posix is not as well-loved in the 3000 community as CI commands or freeware, its touch is bristling with options. The basics:

Touch shall change the last data modification timestamps, the last data access timestamps, or both.

The time used can be specified by the -t time option-argument, the corresponding time fields of the file referenced by the -r ref_file option-argument, or the -d date_time option-argument, as specified in the following sections. If none of these are specified, touch shall use the current time.

Changing the dates of a full system's worth of files might someday be the mission for any 3000 owner who's trying to carry MPE beyond January 1, 2028. The world of 3000 Future Time hasn't been explored much, mostly because the community seems confident some solution will be available in 2027. It's still 12 years away, after all.

Allegro's Steve Cooper knows date representation issues have been addressed by the 3000 development community before -- in an era when many companies were still engineering for the system. He can sound sanguine about the issue because his partner Stan Sieler engineered date repairs for the 3000 during Y2K work at the end of the 1990s.

Allegro sold a Y2K utility as part of its 3000 development toolset. "If anyone cares by then, one will need to do remediation similar to what was done for Y2K," Cooper has said. "Each program will need to be inspected for vulnerabilities, then fixed to use an alternate method of date storage and manipulation."

And, yes, as you suspect, this could arise before then. If, for instance, you are manipulating contract expiration dates in your COBOL program, and are using a 2027-sensitive format, then you will not be able to correctly handle any date past the 2027 cut-off.

If you don't mind, though, I'm not going to lose any sleep over this issue for another several years. Remind me again toward the end of the decade, and I'll ask Stan to look into it.  If I ask him now, I will lose a few months of productivity out of him, while he solves next decade's problems.

The issue is not limited to the HP 3000, even though the 2027 date is unique to MPE/iX. Unix has got the same kind of deadline approaching 11 years later. Cooper pointed to a Wikipedia article that explains the "Year 2038 problem" as an analogy to the 3000's.

Posted by Ron Seybold at 07:03 PM in Hidden Value, Homesteading | Permalink | Comments (0)

June 12, 2015

Find hardware specs, move DTCs, and more

Is there a command or way to see the hardware specs of a HP 3000 via MPE or its installed utilities? This machine has no other utilities, like MPEX. I am looking to document the processors, memory, number of hard drives, and size of those drives. 

Jack Connor replies

Depending on MPE/iX version, you can use SYSDIAG for 6.0 and older or CSTM for 6.5 and later. In SYSDIAG, type SYSMAP, then IOMAP, and GENERAL for the IO components, then exit and go to CPUMAP for the CPU info.

In CSTM, type MAP, then SELECT ALL, then INFO, then IL (InfoLog) to get a listing of everything that MPE owns.

I don't work that much with COBOL these days, but I wanted to compile a  program and I got an error message,"size of data segment greater than 1 gig or 64 bytes" How do I get around this?

Steve Cooper replies

That means that the total space you asked for in your Working Storage Section is more than 1 GB.  Now, there are ways to work around that, but my guess is that you don't need to work around that.  My guess is there is a typo or some other unintended problem, where you are asking for way more storage than you intended. Check your OCCURS clauses and PICs to make sure you mean what they say.

We have to move a DTC into our network. Along the way there are Procurve switches and a Cisco router or two. I know that somehow the switches and routers must be configured so as to allow multicasting on addresses 09-00-09-xx-xx-xx to be forwarded and not filtered, but our Procurve administrators aren't quite sure they know how exactly how to do this. What is Procurve-ese for configuring what's necessary to allow remote DTC operation across our network?

Jeff Kell replied

You don't have to do anything at layer-2.  The layer-3 will have to bridge the traffic.  If the layer-3's are Cisco, you can specify the traffic to bridge, in which case you just want the [08|09]-00-09-x-x-x traffic.

The 09-00-09 is multicast used for discovery, but beyond that, you're going to get some directed layer-2 08-00-09 as well.  You will need to include your HP's DTSLINK NIC MAC prefix as well if it is a later model that does not have the old HP 08-00-09 prefix.

If you own the whole infrastructure path, what we used to do is run L2 trunks between sites, and propagate a common vlan for the DTC/HP traffic, while routing everything else.

Or if the Ciscos are real routers rather than Catalyst-style switches, you can run a L2TP tunnel end-to-end to propagate the HP vlan across the routed hops.

I need to rebuild an environment from one HP 3000 system to another. Trouble is, we want to have groups from the same account end up on different user volumes. Is there a way to do this using BULDACCT? 

Keven Miller replies

BULDACCT was made for processing complete accounts. Do BULDACCT  CHC%VSACCT=MEDADV_1. Then edit BULDJOB1 for the other group, changing MEDADV_1 to _2

Mark Ranft adds 

What I usually do is use BULDACCT to move the entire accounting structure. Then I surgically PURGEGROUP and NEWGROUP (with the appropriate HOMEVS= and ONVS= options, plus CAP= and ACCESS= etc) to duplicate the special groups.

I'm having trouble locating TurboIMAGE limits. What's the max number of items in a dataset?

Paul Edwards replies

From HP’s website: 

The objective of this TurboIMAGE enhancement is to raise the limits on number of items, sets and master paths in a TurboIMAGE database to these values: 

Number of items: 1200
Number of sets per database: 240
Number of paths for master : 64

All new databases created will have the benefit of increased limits. The new limits will not be applicable for old databases created under previous versions of TurboIMAGE. However, these databases will continue to function with the old limits and applications will be compatible.

Posted by Ron Seybold at 06:17 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 27, 2015

Make old PCs do a console's work

HP PortableGot a wheezing PC someplace in your IT shop? Believe it or not, even the creakiest of desktops can still serve your HP 3000: as a console, a la the HP700/92 variety. This is the kind of PC where, as one veteran puts it,"the keyboards have turned to glue."

...Trying to type a coherent instruction (or even worse, trying to talk someone through that task remotely) where random keys require the application of a sledgehammer to make them respond, at which point they auto repeatttttttttttttttt.

It's enough to give a veteran manager a pain in the posterior, but hey -- some HP 3000s (of the 900 Series) demand a physical console as part of their configuration. Can't you just hook up such an antique PC straight to the 3000's special console port and let it work as a console? Yes, you can.

Overheard among chatter between 3000 vets:

You can connect a PC via its serial port to the console port on the HP3000, and then run a terminal emulator via the serial connection, leaving it logged on as the console. That way, using free remote control software (VNC Free) on the PC, you could even have control of the physical console (as opposed to just taking the virtual console) so that things like Control-A/B would work.

You'll need a little physical cabling help to make this work. Even though those desktop PCs are old, most of them have not had serial ports in many years. Think about it. It's all USB by now. So you buy a USB-to-serial converter. You'll need a copy of an HP 3000 terminal emulator on the PC configured to connect via the serial port.

Just make sure the PC stays up and the emulator's window is open and connected. You don't need the console buffer filling up.

Posted by Ron Seybold at 04:31 PM in Hidden Value, Homesteading | Permalink | Comments (0)

May 01, 2015

Message Files Editing, DLT Autoloading Tips

What tools can I use to 'edit' a message file without destroying the file? I learned the hard way that if you’re using FCOPY to copy from a message file, it destroys the records read from the message file. Can you "COPY" a message file without destroying the records that are copied? 

François Desrochers says

You can use non-destructive reads by specifying the COPY option on a file equation as in:

FILE MSGFILE;COPY
FCOPY FROM=*MSGFILE;TO=NEWFILE

There's also an FOPEN Aoptions (bit 3:1) to enable the option.

As for editing a message file, the only safe way I know is to extract all the records into a regular flat file, edit it and load it back into the message file.

If it's easy to identify which records to remove, you can use FCOPY.

:BUILD MSG2;REC=-80,,F,ASCII;MSG
:FCOPY FROM=MSG1;TO=MSG2;SUBSET="Y",1

Extract records with "Y" in column 1. MSG1 is empty at the end of the task. To use non-destructive reads, do: 

:FILE X=MSG1;COPY
:BUILD MSG2;REC=-80,,F,ASCII;MSG
:FCOPY FROM=*X;TO=MSG2;SUBSET="Y",1

MSG1 still has all the original records. 

Use FCOPY's EXCLUDE option to extract all records except the ones with a "Y" in column 1 as in:

:FCOPY FROM=*X;TO=MSG2;SUBSET="Y",1,EXCLUDE

I have a HP Q1567A DDS5 auto loader installed on a 987. I installed and configured it on the SE port on the multifunction IO card. I can store files to it fine, but I can't get it to advance each day to the next tape. How do I do this?

Ryan Melander replies

Does it auto-eject the drive when the backup finishes? The reason I ask is I have the same configuration on my account with a DLT7000 autoloader 8/Slot. I have it set up to auto eject after each backup, moving the tape back to its original position. It then auto inserts and mounts the next tape for the next backup time.

Jack Connor notes

I generally set the DLT autoloaders for no circular, stacker mode. This goes through all the tapes in the unit and then stops.

Jim Hawkins says

DEVCTRL.MPEXL.TELESUP should work to set up the eject option. Put it inside your job to be sure that it is set correctly every time.

Gilles Schipper adds,

Specifically,

!DEVCTRL.MPEXL.TELESUP 7 EJECT=ENABLE LOAD=ONLINE

where the "7" is the LDEV number of the tape drive.

Posted by Ron Seybold at 04:02 PM in Hidden Value, Homesteading | Permalink | Comments (0)

April 27, 2015

Sendmail, Group Purges, and ACD Removal

Is there a proper or "right" way to shut down sendmail?

Donna Hofmeister replies

• Use the Posix kill signal from SERVER.SENDMAIL or any user with SM capability. (The following can be easily turned into a job!)

kill $(head -n 1 /etc/mail/sendmail.pid)

• Only use :ABORTJOB as a last resort! (This is true for all of the Posix things that got ported to MPE)

If you don't need to run a mail server (e.g. sendmail) on your 3000, you shouldn't. In most cases, using a mail client will be "just the ticket." Point the client at your in-house (SMTP) mail server and enjoy.

How can I easily purge all the files in a group without destroying the group structure?

If GRPNAME is the name of the group then either:

1. chgroup GRPNAME and

purgegroup GRPNAME 

Or, purge @.GRPNAME 

[Ed. note: Vladimir Volokh notes this last command does not purge databases from a group, although it purges everything else. You must be an SM user to purge everything in one account from another account. Of course, MPEX's %purge will purge everything, and will report the list of what is to be purged. %purge(ISPRIV) for a selection of databases only, for example.]

How can I convert an SL to an XL?

Jeff Kell and Gavin Scott reply:

You can OCTCOMP an SL, which will make the code in it run in mostly-Native Mode (though using several times the memory) and with exactly the same limitations as the original CM code. OCTCOMP just adds a pre-translated version of the CM code to the end of the file that will be invoked when you run the program or SL on an MPE/iX system.

We have problems accessing the files of one group because someone has added ACD's on all the files within that group. Is there a way to remove them all easily?

Mark Ranft replies

Using MPEX, you can issue the command

ALTFILE @.GROUP.ACCOUNT;DELACD

Keven Miller adds

And if you don't have MPEX, you can use a command file like the one below.

PARM FILES=@, PCMD="ECHO File !"
SETVAR XSI_CMD "!PCMD"
IF FINFO ( "CMDFLST,TEMP", "EXISTS" ) THEN
PURGE CMDFLST,TEMP
ENDIF
FILE CMDFLST;MSG;TEMP;NOCCTL;REC=-40,,F,ASCII;DISC=100000
LISTF !FILES,6;*CMDFLST
FILE CMDFLST,OLDTEMP
SETVAR XSI_CNT FINFO ( "CMDFLST,TEMP", "EOF" )
SETVAR XSI_R 0
WHILE !XSI_R < !XSI_CNT DO
 INPUT XSI_FILE < CMDFLST
 SETVAR XSI_R XSI_R + 1
 SETVAR XSI_FILE RTRIM ( XSI_FILE )

 SETVAR XSI_DO REPL (XSI_CMD, "!", XSI_FILE)
 SETJCW CIERROR=0
 CONTINUE
 !XSI_DO
 IF CIERROR = 0 THEN
    ECHO !XSI_DO
 ENDIF
ENDWHILE
ECHO !XSI_CNT FILES
DELETEVAR XSI_@
PURGE CMDFLST,TEMP

This command file takes 2 parameters. 1. a  fileset (wildcards allowed) 2. an MPE command with a ! exclamation as place holder for the filename It does a LISTF ,6 of your fileset into a MSG file. Then it reads through the MSG file and processes each file with your command.

Posted by Ron Seybold at 11:18 AM in Hidden Value, Homesteading | Permalink | Comments (0)

April 24, 2015

Solutions for Keeping Passwords Fresh

Our management wants our 3000 users to be forced to change their password on a regular basis. Also, certain rules must be applied to the new password. We don’t have VEsoft’s Security/3000, although we do have MPEX. I therefore have two options. 1. Write something myself, or 2. See if there is anything in the Contributed Software Library that will do the job, or can be modified to supply the required solution.

Homegrown and bundled solutions follow. Jeff Vance offered this:

There is a pseudo random password generator available among the Jazz files which knows MPE’s password rules. See RANDNAME. There are also UDCs which force a password to be supplied when using NEWUSER, NEWACCT and NEWGROUP CI commands. These required passwords can be random or user entered with a minimal length enforced. 

Then he added as an afterthought, a strategy to program your own password system:

I haven’t thought about it much, but it seems you could have a password file (maybe a CIRcular file?) for each user on the system. This file would have their last N passwords, and the modified date of the file would be the date their password was most recently changed.

A logon UDC could detect if the password file for that user exists. If not create it and require a new password right then.  If the password file exists then get it’s modified date and compare that to today’s date. If greater than X days then in a loop prompt for a new password. Validate the entered password against previous N passwords and your other rules. Maybe run a dictionary checking program to make sure the password is not common, etc.

Update the user-specific password file with their new password, and then logon the user.

From the support community, Donna Hofmeister weighed in with this advice:

The solution that your management demands is going to cost more for you to develop, implement, or maintain than it’ll take for you to get Security/3000. If you have no choice other than to develop a product, then I’d certainly model it after what VEsoft has already done. That is:

Based on a system-wide UDC, examine all sessions (it is just sessions, yes? By the way, a DSLOGON from inside a job is still a session.) against a ‘database’ (By the way, just how secure is this database?  A real database needs passwords... Who’s going to maintain that? A flat file could be lock-worded, but that’s not a slam-dunk answer.) a database which is looking for the ‘age’ of the password (By the way, are you going to provide an advance warning period?). 

If it is time to change the password, get the ‘new’ password from the user... but writing the rules is a pain, and keeping track of reused passwords is just annoying. Auditors in the states love when you can say the password is one-way encrypted. Dunno what your management is saying for encrypting an MPE password.

Then came a solution rolled up by Paul Christidis

Some years ago I had developed a set of command files that could be used  to require users to have passwords. Later on, mostly as an exercise, I  enhanced the process to age passwords and to automatically assign ‘random’  passwords as they expire.  The random passwords are comprised of  alternating consonants and vowels, they can have a minimum and maximum  length and optionally a random digit can be inserted.

The entire ‘process’ is comprised of a system batch job (should be running  always), a command file that is invoked by a log-on UDC and communicates  with the batch job, a ‘control’ command file that starts and stops the  batch job, a command file to determine the password age and a command file  to generate the random password. Below are the comments from the batch job. They explain some of the details.

!# Author:  Paul H. Christidis
!# Remarks:   This job 'listens' at a message file for any requests to
!#    determine if a user has a password.  Once that determination is
!#    made it passes back to the session an indicator to that effect.
!#    A command of STOP causes the job to terminate.
!#
!#      The request comes via the execution of a command file or a System
!#    wide UDC and it is comprised by the file name where the reply should
!#    be placed and the user's name and account.
!#      This job does NOT return the user's password, it only writes in 
the
!#    message file specified by the client the command:
!#          setvar user_password true/false
!#      The client then executes the command and tests the setting of the
!#    variable 'user_password' to decide what action to take.
!#
!# DATE:    06/08/2004  *** WHILE RETAINING THE ABOVE BEHAVIOR ***
!#   The job logic was changed to assign a new password after 30 days.  A
!# file in the posix space is built using the user's name.  Then at each
!# logon a command file is used to determine the file's age, using its 
modify
!# date, and when it is older than 30 days another command file is used to
!# generate a random password.  Said password is sent back to the session
!# and the user is informed about his new password.
!#
!#   If the 'job/session' name is not to be used in creating the posix 
file
!# that will be used to age the password, then the value of the CI 
variable
!# 'pw_UseSess' should be set to 'FALSE'.
!#
!# DATE:    06/10/2004   [Added alternative aging values functionality]
!#   Alternative aging limits are kept in an ASCII file "pswrdage" 
comprised
!# of 'setvar' commands, for each MPE account, MPE user or Session name. 
It
!# should adhere to the following format:
!#       SETVAR SYS_MANAGER_XTIDIS_pwage   45
!#       SETVAR SYS_MANAGER_pwage          40
!#       SETVAR SYS_pwage                  35
!#
!#  The above have the following implications:
!#    The user's "xtidis,manager.sys" password expires in 45 days
!#    The user's "manager.sys" password expires in 40 days
!#    Any other user of the "sys" account has their password expire in 35 
days
!#    While the 'default' 30 days applies to every other user on the 
system.
!#
!# NOTE 1: Suffix of '_pwage' is required.
!# NOTE 2: A negative or zero setting equals to NO password aging.
!# NOTE 3: The order MUST be 'ActName_UserName_JobName_pwage'.
!#
!# Date:    06/11/2004   [Added code to 'force' a password change]
!#   When the CI variable "ForcePwChange" is set to TRUE in the session 
that
!# executes the command file, the 'passed' code is changed and the batch 
job
!# forces the password change (Unless it was already changed on the same 
day)
!# <---------------------------------------------------->

Dave Powell coded up some of the fine print nicely in his contribution:

It ought to be possible to do everything for free, using just MPE.  Editing the new password is the ugly part, but if you randomly assign it that issue goes away. The next issue is that the :password command seems to like to be purely interactive, as in:

echo oldpass >> tempf
echo newpass >> tempf
echo newpass >> tempf
:password < tempf
password
Command not allowed in noninteractive environment. (CIERR 2500)
PASSWORD WAS NOT CHANGED.

That leaves the altuser cmd, which needs AM cap.  If you don't want a background job (like Paul's suggestion), you can have the command file (called by  your logon UDC) use the echo command to build job with AM cap, which it then streams, kind of like (untested, but I have working examples of cmd-files building other jobs):

ECHO !!JOB ACCTMGR/PASS.SOMEACCT; HIPRI  >> TF
ECHO !!ALTUSER !HPUSER;PASS=!NEW_PASS >> TF
ECHO !!SETVAR STREAMED_BY   WORD(HPSTREAMEDBY,"()",2) >> TF
ECHO !!TELL !!STREAMED_BY Your new password is !NEW_PASS >> TF
ECHO !!EOJ >> TF
STREAM TF
ECHO Please note your new password when it appears
PAUSE 99; JOB = !HPLASTJOB
PURGE TF, TEMP

If I haven't screwed up the fine print too badly, this code in the middle of the password cmd-file runs the job that changes your password, then waits for the job to tell you what the new password is.  The single exclamations before HPUSER & NEW_PASS mean that values that are variables to the session and command-file become hard-wired values for the job.

Before all this your cmd-file checks the date, gets a random password, etc., as posted by others.  After it the cmd-file writes (or just builds) a file that serves as a timestamp.  But I am not too comfy with putting the passwords into a plain-text file, so I might skip that part (remember, the user needs both read and write access to it).  Put the command file in a group that users have xeq access to, but not read/write access.

Posted by Ron Seybold at 08:25 PM in Hidden Value, Homesteading | Permalink | Comments (0)

April 13, 2015

How MPE Talks to Its Network Neighbors

Our networking team reports they're going to refresh the hardware on our IP gateways. Our Telecom manager says they will 

  • Change the physical gateway, because the hardware is being replaced
  • Not change the IP address and gateway address
  • Change the MAC address of the gateway (because of different gateway hardware)

Network NeighborhoodWhat do I need to do on our MPE boxes to ensure that they will see the new hardware? Does MPE cache the MAC address of neighbor gateways anywhere? I was thinking I needed to restart networking services, but I wasn't sure if anything more will be needed.

Jack Connor replies

If you're taking it off the air for the network changes, I'd go ahead and close the network down until the work has completed and then reopen it. MPE will be looking for the IPs as it opens up. I know you can see the MAC addresses in NETTOOL, but I don't think they're of any import other than informational and for DTC traffic.

Donna Hofmeister adds

Halt the network (even the system if possible -- because it's almost the same thing) while the larger network work is being done. When the new gear is in place and seems stable, "wake up" the 3000 and watch what happens.

When you halt the network (presuming you're not taking the box down) be sure to halt/quiesce network-dependent things (like jobs/listeners) just prior. I'd suggest doing an 'openq' on your network printers as well (keep the input side of printing open, but not the output side).

Jeff Kell notes

If you have access to your routers/switches, you can also attack this from the other end. 

Cisco switches/routers (layer-3) have an ARP table that holds the MAC addresses of the hosts on the subnet. They are subject to a timeout. However, they will generate a "gratuitous unicast ARP" request to any host in the table 30 seconds before it is scheduled to expire. If the host is up, it will respond, and reset the learning timer. It will ALSO push the MAC address and IP of the gateway (the router that just performed the unicast ARP) into the host.

Unfortunately the default ARP timeout on a Cisco device is 4 hours.

You may tweak this as desired on a per-interface basis via the "arp timeout xxx" configuration directive.

If you are doing network monitoring by any SNMP tools, they work best when the mac-address tables (CAMs/TCAMs/etc) are loaded with the host values, and the ARP tables are populated with the current hosts. The mac-address table timeout default is only 300 seconds (I'd suggest moving that up, depending on how volatile your connections may be; we use 600 seconds).  This will cause the "gratuitous ARP" to occur at 530 seconds, and if the host answers, it perfectly repopulates the tables for your management tools to read.

Howard Hoxie adds (with a prompt from Gilles Schipper)

MPE caches the MAC for the gateway, and that there is a command that updates with an "INTERNET=@" or "INTERNET=ALL" clause.

(The command is NETCONTROL net=netname; update=all, where netname is probably LAN or LAN1)

Posted by Ron Seybold at 06:08 PM in Hidden Value, Homesteading | Permalink | Comments (0)

April 03, 2015

That final 3000 IO upgrade is still in use

Pass ThroughMore than five years after HP rolled out the ultimate release of MPE/iX, the vendor finished its work on an SCSI Pass-Through driver for the HP 3000. It was an one of the last HP-designed MPE enhancements. Independent support companies have the tech resources to create customized patches for their customers. The HP driver still makes it possible to connect and configure SCSI storage devices which HP has not certified for 3000 use.

Full instructions on how to use the software are on the ManualShelf free website. It's a tool for permitting an application to address SCSI devices without the use of the MPE/iX file system or high-level IO interfaces. But the software itself was built, lab-tested, then placed on the HP software improvements leash: It was only available to the HP support customer who was willing to take SPT, as HP called it, as a beta test version.

Patches MPENX01A, MPENX03A and MPENX04A were beta patches required to make the SPT work on MPE/iX 7.5. HP still makes these patches available to any 3000 customers at no charge. Two years ago, Allegro's Donna Hofmeister said "the magic incantation when dealing with the Response Center folks is to use transfer code 798. That’ll get you to an MPE person." 

Consultants and companies which provide support have many of these patches in their resource bins. The entire patch collection is just 1.27GB, small enough to fit onto a giveaway thumb drive.

HP always described the SPT as a tool with many caveats. While the software has a wide scope of help for the customer who wants to add the latest SCSI disk to the 3000 in the future, HP has printed plenty of cautions such as these in its Communicator article:

Sending SPT commands to a device in use by MPE or other applications may result in data loss, data corruption and/or System Aborts. We do not recommend sending SPT commands to Disks with MPE/iX Volumes present nor should one access tapes devices which are used for normal back-up or data logging purposes. Rule of thumb: Don’t do SPT to tape or disk media you cannot recover at a time you don’t wish to cause a system outage.

In 2007, SPT joined the six dozen or so HP developed enhancements and fixes stalled in beta test status. Customers still purchasing HP support were expected to test enough to get SPT into the full community as a general released patch, but few wanted to use beta-grade software in production.

Posted by Ron Seybold at 06:40 PM in Hidden Value, Homesteading | Permalink | Comments (0)

April 01, 2015

File transfer tips flow to move databases

System managers in the 3000 community still want to know how to use FTP to ensure a safe backup of 3000 data. Of special interest is the KSAM XL database, but most managers don't know that FAK files are HP's special Keyed Sequential Access Method database files. What appears to be program files are moved over, but database files get left behind. There's a trick to getting such files over to a Windows server.

One rule of 3000 operations is that database files act differently than all others in transfers. So FTPing them to a Windows 2003 Server won't be a successful way to ensure a safe data recovery. Third party tools can help, but if a customer is stuck on an aging HP system running MPE/IX, it's probably going to have only the budget for the included HP STORE for file backups and transfers.

Donna Hofmeister, who's spent a career helping 3000 users via the community's newsgroup, suggests starting with creating a file called mystd to store the 3000 files to disk -- then transferring that Store To Disk file.

 "First try this little experiment," Hofmeister begins in her advice.

    file mystd;dev=disc
    store a@.pub.sys;*mystd;show

If it works, you just stored/archived all the files that begin with 'a' in the .pub group of the .sys account into a file called 'mystd' (my store-to-disc).  You can expand the number of files being stored into your STD file by modifying your store command to:
    @.pub.sys -- all files in .pub.sys
    @.@.sys   -- all files in .sys
    @.@.acct  -- all files in .acct (for example)
    @.@.@     -- all files on the system (and it's actually better to say '/' instead of '@.@.@')

Keep in mind that as you increase the scope of what you're storing, so does the size of your STD file. In other words, to store the whole system you need 50 percent or more free space , which you probably don't have. So, break what you're storing into chunks (do one account at a time) and things should go smoothly.

If STD doesn't work, you might be able to get tar to work. The same space precautions apply.  One advantage of using tar is you should be able to verify the tar file on the destination system -- something you can't do with STORE without a 3000 in the mix.

3K Associates' Chris Bartram, who operates the 3000 information treasure-house 3k.com, added explicit advice about FTP and the 3000's files.

If you package all the MPE files up in either a store-to-disk (aka std) or a tar "wrapper" (disc file) you can transport that file around at will -- as a BINARY file -- don't try to transfer it as ASCII or CR/LF translations will trash it.

Once you get it back on the 3000, a simple file equation directing your source (tape) to the new (std) file name (and add the ;DEV=DISC to the file equation) will allow you to restore the files onto the 3000 preserving all the MPE specific file attributes they started with. Tar will work similarly for almost all MPE files, but can't handle database/PRIV files and probably not MSG (message) files and a few other very MPE-specific files.

tar works the same way on MPE as on *nix boxes. But is much more "familiar" if you run it from the Posix shell (sh.hpbin.sys), though that's not necessary. Treat the tar file just like you would on a *nix box. 

For store-to-disc files, you use the same MPE syntax for storing files as you do normally; the only difference is that the output device is file-equated to ;dev=disc. As mentioned, be aware of the disc space required to store another copy of your backed-up files online. 

Likewise, when you restore instead of pointing to tape, you point to the disc file -- and don't forget to add ;dev=disc to that file equation as well. If the store-to-disc files are going to be very large (several gigabytes) you can use some additional syntax to break them into chunks - but hopefully you needn't worry about that for now. 

Treat a store-to-disc file just like a tar file. Record size and most other attributes aren't so critical, but if you move it around do NOT let FTP transfer it in ASCII mode or it will corrupt the file.

As for examples; I back up my primary HP 3000 to a disc file then transfer (FTP) it to a Linux server. Here's the gist of my JCL:

!FILE t=FULLB.PUB.BACKUP;DEV=disc
!CONTINUE
!STORE / - /BACKUP/ ;*T &
! ;SHOW=SHORT,dates,path;progress=5;directory;tree;&
! onvs=MPEXL_SYSTEM_VOLUME_SET,WORK;STATISTICS;
!COMPRESS=HIGH


!ftp.arpa.sys
open ftpserver
user userid password
del hp3000-full
binary
put FULLB.PUB.BACKUP hp3000-full
quit

Posted by Ron Seybold at 05:59 PM in Hidden Value | Permalink | Comments (0)

March 26, 2015

Checkup Tips to Diagnose Creeping Crud

When an HP 3000 of the ultimate generation developed trouble for Tom Hula, he turned to the 3000 newsgroup for advice. He'd gotten his system back up and serving its still-crucial application to users. But even after a restart, with the server looking better, things just didn't seem right to him. 

I am concerned, since I don't know what the problem was. It almost reminded me of something I used to call the Creeping Crud, where people started freezing up all over the place, while some people were still able to work. The only thing was a reboot. But in this case, it seemed worse. Only a few people on our 3000 now, but we still depend on it for a high-profile application. What should I check?

CrudThe most revealing advice came from Craig Lalley, who told Hula he'd try a Control-B into the 3000's system log. The steps after the Control-B command are SL (for System Log) and E (for Errors only.) Typing CO puts the 3000 back in console mode. Hula's system had lost its date and time on one error, and the Alert Levels showed a software failure along with lost boot functionality.

But amid the specifics of eliminating the Creeping Crud (it may have been a dead battery) came sound advice on how to prepare for a total failure and where to look for answers to 3000 hardware problems. The good news on the battery is that it's not in a Series 9x7. Advice from five years ago on battery replacement pointed to a hobbyist-grade workbench repair. More modern systems like Hula's A400 at least have newer batteries.

Using a DSTAT ALL was suggested, as well as checking the status of available storage with DISCFREE. Mark Ranft said "I would make sure there's a good full backup. (Just in case you need it for recovery.)  If you don't have one, doing one may help identify a disk issue. I would check the system logs especially for disk errors. I would check for network errors, using linkcontrol @,all." He shared his own recovery experience.

I had a system acting strangely this past weekend. It was basically hung but allowed new logons. I could not abort anyone. When I got to the point where I tried to stop the network, I got a system abort 1458 from Subsystem 102. I didn't bother to take a dump. I completed the boot and everything was better.

Chuck Trites reminded Hula to create a current CSLT tape, and "run BULDJOB to create the BULDJOB1 and 2 files — in case you need to recreate the accounting structure and UDCs — and store them to tape."

Hula's own check list included the following:

During the reset, the 3000 got up to the date and a little past and seemed frozen. I pulled the plug and restarted again. It took 2-3 times as long as normal and at first, the red fault light was on (I never saw that on before). After it got a bit into the restart, the fault light turned off by itself. The only attention message I got about the whole thing was a message with everything unknown on the 3000.

When the computer came all the way up, it still seemed very sluggish. I scheduled the nightly update and backup and went home to look at it more in the morning. I logged on from home and the backup seemed to be running okay.

This morning I tried resetting the GSP and checked the connections to the console terminal. I also found out that someone else had a hard time getting on the 3000 towards the end of the day. Very sluggish. But this morning, everything seems back to normal.

Hewlett-Packard's hardware builds have been extraordinary, but a server that's been churning out critical data for more than 12 years (A-Class boxes production stopped in 2003) can develop crud. Something as simple as replacing a dead battery might be the answer to the woes. Advice for the crud also came from Gilles Schipper, Jack Connor and the others mentioned. What they've got in common is working in a support practice, or at least a consulting business that includes 3000 sites.

Self-maintenance is common in a community like the 3000's. It's also a good practice to have a support vendor, one who knows the system as well as the volunteers posting to the newsgroup.

Posted by Ron Seybold at 09:20 PM in Hidden Value, Homesteading, User Reports | Permalink | Comments (0)

March 17, 2015

Tips to Reinstall Posix, DLT/LTO Tape Drives

What is the patch that installs Posix? I seem to have a corrupt version of Posix.

Donna Garverick of Allegro replies:

These are your instructions for MPE/iX 5.5 and 6.0.

Load the 5.5 or 6.0 FOS tape on a tape drive. For this example, tape drive on ldev# 7 is used. Log on as MANAGER.SYS

:FILE T;DEV=7
:RESTORE *T;@.HP36431.SUPPORT,I0036431.USL.SYS;CREATE;SHOW
:STREAM I0036431.USL.SYS

Please note:

  • HP36431 is the master product number of the Posix 2 Shell.
  • I0036431.USL.SYS is the installation file.
  • When launched, the job I0036431 should run for less than 5 minutes. When it is done, the Posix environment is re-installed.

[Gilles Schipper notes the process for 7.5 is the same, working from the MPE/iX 7.5 FOS tape.]

I have access to a Tandberg Data Ultrium LTO 3 tape drive. It has a SCSI Ultra160 interface. Would I have any luck hooking one up to an N-Class?

Chad Lester of MPE Support Group replies:

It's worth trying. You might have issues with the dual-port SCSI cards. Also, make sure the firmware is the latest on the single SCSI U160 card.

We recently upgraded our customers to the hot-swappable LTO drives designed for the TA-5300 Array. The array is $350 with an SCSI Cable. Two Q1540A LTO 3s are $1,350, for a grand total of $1,700. That includes phone support from us for installation. 

I have a DLT4000 that will connect to an HP 3000 on path 32.2.0. How do I set the 12 dip switches on the back of the DLT for this path?

Mark Ranft of Pro3K replies:

Off of Google, I’ve found this:

For HP 7980S emulation, 

     1,2 - OFF

     3,4,5 - ON

     6,7,8,9 - OFF

     10,11,12 - SCSI ID (suggest all OFF)

So 10 is the 4's place
And 11 is the 2's place
And 12 is the 1's place

If 10, 11 and 12 are off (down), the SCSI ID will be zero.

Gilles Schipper of GSA adds:

I believe there are two different versions of DLT4000. One has a single-ended SCSI interface and the other has a FW-SCSI interface. I think the one Mark described is for the former.

Posted by Ron Seybold at 05:18 PM in Hidden Value, Homesteading | Permalink | Comments (0)

February 16, 2015

Classic MPE tips: Tar, kills, and job advice

How do I use the tar utility to put data onto tape on an HP 3000?

1) Create a tape node

:MKNOD “/dev/tape c 0 7”

2) Enter posix shell

:SH -L

3) Mount a blank tape and enter the tar command

shell/ix>tar -cvf /dev/tape /ACCOUNT/GROUP/FILENAME

How can I determine the validity of an SLT tape?

Use CHECKSLT.MPEXL.TELESUP option 1.

What is the command to abort a hung session? I tried ABORTJOB #s3456. I seem to remember there is a command that will do more.

You can use =SHUTDOWN. But seriously, there is a chance that if it is a network connection, NSCONTROL KILLSESS=#S3456 will work. If it is a serial DTC connection, ABORTIO on the LDEV should work. Finally, depending upon what level of the OS you are on, look into the ABORTPROC command. This might help as a last resort.

I recently had a perfect application for the NEWJOBQ feature. We have two groups of users. One group submits jobs that take about 30 seconds each. Typical jobs for the other group take about 5 minutes each. So I thought I’d give the second group of users their own job queue.

NEWJOBQ ALTJOBQ;LIMIT=1
LIMIT 1 (for HPSYSJOBQ)

When I submit a long job into the ALTJOBQ queue, and a quick job into the default job queue, the second job goes into the WAIT state. Why?

Your NEWJOBQ statement is correct, but your second statement didn’t do what you thought. To put a limit of one on the HPSYSJQ job queue, your statement should read 
:LIMIT 1;JOBQ=HPSYSJQ.

By saying :LIMIT 1, you are changing the total job limit on the system to one. Since the total limit is one, and the long job in ALTJOBQ is still running, the second job waits even though he is the only job in his queue.

What does HPSWINFO.PUB.SYS show? All software or only installed software? How do I find out what HP software is installed?

Generally speaking, HPSWINFO.PUB.SYS is a record of system software level and patching activity. If you want information on HP software installed then you want to run psswinvp.

How can I sync the time on my 3000 with my Windows network? The PC side does regular, automated sync to NIST.

First, ensure your timezone is absolutely correct (:setclock/:showclock) and you have a system logon UDC to setvar TZ to the correct timezone.

Install NTP and use the ‘ntpdate’ function to sync your clock to the PC servers. Do this in a batch job that issues the ntpdate command, and then :STREAMs itself;IN=xx to periodically perform the synchronization.

Posted by Ron Seybold at 11:15 PM in Hidden Value, Homesteading | Permalink | Comments (0)

February 10, 2015

Multiple Parallel Tapes on 3000 Backups

Editor's note: When I saw a request this week for a copy of HP patch MPEMX85A (a patch to STORE that enables Store-To-Disk) for older MPE/iX releases, it brought a storage procedure request to mind.

I'm dealing with some MPE storage processes and need assistance. You would think after storing files on tapes after 10-plus years, we would have found a better way to do this. We use TurboStore with four tape drives and need to find a way to validate the backup. Vstore appears to only have the ability to use one tape drive. Currently I have some empty files scattered through the system and use a separate job to delete them, remount the tapes and restore, trying to access all four drives. 

When using vestore:

vstore [vstorefile] [;filesetlist]

It seems that vstorefile is looking for a file equation similar to:

File t; dev=tape
vstore *t;@.@.@; show

This is why it appears that I can't use more than one tape drive, unless they are in serial, while we want to use four drives in parallel. What method or software should I be using?

Mark Ranft of Pro3K replies:

We always found that DLT 8000 tapes worked well in parallel. When the backup got so big that it wouldn't fit on two DLT 8000 tapes, we split the backup, putting the databases on two tapes in parallel and everything else on a third tape. Keep in mind, we didn't have a backup strategy. We had a recovery strategy and backups were a part of that. We found, for us, organizing backups in this manner allowed us to speed recovery — which was far more important than anything else.

You can achieve good times doing Store-to-Disk backups. But then what? Do you back up the STD to tape and send it offsite? FTP it somewhere? The recovery times on getting this back are too slow.

Tracy Johnson adds

I think you can use VSTORE to read multiple tape drives in parallel or series using the ;RESTORESET parameter.

So you make four file equations.

Drop the beginning file single backreference to a equation (like we learned in olden times), and put the four new ones with the ;RESTORESET= parameter instead. It is one of those things that fooled me first time I saw it, and it took about 10 minutes getting used to seeing it.

The parenthesis around the file equations are placed differently:

Serial:

 ;RESTORESET = (*tape1,*tape2,*tape3,*tape4)

Parallel:

 ;RESTORESET=(*tape1),(*tape2),(*tape3),(*tape4)

But if the tapes were not also created in parallel, it may not help in the latter case.

Ray Legault adds

I use three DLT8000's and run a Vstore every week.

! setvar _drive "(*p1),(*p2),(*p3)"
!#
!vstore ;@.@.@;restoreset=!_drive;show;progress=5;nodecompress
!#

Posted by Ron Seybold at 09:35 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 26, 2015

How to Use MPE/iX Byte Stream Files

Back when HP still had a lab for the HP 3000, its engineers helped the community. In those days, system architect and former community liaison Craig Fairchild explained how to use byte stream files on the 3000. Thanks to the memory of the Web, his advice remains long after the lab has gone dark.

Mountain-streamThese fundamental files are a lot like those used in Windows and Linux and Unix, Fairchild said. HP has engineered "emulation type managers" into MPE/iX, an addition that became important once the 3000 gained an understanding of Posix. In 1994, MPE/XL became MPE/iX when HP added this Unix-style namespace.

Understanding the 3000 at this level can be important to the customer who wants independent support companies to take on uptime responsibility and integration of systems. Fairchild explained the basics of this basic file type.

Byte stream files are the most basic of all file types. They are simply a collection of bytes of data without any structure placed on them by the file system. This is the standard file model that is used in every Unix, Linux and even Windows systems.
 
MPE's file system has always been a structured file system, which means that the file system maintains a certain organization to the data stored in a file. The MPE file system understands things like logical records, and depending on the file type, performs interesting actions on the data (for example, Circular files, Message files, KSAM files and so on).

Fairchild detailed how HP has given bytestream files the knowledge of "organization of data" for applications.

To bridge the gap between standard byte stream file behavior (only the application knows the organization of data) and traditional MPE file type behavior (the file system knows what data belongs to what records), emulation type managers were created. To an MPE application, a byte stream file looks and behaves like a variable record file, even though the data is stored in a way that would allow any Posix application to also read the same data. (Posix applications also have emulator type managers that allow them to read fixed, variable and spool files in addition to plain byte stream files.)

The way that the byte stream emulator detects record boundaries is through the use of the newline (\n) character, which is used, by convention, to separate data in ASCII text files on Unix-based systems.

The underlying properties of a byte stream file are that each byte is considered its own record. In MPE file system terms, a record is the smallest unit of IO that can be performed on a file. (You can write a partial record fixed length record, but the file system will pad it to a full record.) Since the smallest unit of IO that can be performed on a byte stream file is a single byte, that becomes its MPE record size.
 
In the MPE file system, the EOF tracks the number of records that are in a file. Since the record size of a byte stream file is one byte, the EOF of a byte stream file is also equal to the number of bytes in the file. This is why one 4-byte variable sized record is equal to 5 byte stream records (4 bytes of data + 1 \n character).

It's also worth noting that any file can be in any directory location and will behave the same way. (Well, almost. CM KSAM files are restricted to the MPE namespace. And of course the special files (that you don't normally see) that make up the file system root, accounts and groups are also restricted: one root, accounts as children of the root, groups as children of accounts. And lockwords aren't allowed outside the MPE namespace. But other than that, the opening sentence is true.) 

The general model that we had in architecting the whole Posix addition was that behavior of a file does change regardless of where it is located. This was summed up in the saying, "A file is a file." So there are no such things as "MPE files" and "Posix files". There's just files.

What does change is the way you name that file. Files in the MPE namespace can be named either through the MPE syntax (FILE.GROUP.ACCOUNT), or through the HFS syntax (/ACCOUNT/GROUP/FILE). You can also use symbolic links to create alternate names to the same file. This was summed up as a corollary to the first saying, "But a name is not a name."

Posted by Ron Seybold at 09:47 PM in Hidden Value, Homesteading | Permalink | Comments (0)

January 06, 2015

Essential Steps for Volume Reloads

When a 3000 drive goes dead, especially after a power outage, it often has to be reloaded. For example, when an LDEV2 has to be replaced. For a cheat sheet on reloading a volume, we turned to our Homesteading Editor Gilles Schipper.

By Gilles Schipper

Assuming your backup includes the ;directory option, as well as the SLT:

1. Boot from alternate path and choose INSTALL (assuming alternate path is your tape drive) 
2. After INSTALL completes, boot from primary path and perform START NORECOVERY. 
3. Use VOLUTIL to add ldev 2 to MPEXL_SYSTEM_VOLUME_SET. 
4. Restore directory from backup (:restore *t;;directory) 
5. openq lp
6. Perform a full restore with the following commands
:file t;dev=7(?)
:restore *t;/;keep;show=offline;olddate;create;partdb;progress=5 7.

Perform START NORECOVERY

I would suggest setting permanent and transient space each equal to 100 percent on ldev 2. The 75 percent default on ldev 1 is fine as long as you don’t need the space. And if you did, your solution shouldn’t really be trying to squeeze the little extra you’d get by increasing the default maximum limits.

The reason for limiting ldev1 to 75 percent is to minimize the otherwise already heavy traffic on ldev 1, since the system directory must reside there, as well as many other high traffic “system” files.

You won't want to omit the ;CREATE and ;PARTDB options from the restore command. Doing so will certainly get the job done -- but perhaps not to your satisfaction. If any file that exists on your backup was created by a user that no longer exists, that file (or files) will NOT be restored.

Similarly, if you omit the ;PARTDB option, any file that comprises a TurboIMAGE database whose corresponding root file does not exist, will also not be restored.

I suppose it may be a matter of personal preference, but I would rather have all files that existed on my disks prior to disk crash also exist after the post disk-crash RELOAD. I could then easily choose to re-delete the users that created those files -- as well as the files themselves.

Another reason why the ;SHOW=OFFLINE option is used is so that one can quickly see the users that were re-created as the result of the ;CREATE option. Purging the "orphan" datasets would be slightly more difficult, since they don’t so easily stand out on the stdlist.

Finally, it’s critical that a second START NORECOVERY be performed. Otherwise, you cannot successfully start up your network.

Posted by Ron Seybold at 10:37 PM in Hidden Value, Homesteading | Permalink | Comments (0)

December 10, 2014

Getting Macro Help With COBOL II

GnuCOBOL An experienced 3000 developer and manager asked his cohorts about the COBOL II macro preprocessor. There's an alternative to this very-MPE feature: "COPY...REPLACING and REPLACE statements. Which would you choose and why?"

Scott Gates: COPY...REPLACING because I understand it better.  But the Macro preprocessor has its supporters. Personally, I prefer the older "cut and paste" method using a decent programmer's editor to replace the text I need. Makes things more readable.

Donna Hofmeister: I'm not sure I'm qualified to comment on this any longer, but it seems to me that macros were very efficient (and as I recall) very flexible (depending on how they were written, of course). It also seems to me that the "power of macros" made porting challenging. So if your hidden agenda involves porting, then I think you'd want to do the copy thing.

There was even porting advice from a developer who no longer works with a 3000, post-migration.

Tony Summers: When we migrated in 2008 we chose Acucobol partly because of its HP 3000 compatibility, including macro support. However had we gone down a different route, I had already proved that I could pre-process the raw code myself and expand the macros before calling the compiler.

Robert Mills, who started the discussion on the 3000-L, said in reply to Donna, “I admit that I do have a hidden agenda, but the main reason does not involve porting.”

For many years I have used macros to make my life easier. When I left the e3000, back in 2008, and did some work on other platforms I found I missed them. I'm now in semi-retirement and have been using the free version of Micro Focus COBOL (a couple of years) and GnuCOBOL (this last year) to write software for friends, family and my own use.

A couple of times since 2008 I had thought of writing by own macro preprocessor to emulate the one on the e3000. A few months ago I decided to do it and release it as open source under the GNU GPL. The development of preprocessor, using GnuCOBOL, is now completed and in final Beta Testing and I'm writing the manual. Was hoping that I could some additional reasons, from others, as to why you would use macros instead of the copy...replacing and replace statements.

Because a port of GnuCOBOL is a available on several platforms, and my preprocessor is written in GnuCOBOL, I see no problem in taking my macros with me nearly every wherever I go. If I end up doing work on a platform that does not support a feature that it is using it shouldn't be to difficult to develop a workaround.

As it turns out, GnuCOBOL is a newer version of OpenCOBOL — a compiler that Donna says bears a close resemblence to COBOL II. (OpenCOBOL has been ported into a commercial product, too, called IT-COBOL.) Adding that she obviously thinks macros are cool, she explained.

Do my mis-firing neurons recall that GnuCOBOL was formerly OpenCobol... which was actually very close to MPE’s COBOL?  (or something like that?)

I inherited a outstanding collection of macros at one job. Many of them were 'toolbox' functions.  Want to center a string and the overall length of the string doesn't matter?  Got a macro for that.  Want to use a 'db' call? Got a macro for that.  These went far beyond modifying code at compile time -- and that's what made them so valuable (at least to me).

Posted by Ron Seybold at 06:10 PM in Hidden Value, Homesteading, User Reports | Permalink | Comments (0)

December 08, 2014

IMAGE data schemas get visualized

Is there any program that will show the network of a TurboIMAGE database? I want to output the relationships among sets and items.

CFAWireframeIn 2011, Connie Sellitto researched the above question, while aiding new programmers who were charged with moving a pet organization's operations to a non-MPE system. Understanding the design of the database was important to this team. Sellitto mentioned a popular tool for PCs, but one not as essential as an IT pro's explanations.

You might try Microsoft's Visio, and you may need to have an ODBC connection to your IMAGE database as well. This produces a graphical view with search paths shown, and so on. However, there is still nothing like a detailed verbal description provided by someone who actually knows the interaction between datasets.

To sum up, we can refer to ScreenJet founder's Alan Yeo's testing of that Visio-IMAGE interplay

Taking a reasonably well-formed database into Visio and reverse engineering, you do get the tables and items. It will show you what the indexes in the tables are, but as far as I can see it doesn't show that a detail is linked to a particular master. Automasters are missing anyway, as they are really only for IMAGE.

My conclusion: if you have done all the work to load the databases in the SQL/DBE and done all the data type mappings, then importing in Visio might be a reasonable start to documenting the databases, as all you would have to do is add the linkages between the sets.

If you don't have everything in the SQL/DBE, then I would say we are back where we started.

ScreenJet knows quite a bit about moving 3000 engineering into new formats. It built the EZ View modernization kit for 3000 user screens that are still in VPlus. Yeo said the ubiquitous Visio might be overkill for explaining relationships.

If you have Adager, Flexibase, or DBGeneral -- or already have a good schema file for the databases -- just generate the schema files and import them into Word or Excel and give them to [your migrators]. If they can't put together the data structure from that, no amount of time you can spend with Visio is going to impart any more information.

Visio has free and open source competition, software which HP support veteran Lars Appel pointed out. "Perhaps Visio has similar 'database graph' features, such as the free or open source tools like dbVisualizer or SquirrelSQL."

Barry Lake of Allegro pointed out that users "may want to take a look at Allegro's DBHTML product, which creates a browser viewable HTML file documenting the structure of an IMAGE database." Allegro's site has an example DBHTML output on its website, although it doesn't draw pretty pictures.

At a more fundamental OS level, Michael Anderson points out to understand the structure of a TurboIMAGE database, "you could use QUERY.PUB.SYS, then issue the command FO ALL, or FO SETS."

Posted by Ron Seybold at 10:31 PM in Hidden Value, Homesteading | Permalink | Comments (0)

November 25, 2014

Open source SED manages 3000 streams

Open source resources make it possible to use SED, a stream editor built in the open source community. Since 2001 SED has worked on the HP 3000, thanks to Lars Appel, a former HP support engineer who ported Samba to the platform in the 1990s.

SED's main MPE page is on a page of Appel's. SED is an at your own risk download, but support is available through the 3000 community.

Dan Barnes, working on a problem he had to solve in his 3000 environment, asked:

The issue is incoming data from another platform that is being fed into MM 3000. This data occasionally has some unprintable characters, which of  course wrecks havoc on the MM application when it is encountered. To address this, the user, using a cygwin (Unix-like) environment on their Windows PC, developed a SED script. When they test the script in the cgywin environment it works just fine. But when done on the target HP 3000 it gets an undesirable result.

Barnes added that "The user thought that because MPE/iX is Posix-compliant, that this should work." He explained his user created the expression

sed -e 's/[\x7F-\xFE]/*/g' < COMSHD > COMSHD1

But Appel noted that hex 7F thru hex FE portion of the expression isn't supported on the MPE/iX version of SED. It's a limitation of MPE/iX, but there's a workaround.

Not sure if the regular expression usage here matches Posix or GNU specs, but my guess is the "\xNN" format, that seems to indicate a char by hex code, doesn't work.

How about something like using the command sed -e 's/[^ -~]/*/g' instead, i.e. map the characters outside the range space through tilde?

Posted by Ron Seybold at 10:05 AM in Hidden Value, Homesteading | Permalink | Comments (0)