Who's to mind the CALENDAR?

Last year we took note of the HPCALENDAR intrinsic and its ability to create accurate timestamps for decades to come on the HP 3000. The intrinsic isn't new, though, even though HP advised its customers in November to begin using it on HP 3000s.

No, HPCALENDAR harks back to version 5.5 of MPE/iX. Its power lies in the 3000 for use by programmers who want accurate dates beyond 2038 for application files. But the operating system itself? It continues to use the old CALENDAR intrinsic, which only gives an accurate timestamp to 2027.

Is it foolish to be considering the timestamping ability of a 3000 some 19 years into the future? HP must have thought so while it made technical decisions for this system over the past seven years, knowing the vendor would step out of the 3000 community. You see, HPCALENDAR was never integrated into the operating system itself.

Now, with the 3000's development labs closed down, the community can wonder who'll keep the calendar functions up to date for MPE/iX.

Vesoft's Vladimir Volokh called to update us on the CALENDAR mistake, based on an error we made in our November printed issue. Although I carefully reproduced all of the HP technical details about using HPCALENDAR, a "display quote" on the page didn't get the facts correct"

The newer intrinsic extends the 3000's date accuracy for more than 30 years beyond 2008. 2038 will be the last year to accurately store timestamps.

Actually, it's Unix that's going to lose the ability to store timestamps accurately by 2038. Volokh explained that since HPCALENDAR uses 23 bits to store timestamps, there are 8.3 million places to store a date. If only HPCALENDAR had been wired into MPE/iX, instead of just available for application programmers to use as an intrinsic.

3000 goes in an open direction

More than 11 years ago, HP was teaching HP 3000 skills to the world. George Stachnik, an HP employee who communicated 3000 advantages to customers, wrote a series of articles for HP 3000 newbies. In an early part of his series that started in 1997, he summed up HP's view of the system's future (Where's the HP 3000 Going?) as the company saw it back then.

The evolution of the HP 3000 has been driven by the open systems revolution that swept across the IS industry beginning in the 1990s. By 1990, most new computer applications and technologies were being developed on (and for) Unix computers. This trend threatened to leave proprietary architectures like the HP 3000 out in the cold.

In response, HP began bringing industry standard interfaces from the HP 9000 to the HP 3000, focusing first on functions that were standardized by IEEE’s Posix committees. Version 4.0 of MPE XL was renamed to“MPE/iX” (the iX stands for “Integrated PosiX”). The Posix functionality made it easier than it had been to port software from Unix to the HP 3000. Other industry standards (BSD Sockets, SQL, ODBC, Java) have been brought to the 3000 by HP in subsequent OS releases. All this open systems functionality has continued to be enhanced on subsequent releases.

Of course, that Posix functionality remains in MPE after seven successive releases. HP has not eliminated much from the 3000's feature set after more than 30 years of development. Posix makes the HP 3000 behave like Unix systems. HP was betting in 1997 that this similarity could preserve the system. Even though HP shifted its bets four years later about the 3000, using the Posix shell is a way to get an IT staffer introduced to the 3000 from a Unix perspective.

Consider that this weekend starts the eighth year of 3000 survival after HP changed its bet. Adding Posix may not have had the effect HP intended for the vendor's 3000 business. But it edged the system into open source, which could be a key to surviving another seven years.

It's good to remember how much hope HP projected, as well as how much effort the supplier made, here at the end of the seventh year of The Transition. Keeping the system in growth mode was a challenge too complex for Hewlett-Packard to meet. HP had failures in the past with the 3000, like the abortive System 3000 introduction in 1972.

Stachnik explained how Posix would change interfacing with a 3000 in his article. But he underlined the design choices that make this computer a lasting value for those who are staying with it, as well as those taking longer than expected to leave it.

Many computer vendors say that their systems software is “tuned for transaction processing” but in the case of the HP 3000, this is no idle claim. A tremendous amount of R&D work was done at HP to understand exactly what kinds of stresses are placed on computer systems by commercial transaction processing workloads. And the payoff from this R&D was an HP 3000 that was tuned for the best possible performance.

HP got its payoff in open source applications not long after Stachnik's article, earnings that continue to deliver today in DNS services harder to hack than any "industry standard" system, Samba file sharing and more. It all began with an integration of Unix into the HP 3000, differences Stachnik explained in an accompanying article. Have a look at what he wrote, one of the "3000 for Dummies" lessons which continues to teach, here at the end of the seventh year of migrations. HP was directing this system out of the cold in the 1990s. It's still warm to the touch today.

Keep the CALENDAR up to date

The year 2027 has been notable for customers who don't plan to leave the HP 3000. That's the year when timestamps stop being accurate, because the CALENDAR intrinsic in MPE/iX only uses 7 bits to store year information.

If your HP 3000 apps are using CALENDAR, HP advises that you use the newer HPCALENDAR. The newer intrinsic extends the 3000's date accuracy for more than 30 years beyond 2008. Yes, that's right; 2038 will be the last year to accurately store timestamps.

HP's advisory, which got referenced by its support and patch tracker today, explains the differences. At least in part:

The original MPE timestamp format was that used by the CALENDAR intrinsic, a 16 bit quantity allowing 9 bits for the day of the year and 7 bits for the year, added to 1900. Since the largest number represented by 7 bits is 127, this format is limited to accurately storing years up to 2027.

The newer HPCALENDAR intrinsic uses a 32 bit quantity, allowing 23 bits for the year, since 1900 and the same 9 bits for the day of the year. This format provides a significantly longer period of timestamp accuracy.

When HP began to talk about a Posix timestamp function that works on the 3000, the advice needed a bit of explanation from HP's 3000 lab engineer Bill Cadier.

Cadier reports

If, for example someone needs to store the maturity date for a 30 year mortgage started this month, neither the traditional CALENDAR format nor the time() format will work as they are only accurate to 31 December 2027 and 19 January 2038 respectively. The HPCALENDAR date format provides 23 bits to store the year added to 1900 — and since one can store 8,388,607 in those 23 bits, this format provides the best accuracy for storing future dates on the e3000.

The advisory, which you can read for yourself at the HP IT Response Center Web site, says in part

Certain POSIX applications may use the time() function as the basis for timestamps; and may therefore, store timestamps in the format used by time(), which is a 32 bit quantity representing the number of seconds from the epoch 1 January 1970. This format is limited to accurately storing timestamps up to 19 January 2038.

If your applications have a need to create and store future transaction timestamps, HP recommends using HPCALENDAR, HPDATECONVERT, HPDATEDIFF, HPDATEFORMAT or HPDATEOFFSET to ensure they are created correctly.

HP built MPE to an extraordinary level of durability. Not even Unix, which relies on the time() function, is going to be able to handle dates as long as MPE/iX, using the invented-in-HP HPCALENDAR intrinsic. This is a good example of why vendor engineering, beyond industry standards, gives "legacy" platforms a longer life — sometimes longer than even the vendor estimates.

The 3000: Always New to Someone

Even a computer more than 15 years old can appear new. When this happens — like when a company's lone IT manager first takes on a 3000, still running critical apps — then MPE/iX and the PA-RISC hardware seem like unfamiliar territory.

But this server of more than three decades has a mature community of experts online, cruising the newsgroup for the HP 3000. In messages on comp.sys.hp.mpe, a team of veterans led a newbie through the fundamentals of 3000 use.

"What a blast the past couple weeks have been," said Paul Raulerson. "I have fallen in love with this little HP 3000. I've been coming home from work and playing with it, learning and having a grand old time. Even running RPG programs on it. Great fun! I've been having an argument with it about talking on the network, and just got it completely up and operating."

Paul had questions a-plenty about his 9x7 system — a generation of HP 3000, by the way, which is still running some companies around the world.

Raulerson said,

Tapes are a fascinating subject — how might one read the directory off a tape? I have a few tapes that were sent with the machine, and absolutely no clue of how to find out what is on them.

Also, the manual talks about labeling tapes, but I think it means actually physical labels! Do I need to to somehow initialize a tape with a standard, ANSI, or no label format?

And speaking of which, can I back up the entire system to a tape? Do I need to do that with an ISL utility?

Alternatively, can I attach some external DASD and back up the system volume to one of those?

Craig Lalley answered,

TAPEDIR displays the contents of CM STORE tapes. TAPEDIR is a subset of the TINDEX Tool, which handles NM and CM STORE tapes, TAR tapes, SLT/SYSGEN tapes, memory dump tapes, and others (LZW, std, plain text information).

This nice little utility is brought to from our good friends at Allegro. (Well known name in our little industry/)

You can download TAPEDIR from www.allegro.com/software/hp3000/stuff/TAPEDIR.std

But if you want to have fun, look at all the other “cool” stuff.

Gary Robillard added,

If the tapes turn out to actually be MPE/V format tapes, the VALIDATE program can list the files on an MPE V/E store format tape.

If you have the validate utility it would be in the PUBXL group in the TELESUP account.  To get information about the program, run the program with the HELP entrypoint (the command would be RUN VALIDATE.PUBXL.TELESUP,HELP).

There is also a utility in the ROBELLE account named TAPEDIR.QLIB.ROBELLE (to get info about it RUN TAPEDIR.QLIB.ROBELLE,HELP)

Finally, OpenMPE director Tracy Johnson explained,

Sometimes third party backups will appear as MPE/V format.  Backpack (a.k.a. Roadrunner) tapes may do this. Rather than belabor whether it is an MPE/V tape, it may be easier just to see if it is a third party backup tape. Just go ahead and do a RESTORE of two possible files:

FILE T;DEV=7
RESTORE *T;BACKREST.PUB.SYS,ROADREST.PUB.SYS;SHOW

If either one of these files appear on the RESTORE, then it is a tape made by a third party backup utility. Both Backpack and Roadrunner are programs (made by two incarnations of the same company) that will perform tape restore operations on these tapes only.

Yes, HPSUSAN, there is a transfer

When buying an HP 3000, how do you transfer HPSUSAN numbers legally?

OpenMPE director Donna Hofmeister replies:

I believe that you do not transfer MPE licenses. If you buy a 3000, you buy the license as well.

When another community member explained that HP transfers HPSUSAN numbers to new systems, Lars Appel, a former HP support engineer now consulting with Marxmeier Software, added:

Keep in mind, even if you arrange to move the HPSUSAN number from the old system to the new system... this might not solve all your license code issues with third party applications... because some of them might also check HPCPUNAME if the license depends on the system “size” as well... and your HPCPUNAME might be different when your new system is not of the same model and CPU count.

How should parts and entire HP 3000 systems be stored if stockpiled?  What ration of parts will be functional after being stored for 6-24 months?  What will be the level of workability for parts and systems after 2008?

Donna Hofmeister replies:

I’m not sure you (as a company) should be worrying about stockpiling hardware unless you’ve got some circumstance (like being based in the Yukon or something) that warrants doing so.  The hardware support vendor that you partner with should being doing this for you. This is an example of the kind of question you need to ask your hardware partner.

Hofmeister went on to deliver some supplemental hardware advice to help keep a 3000 running uninterrupted.

Having said that, I’ll urge you as strongly as possible to look at upgrading your disc. It’s probably the one component most likely to fail on your box — especially if you’re still running internal 4Gb or 8Gb drives. If your drives have been in service for more than 8-10 years (did you put in new disc to upgrade to 5.5 for Y2K?) in my opinion you’re skating on thin ice! It’s a matter of time before you’ll be changing disc in a panic instead easily over some weekend. So ask your vendor — do you have staff that can assist with a disc migration?

I’ll add in one other comment.  Do not even think of trying to homestead without hardware support. Sure, it looks cheap to buy hardware only when something breaks. However, if you come to your local hardware vendor five years from now wanting a replacement “x” and you’ve no previous relationship with this company, I wouldn’t much blame them for laughing uproariously after you get off the phone. On the other hand, by establishing a relationship (e.g., contract = spending money) with them now, you can jointly determine your best course of action.

Programming Made Easier

Historic facts can expire, their sell-by dates causing what we know to become untrue. Take the history of the HP 3000's advances. In 2000, HP's pledge to take MPE/iX onto the Itanium architecture was already history, since the vendor made the promise several years earlier. Then in late 2001, well, that history became invalid, and to some customers, simply untrue. But some artifacts of history hold facts that remain true no matter what their date, especially if you own or operate a 3000 of any vintage.

Durable truth is hard to come by in the computer industry. So much is paved over every year that knowledge becomes arcane quickly in the name of advances. But consistency is also a value worth preserving, and so a good share of the 3000 community is still using the system HP built, then dropped from its 21st Century sales plans.

That constant use is what makes a recent addition to our archives more than a relic. Today we received a copy of the Using The HP 3000 "an introduction to interactive programming," circa early 1979. (Thanks to Roger Smith, IS Director of Tulare County Office of Education, for the addition; click on any photo here for a larger version.) In that springtime of 1979, the HP 3000 had two means of interactive access: the 2645A terminal and a hardcopy-only cousin, the 2635. But the commands from that MPE III version of the OS still run today, nearly 30 years later.

That's more than historic. It borders on legendary — but it's also why HP had to admit the 3000 business was too big for it to maintain. Too large in time-span, anyway.

HP wrote this book for the "professional computer programmer" as well as "someone who has never seen a computer before. And we know from experience that both categories are well represented in the HP 3000 user base."

To be sure, the last part of that sentence will be viewed as history. You may not be able to find someone who has never seen a computer before. However, it's not that hard to find someone who has never seen a business server before, and that's what kind of computer the HP 3000 remains today.

The manual is fun, and full of reminders of how much easier programming has become in 29 years' time. The sections on how to delete a line or characters within a line make me wonder how anyone had time to compete a project. But then projects deadlines were measured in months instead of weeks for most customers. Plus, completing a project on the first attempt was a genuine measure of success. Still, all that control-X and control-H had to slow down the creative process. Maybe it was like learning to finger the keys before you compose the concerto.

This document had some unfortunate choices of layout, the worst being the use of yellow type to indicate the HP 3000 responses to commands. Like everybody in 1979, HP was learning how to teach its customers about the use of this new tool. Interactive computing was the reason that the HP 3000 took off in an era dominated by IBM mainframes, and HP probably wanted to show how lively the interactive experience could appear. Later on, you could actually see yellow letters in HP responses, on certain types of terminals.

Each year from 1979 to the present, HP has worked to ensure the largest number of HP 3000s could run the programs crafted with the help of this manual and successors. That makes the 3000-using universe unparalleled among any computer launched in the 1970s. Rogers said that the software written in the 1980s ran during this century.

When I started here 1985 we had a Series III and a Series 44. We then upgraded the 44 to a 48 and changed the III to a Series 70. The next step was changing both to a 960. The last one we got was the 969KS/200.

We still had software we wrote on the Series III that was still running on the 969.  Amazing.

When it adopted a go-go grow business mantra in the 21st Century, HP couldn't find the motivation to keep up anymore with its 3000 legend. Perhaps a dedicated base of users, full of expertise and experience, can carry on into a fresh decade of the 21st, starting in 2010. Whether it's a manual for an HP storage device, or a programming aid written before Ronald Reagan took office, nothing seems to expire altogether in your community. How many others can claim that kind of history?

Measuring the 3000's Posix empire

These past few days have seen me working on a project to process files in the HP 3000's HFS Posix directory. My Series 917LX is running under 6.5 PowerPatch 3. What is the maximum size a 3000's Posix structure can be?

HP's community liaison to the 3000 world Craig Fairchild replies:

There are no system enforced limits on directory depth or absolute system path name. Most interfaces won’t gracefully handle pathnames longer than PATH_MAX which is 1024 characters. So pick a number (say 1024) and return an error if you ever see a path longer than that.

As an addendum, Fairchild revealed more about how HP set up Posix the interface most like Unix, on the 3000.

As an aside, even though the CI’s command buffer size is much smaller than PATH_MAX, you can still use relative naming to perform any operation on any file or directory, no matter how long its absolute path name. There is a limit on the maximum length of any one directory component (255 characters in an HFS directory, 16 characters in the root, account and group directories) which is well within the CI command buffer length.

I find that the ability to have names that are longer than eight characters, and a hierarchical directory structure that allows for a more logical structure, as some of the compelling reasons not to be limited to the 8.8.8 MPE namespace.

Storage on the biggest 3000s

Somewhere in Minnesota, a farm of more than 30 HP 3000s tracks airline ticket transactions. The farm, one of the biggest collections of 3000 horsepower, uses plenty of storage. How much is hard to say, since the operation is under wraps and not open to press coverage.

But N-Class HP e3000s need storage that outstrips the requirements of most other HP 3000 models. A customer recently asked about options for data storage on these big systems.

We are looking at using a N-Class system and I’m trying to find out what storage options are available. What kind of disc options are being used out there? I understand that HP does not support EMC storage on the N-Class, but is anyone using it?

Craig Lalley, whose EchoTech consulting firm specializes in putting newer and better storage into 3000 shops, was at the ready with answers.

Lalley, who consults along with Jeff Kubler from their US Northwest HQ, said

The N-Class has native fibre, and while the EMC storage devices are not supported, I guarantee they work just fine.  (I do think the performance of HP's XP1024 arrays are much better).

The N-Class can connect to a VA7410. This storage device supports HAFO (High Availability Fail Over) and is rated at 35,000 I/Os per second.  It has 2GB of cache.  It is a Virtual Array, and is about the size of a MOD20, about 10U. (That's a guess).

The other options are

• The XP512, which does about 300,000 I/Os per second; it is much older and about the size of one large refrigerator.
• The XP1024 or XP128 are rated at 450,000 I/Os per second and are a great option today. 
• The XP128 has less expansion capabilities that the XP1024, but the same core, and can go to 64GB of cache.

As for pricing, it depends on how much space you need.

The payoff between a VA7410 or an XP128 is about three VA7410s  — which means it is less expensive to purchase one XP128 than three VA7410’s.

MPE shell expands, with explanation

Way back in the middle 1990s HP added the Posix shell to the HP 3000, so customers who had Unix and MPE running in the same shop could train operators and managers with a single set of commands. Posix was a plus, making the 3000 appear more Unix-like (which seemed important at the time).

Over the years, however, Posix has been a feature to be discovered for most 3000 managers and operators. (By the way, the computer's operating system was renamed from MPE/XL to MPE/iX just for this added Posix feature).

But enough history; Posix is still on the 3000 and remains a powerful interface tool, an alternative to the CI interface that HP created for the system. You can even call Posix commands from the CI, a nifty piece of engineering when it can be done. That's not always possible, though. A customer wanted to know how to "expand wildcard shells" using Posix. He tried from the CI and had this story to relate.

:LL /BACKUPS/HARTLYNE/S*
ls: File or directory “/BACKUPS/HARTLYNE/S*” is not found

So how do I do this? I need to be able to tell tar to archive all of the reels of a STD STORE set via a regexp.  It does not work in tar, and it apparently does not in ls, so I speculate that there is something special about the innovation of Posix utilities from the CI that I am not aware of. What is it?

Jeff Vance, the 3000 CI guru at while at HP who now develops at K-12 app vendor QSS, had this reply:

   Wildcards on most (all) Unix systems, including Posix implementations, are done by the shell, not the individual programs or in-lined shell commands, like ls in your example. A solution is to run the shell and execute ll from within.

Greg Stigers then supplied the magic Posix shell command to do the expansion:

SH.HPBIN.SYS '-c "/bin/ls -l /BACKUPS/HARTLYNE/S*"'

In a note of thanks, the customer said that getting the answer by working with the HP 3000 community's newsgroup "is like having an entire IT department right outside my door."

An interesting footnote if you've read this far: The Posix shell for the 3000 is one part of the operating system not built by HP. The shell was licensed by HP from MKS, and Hewlett-Packard pays royalties to MKS so Posix can work inside of MPE/iX. That's an issue that OpenMPE and HP will have to clear up before any emulation or source-license arrangements can be worked out.

For now, enjoy using Posix as a way to get familiar with the commands in Unix systems. In the great majority of instances, these commands are the same.

Going cold turkey, coming on board

Some stay, and some just have arrived. After Gary Sielaff told the community he was about to unplug his HP 3000 — and was preparing to hear screams from users about the replacement Windows system's Blue Screen of Death — a new user asked about basic database backup.

I just started learning an HP 3000 — I guess we are going in different directions. I have an HP 3000 connected to an Win32 IIS box. How would I go about backing up the database before I go running DOALL commands?

Matt Perdue replied:

Start out by doing a test:

:FILE STORDISC;DEV=DISC :STORE @.PUB;*STORDISC;SHOW.

This should backup all the files in the PUB group of the account you're logged in currently and if it works will prove your version of STORE can do a store to disc backup. If it doesn't work do: :FILE STORFILE;DEV=TAPE :STORE @.PUB;*STORFILE;SHOW and the operator (perhaps you) will have to put a tape in and respond to the tape request on the console.

Substitute the "@.PUB" for the database name (e.g. ARDB@.DATA if your database is named ARDB and it's in the DATA group) and store your database to either disc or tape.

You can read the full thread of comments, including Gary's, at the 3000 newsgroup archives.

Perdue, a member of the OpenMPE board, added some general advice for the new HP 3000 user:

See docs.hp.com for the MPE/iX documentation or use the link off HP's Jazz Web site at http://jazz.external.hp.com or just directly to

http://www.docs.hp.com/en/mpeixall.html.

Look for "MPE/iX System Utilities Reference Manual" for the version of the operating system on your machine. To tell that, do a SHOWME and the version will be shown on a line such as:

RELEASE: C.75.00 MPE/iX HP31900 C.45.05 USER VERSION: C.75.00

or

RELEASE: C.70.01 MPE/iX HP31900 C.39.06 USER VERSION: C.70.01

The first example is 7.5 and the second is 7.0

Create PDF files from 3000 output

Last year we updated the abilities of the Sanface Software solution to create PDF files out of HP 3000 output. But there are other ways and tools to do this, a task that's become essential to sharing data reports between HP 3000s and the rest of the world's computers.

On the HP 3000 newsgroup, a veteran 3000 developer asked,

Has anyone any experience involving taking a file in an output queue and creating a PDF version of it?

"We use text2pdf v1.1 and have not had any problems since we installed it in October 2001," says Robert Mills of Pinnacle Entertainment. "I have e-mailed a copy of this utility and our command file to 27 people. Never knew that so many sites wanted to generate PDFs from their 3000s."

The program is a good example of 3000 source code solutions. This one was created as far back as the days of MPE/iX 6.0, a system release which HP has not supported since 2005.

Lars Appel, the former HP support engineer who built such things on his own time while at HP support in German, has source code and a compiled copy of the utility.

Such solutions, and many more, are hosted on the Web server at 3K Associates, www.3k.com. Check the Applications Ported to MPE/iX section of the Public Domain Software area at 3K's Web site.

You'll also find a link to GhostPCL up at the site, another Appel creation, one which he describes as

A program that reads PCL input files and converts them to a variety of output formats, including PDF or JPEG, for example. Combined with my little FakeLP Java program, you might even use it to capture MPE/iX network spooler output and generate PDF or JPEG from an MPE/iX spoolfile.

Open source solutions like these have been an HP 3000 community tradition. Way back in 2000, we reported in the NewsWire about that FakeLP Java program, helpful in getting text2pdf to do its PDF magic. (For the record, both Mark Bixby and Appel are still working in the 3000 community, but not for HP; Bixby joins QSS this month, and Appel has been helping in support for the producer of the database Eloquence, Marxmeier Software)

A roadblock to using the text2pdf program: the spoolfiles had to be in text file format to work with it. But Lars Appel offered a free solution to make 3000 spoolfiles that don't rely on CCTLs ready for their PDF closeups:

"I have a small Java program that listens to a given port, for example 9100, and 'pretends to be a network printer' i.e. gets all the data sent and writes it to a flat file. This might be a start, as OUTSPTJ.PUB.SYS should have converted CCTL to plain PCL when sending to a JetDirect printer. However, this little Java program is just a quick and dirty experiment. Use at your own risk; it worked on my 3000, but your mileage may vary."

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cut here _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

// FakeLP pretends network printer to capture spooler PCL output

import java.net.*;
import java.io.*;

class FakeLP {

public static void main( String args[] ) throws Exception {

int port = 9100;
int next = 1;

if (args.length > 0) port = Integer.parseInt(args[0]);
if (args.length > 1) next = Integer.parseInt(args[1]);

ServerSocket serv = new ServerSocket( port );

while (true) {

System.out.println("FakeLP listener ready");

Socket sock = serv.accept();
byte[] buf = new byte[4096];
String name = "F" + (next++);

System.out.println("Capturing spoolfile to " + name);

InputStream si = sock.getInputStream();
OutputStream fo = new FileOutputStream(name);

for (;;)
{
int got = si.read(buf);

if (got != -1)
fo.write(buf, 0, got);
else
break;
}

fo.close();
si.close();
}
}
}