May 08, 2015
Wiping An MPE Past Clean: Tools and Tips
The 3000 newsgroup readers got a query this week that's fit for our migrating epoch. "It's the end of an era, and we're going to dispose of the HP 3000," said Krikok Gullekian. "After deleting all of the file, is there a way to wipe out the operating system?"
Such wipe-outs are the closing notes of the migration's siren song. Nobody should leave evidence behind of business data, even if that 3000 is going out to a tech recycle house. A piece of software, a classic part of hardware, and even wry humor have been offered to meet the wipe-out request.
Donna Hofmeister of Allegro Consultants pointed to WipeDisk, a program that's hosted on the computer that will no longer know its own HPCPUNAME once the software finishes its job. It will sanitize an MPE/iX disk drive. (Versions for MPE/V, HP-UX, Mac OS X and Linux are also available.)
"You install WipeDisk on your target system and run it when you're really, really really sure you're ready to say good-bye to your old friend," she said.
It's not complete enough just to run MPE's VOLUTIL>FORMATVOL command, Allegro notes on the product's webpage. "You cannot count on VOLUTIL>FORMATVOL to ‘erase’ a disk. It might, or might not, depending upon the disk vendor’s implementation of the device firmware."
Hardware to fully erase the disks magnetically was also offered as a solution. Then there was the reference to the Hewlett-Packard of the era of this month's new Presidential candidate, Carly Fiorina.After a few suggestions to take a hammer, chain saw, or wood chipper to the drives, Denys Beauchemin pointed managers at the everlasting legacy of magnetic degaussing. "I would think that degaussing the disk drives, or simply taking the disk drives out and destroying them separately, would be the most secure method, if you have any concerns with anyone ever being able to read from these disks. Seems a shame to destroy good hardware, though."
Alan Yeo of ScreenJet got the sharpest word in, during a week when 3000 users began commenting on their ardor for Fiorina's candidacy. To wipe out MPE, Yeo said, "just leave it in the hands of HP. They started a fairly good job in November of 2001."
Fiorina probably never knew of the server, but it was on her watch HP pulled its plug. The vendor failed to wipe out MPE altogether, though. Only 2028 will do that, and even that date might not be able to complete the wipeout.
"You're asking advice on committing a sin," added Michel Adam, Systems Analyst for Canada's Government of the Northwest Territories.
Get e-mail notice when the NewsWire blog gets a new entry. Just say "Blog Me" in a message to email@example.com.
March 26, 2015
Checkup Tips to Diagnose Creeping Crud
When an HP 3000 of the ultimate generation developed trouble for Tom Hula, he turned to the 3000 newsgroup for advice. He'd gotten his system back up and serving its still-crucial application to users. But even after a restart, with the server looking better, things just didn't seem right to him.
I am concerned, since I don't know what the problem was. It almost reminded me of something I used to call the Creeping Crud, where people started freezing up all over the place, while some people were still able to work. The only thing was a reboot. But in this case, it seemed worse. Only a few people on our 3000 now, but we still depend on it for a high-profile application. What should I check?
The most revealing advice came from Craig Lalley, who told Hula he'd try a Control-B into the 3000's system log. The steps after the Control-B command are SL (for System Log) and E (for Errors only.) Typing CO puts the 3000 back in console mode. Hula's system had lost its date and time on one error, and the Alert Levels showed a software failure along with lost boot functionality.
But amid the specifics of eliminating the Creeping Crud (it may have been a dead battery) came sound advice on how to prepare for a total failure and where to look for answers to 3000 hardware problems. The good news on the battery is that it's not in a Series 9x7. Advice from five years ago on battery replacement pointed to a hobbyist-grade workbench repair. More modern systems like Hula's A400 at least have newer batteries.Using a DSTAT ALL was suggested, as well as checking the status of available storage with DISCFREE. Mark Ranft said "I would make sure there's a good full backup. (Just in case you need it for recovery.) If you don't have one, doing one may help identify a disk issue. I would check the system logs especially for disk errors. I would check for network errors, using linkcontrol @,all." He shared his own recovery experience.
I had a system acting strangely this past weekend. It was basically hung but allowed new logons. I could not abort anyone. When I got to the point where I tried to stop the network, I got a system abort 1458 from Subsystem 102. I didn't bother to take a dump. I completed the boot and everything was better.
Chuck Trites reminded Hula to create a current CSLT tape, and "run BULDJOB to create the BULDJOB1 and 2 files — in case you need to recreate the accounting structure and UDCs — and store them to tape."
Hula's own check list included the following:
During the reset, the 3000 got up to the date and a little past and seemed frozen. I pulled the plug and restarted again. It took 2-3 times as long as normal and at first, the red fault light was on (I never saw that on before). After it got a bit into the restart, the fault light turned off by itself. The only attention message I got about the whole thing was a message with everything unknown on the 3000.
When the computer came all the way up, it still seemed very sluggish. I scheduled the nightly update and backup and went home to look at it more in the morning. I logged on from home and the backup seemed to be running okay.
This morning I tried resetting the GSP and checked the connections to the console terminal. I also found out that someone else had a hard time getting on the 3000 towards the end of the day. Very sluggish. But this morning, everything seems back to normal.
Hewlett-Packard's hardware builds have been extraordinary, but a server that's been churning out critical data for more than 12 years (A-Class boxes production stopped in 2003) can develop crud. Something as simple as replacing a dead battery might be the answer to the woes. Advice for the crud also came from Gilles Schipper, Jack Connor and the others mentioned. What they've got in common is working in a support practice, or at least a consulting business that includes 3000 sites.
Self-maintenance is common in a community like the 3000's. It's also a good practice to have a support vendor, one who knows the system as well as the volunteers posting to the newsgroup.
March 20, 2015
3000s still worthy of work to secure them
While an HP 3000 might be an overlooked resource at some companies, it's still mission-critical. Any server with 40 years of history can be considered essential if it's still part of a workflow this year. Managers of 3000s don't automatically think of protecting their essential resource from the malware and hackers of 2015, though.
That was illustrated in a recent thread on the 3000 newsgroup traffic. A 3000 manager serving the Evangelical Covenant Church needed help restarting an old Series 9x7. (By definition, any Series 9x7 is old. HP stopped building this first generation of entry-level 3000s more than 20 years ago.) The manager said the 9x7 had been "in mothballs," and he wanted to run an old in-house app.
I was able to boot up and login as OPERATOR.SYS but cannot remember/find the password for MANAGER.SYS. Is there anyway to reset, clear, or overwrite the password file? I know the old machine is a very secure one, but now I am hoping there is a way around it.
And then on the newsgroup, advice on how to bypass 3000 security began to emerge. It surprised one consultant who's recently closed down a big 3000 installation full of N-Class servers. Should the community be talking about how to hack a 3000, he wondered? The conversation really ought to be about how to ensure their security, practices we chronicled a few years ago.It's not like the bypassing information shared was certain to sidestep MPE's security. But Mark Ranft of Pro3K thought these answers should be taken offline.
Please remember that even though these are legacy systems, providing expert level security tricks and secrets to help people break into systems is still probably not a great idea on an open forum. I suggest you reply with your hacking suggestions in private email messages.
Not many 3000s sit on open, public networks. But the servers which they communicate with are often on accessible networks. Who's to say what's even accessible these days? Unisys, which is a long way from relevant in the enterprise computing field by now, is selling its newest products as Stealth Computing. "You can't hack what you can't find," they say in their ads on NPR.
Security is never so simple as that. But hackers navigate complex protection all the time. HP sold a security software product that one support expert said "implements directory encryption." The kingpin of 3000 security is of course Security/3000 from Vesoft. (Founder Vladimir Volokh called today to report things are looking up in his company, so to speak. Q1 of 2015 is more robust than Q1 of 2014.)
Controlling who can login as an operator is a great way to enable tighter security for a 3000. Passwords for OPERATOR.SYS are an excellent practice. If a 3000 is in mothballs with no sensitive data on it, these kinds of habits aren't essential. But how can you be sure no data is essential, of no use to hackers or competitors? Better secure than sorry.
March 04, 2015
Tablet opens new access window on 3000
HP 3000s have the ability to communicate with iPads, although the inverse is even more true. The software that makes this possible is in regular use at an ecommerce company in the US. A seasoned manager at the company checked in with us, on her way to setting up a link between an Ecometry box and Apple's tablet.
Chris McCartney of Musical Fulfillment reached out for assistance with configuring her 3000 and the TTerm Pro app from Turbosoft. Musical Fulfillment is the parent company to American Musical Supply, zZounds.com, ElectricGuitar.com, and SameDayMusic.com
Once McCartney located a back copy of the Newswire, she says, she found Jon Diercks article about the app when the software was first released in 2013. "We've been using Red Prairie Direct Commerce (aka Ecometry, Escalate, MACS) for more than 10 years and we moved to the [N Class] several years ago. We were hoping to get a few more years out of it before we had to make a decision to upgrade or move to a different ERP system."
By deploying TTerm Pro, McCartney now has a mobile way to check on the status of that N-Class server.
I am up and running on my iPad for those ‘just in case’ times when I am away from my office or laptop and I need to log in to check something on the 3000 or in Ecometry/JDA Direct Commerce. I am going into work over the VPN and using TTerm Pro to connect to our HP. I use the on-screen keyboard, but might switch to a wireless keyboard, so I have a little more screen and the comfort of a physical keyboard.
The 3000 at the company is established as a sensible solution. Up to now, there's been no compelling return on the investment to move to Ecometry hosted on Windows systems.Making these kinds of decisions, year by year, about migration's rewards can be a hard place from which to do ecommerce. There's some debate over whether there's a sensible package to replace Ecometry on the 3000, as the server continues to perform its stable, steady mission.
We've heard of some 3000 sites where they're dealing with some upper-level mission to leave the 3000. Reasons for departing the 3000 vary, but they often revolve around withdrawn support from the system's vendor. Diligent managers like McCartney arrange for independent support. They also have the advantage of ground-breaking interfaces like tablets to monitor their 3000s.
Timetables and budgets for migrations vary. It's often in the best interests of a company to get the maximum use, within sensible and safe limits, from existing applications. A product that makes the 3000 easier to manage, created within the last few years, is something of a high note here in the fifth decade of MPE service.
February 04, 2015
Checks on MPE's subsystems don't happen
Once we broach a topic here on your digital newsstand, even more information surfaces. Yesterday we reported on the state of HPSUSAN number-checking on 3000 hardware. We figured nobody had ever seen HPSUSAN checks block a startup of MPE itself, so long as the HPCPUNAME information was correct. The HP subsystems, though, those surely got an HPSUSAN check before booting, right?
Not based on what we're hearing since our report. Brian Edminster of Applied Technologies related his experience with HP's policing of things like COBOL II or TurboStore.
Like several of our other sources, Edminster knows that the third-party providers, especially the big-name players, use HPSUSAN to make sure that vendor knows where its software is booting up. Because of those exacting checks, "You've got to have some sort of plan in place to cover having to use any alternate hardware for disaster recovery," he reports, "and still expect to have your third party tools work beyond a limited time-frame."
I can't claim to be an expert in all things regarding to software licensing methods. But I can tell you from personal experience that none of HP's MPE/iX software subsystems that I've ever administered or used had any sort of HPSUSAN checks built into them. That would include the compilers (such as the BASIC/3000 interpreter and compiler), any of the various levels of the HP STORE software versions, Mirror/iX, Dictionary/3000, BRW, or any of the networking software. (I'll note that the networking software components were quite picky in making sure that compatible versions of the various components were used together, in order for everything to work properly.)
The only time I saw HP-provided software examined using the HPSUSAN was when server hardware was upgraded. It checked the CPU upgrades, or number of CPUs in a chassis.
But there's no dissenting story out there regarding what's ethical to do with intent about respecting software checks and licensing. There are always such possibilities for managers who live outside the lines. And sometimes it might be an oversight. As an example, O'Pin Systems -- a first-issue advertiser in the 3000 Newswire more than 19 years ago -- still has Reveal customers out in the MPE world relying on that reporting tool.
One such site was having a hard time with a boot-up on a different MPE/iX server. A START command to Reveal's RSPCNTL will stream a job, but RSPCNTL would terminate before a prompt was given. "I think this may indicate that the product is not validated on the new machine -- which would require re-validation," a former developer for O'Pin told us. "I don't recall exactly what's checked, but the variables HPSUSAN and HPCPUNAME are almost certainly checked for a match. VAR OPINSERIAL will appear to be set to 0, if RSPCONTROL determines that there is a validation fault."
Re-validation can be a matter of placing a call to the vendor's support line -- if there's anyone left on the vendor's staff who understands that the company still has an MPE/iX product in the field. O'Pin has such a support staffer.
Edminster had a cogent comment about this need for this validation during an era when 3000 outposts are shrinking.
I'm don't propose that software purchased for one system be moved to another, unless that's within the bounds of the original software agreements. Just because a vendor has stopped selling a product, or stops pursuing license violations of that product, doesn't make the product freeware.
It also does not make that product yours to use as though you owned it.
Most software was 'sold' with a 'right to use' license. That doesn't mean you own it, now or ever. It means that you are licensed to use it under the terms in the original contract, or as amended since.
That may sound like splitting hairs. But as intellectual property goes, it can make the difference between being able to make a living on the fruits of your work, or not.
December 11, 2014
Big, unreported computing in MPE's realm
When members gather from the 3000 community, they don't often surprise each other these days with news. The charm and challenge of the computer's status is its steady, static nature. We've written before about how no news is the usual news for a 40-year-old system.
But at a recent outing with 3000 friends I heard two pieces of information that qualify as news. The source of this story would rather not have his name used, but he told me, "This year we actually sold new software to 3000 sites." Any sort of sale would be notable. This one was in excess of $10,000. "They just told us they needed it," my source reported, "and we didn't need to know anything else." A support contract came along with the sale, of course.
The other news item seemed to prove we don't know everything about the potential of MPE and the attraction of the 3000 system. A company was reaching out for an estimate on making a transition to the Charon emulator. They decided not to go forward when they figured it would require $1 million in Intel-based hardware to match the performance of their HP 3000.
"How's that even possible?" I asked. This is Intel-caliber gear being speficied, and even a pricey 3000 configuration shouldn't cost more than a quarter-million dollars to replace. It didn't add up.
"Well, you know they need multiple cores to replace a 3000 CPU," my source explained. Sure, we know that. "And they had a 16-way HP 3000 they were trying to move out."
Somewhere out there in the world there's an HP 3000, installed by Hewlett-Packard, that supports 16 CPUs. Still running an application suite. The value is attractive enough that it's performing at a level twice as powerful as anything HP would admit to, even privately.
A 4-way N-Class was as big as HP would ever quote. Four 500-MHz or 750-MHz PA-8700 CPUs, with 2.25 MB on-chip cache per CPU, topped the official lineup.
Unix got higher horsepower out of the same HP servers. An 8-way version of the same N-Class box was supported on HP-UX; HP would admit such a thing was possible in the labs, and not supported in the field. But a 16-way? HP won't admit it exists today, and the customer wouldn't want to talk about it either. Sometimes things go unreported because they're too big to admit. It made me wonder how much business HP might've sustained if they'd allowed MPE to run as fast and as far as HP-UX ran, when both of those environments were hosted on the same iron.
December 10, 2014
Getting Macro Help With COBOL II
An experienced 3000 developer and manager asked his cohorts about the COBOL II macro preprocessor. There's an alternative to this very-MPE feature: "COPY...REPLACING and REPLACE statements. Which would you choose and why?"
Scott Gates: COPY...REPLACING because I understand it better. But the Macro preprocessor has its supporters. Personally, I prefer the older "cut and paste" method using a decent programmer's editor to replace the text I need. Makes things more readable.
Donna Hofmeister: I'm not sure I'm qualified to comment on this any longer, but it seems to me that macros were very efficient (and as I recall) very flexible (depending on how they were written, of course). It also seems to me that the "power of macros" made porting challenging. So if your hidden agenda involves porting, then I think you'd want to do the copy thing.
There was even porting advice from a developer who no longer works with a 3000, post-migration.Tony Summers: When we migrated in 2008 we chose Acucobol partly because of its HP 3000 compatibility, including macro support. However had we gone down a different route, I had already proved that I could pre-process the raw code myself and expand the macros before calling the compiler.
Robert Mills, who started the discussion on the 3000-L, said in reply to Donna, “I admit that I do have a hidden agenda, but the main reason does not involve porting.”
For many years I have used macros to make my life easier. When I left the e3000, back in 2008, and did some work on other platforms I found I missed them. I'm now in semi-retirement and have been using the free version of Micro Focus COBOL (a couple of years) and GnuCOBOL (this last year) to write software for friends, family and my own use.
A couple of times since 2008 I had thought of writing by own macro preprocessor to emulate the one on the e3000. A few months ago I decided to do it and release it as open source under the GNU GPL. The development of preprocessor, using GnuCOBOL, is now completed and in final Beta Testing and I'm writing the manual. Was hoping that I could some additional reasons, from others, as to why you would use macros instead of the copy...replacing and replace statements.
Because a port of GnuCOBOL is a available on several platforms, and my preprocessor is written in GnuCOBOL, I see no problem in taking my macros with me nearly every wherever I go. If I end up doing work on a platform that does not support a feature that it is using it shouldn't be to difficult to develop a workaround.
As it turns out, GnuCOBOL is a newer version of OpenCOBOL — a compiler that Donna says bears a close resemblence to COBOL II. (OpenCOBOL has been ported into a commercial product, too, called IT-COBOL.) Adding that she obviously thinks macros are cool, she explained.
Do my mis-firing neurons recall that GnuCOBOL was formerly OpenCobol... which was actually very close to MPE’s COBOL? (or something like that?)
I inherited a outstanding collection of macros at one job. Many of them were 'toolbox' functions. Want to center a string and the overall length of the string doesn't matter? Got a macro for that. Want to use a 'db' call? Got a macro for that. These went far beyond modifying code at compile time -- and that's what made them so valuable (at least to me).
November 18, 2014
Replacing rises as migrator's primary choice
It's the end of 2014, just about. Plenty of IT shops have closed down changes for the calendar year. Many 2015 development budgets have been wrapped up, too. Among those HP 3000 operations which are still considering a strategy for transition, there's only one assured choice for most of who's left. They'll need to replace their application. Not many can rehost it.
We've heard this advice from both migration services partners as well as the providers of tools for making a migration. An HP 3000 is pretty likely to be running an application with extensive customization by this year. We've just now edged into the 14th year since HP announced a wrap-up of its interest in all things MPE/iX. Year One began in mid-November of 2011. After completing 13 years on watch during the Transition Era, there's a lot of migration best practices to report. More success has been posted, at a better price and on schedule, when a replacement app can be integrated along with a new server and computing environment.
Of course, massive applications have been moved. One of the largest was in the IT operations of the State of Washington Community College Computing Consortium. It was a project so large it was begun twice, over enough elapsed time that the organization changed its name. The second attempt better understood the nuances of VPlus user interface behaviors. There were 40 staffers and at least four vendor services groups working on the task.
One of the issues that's emerged for rehosting organizations is a reduction in MPE expertise. Companies can still engage some of the world's best developers, project managers, and rewriting wizards for MPE/iX. It's harder to assign enough expert human resources who know your company's business processes. That's why a top-down study of what your apps are doing is the sort of job that's been going out-of-house. By this year, it would be better to engage an outside company to replace what's been reliable. This hired expertise ensures a company doesn't lose any computing capability while it makes a transition.
You'll need the use of tools to manage data in a replacement, though. Everything else is likely to change, even in a replacement, except for the data. "Replacement requires reorganizing data," Birket Foster of MB Foster told us this summer. "You could start cleaning your data now." Foster is presenting a Webinar on the subject of the Three Rs -- Rehosting, Replacing, or Retiring -- tomorrow (Wednesday) at 2PM Eastern Time.A company making a transition to a replacement app needs to understand what data will be needed, at what detail level, and in what timeframe. The best answers to those questions might come from outside of the IT group. In face, Foster says they often do. A solid team of transition stakeholders always includes an important seat for a member from the business group.
Replacement of a 15- or 20-year MPE/iX app suite also might not be a favored choice in the IT group. That group includes the experts who know the programs best. Nothing seems like it will be a clean, quick fit for what's been running the company -- not at first. Replacing with a non-MPE version of the app sometimes leaves key integrated surround code at the curb, too. Replacing surround code is a good project for outside expertise. Companies which consult on that task have field experience on success to share.
The good news: replacing a business suite is not as dangerous as replacing a joint. You get to shop and specify and test for replacement software, even while the worn-down hip of the business suite continues to bear the weight of the company's enterprise. Backing out of a replacement -- replacing the replacement -- is just as extensive in software as it is in medicine. It's like doing it all over again. But replacing after an attempt at rehosting? That's the least effective strategy of all.
November 17, 2014
HP's 3000 power supply persists in failure
Amid a migration project, Michael Anderson was facing a failure. Not of his project, but a failure of his HP 3000 to start up on a bad morning. HP's original hardware is in line for replacement at customers using the 3000 for a server. Some of these computers are more than 15 years old. But the HP grade of components and engineering is still exemplary.
"I was working with a HP 3000 Series 969, and one morning it was down," he reported. "All power was on, but the system was not running; I got no response from the console. So I power-cycled it, and the display panel (above the key switch) reported the following."
Proceeding to turn DC on
Anderson explained that Charles Johnson of Surety Systems replaced the power supply for the system.
On the console it displayed garbage when the power was turned on, but the message on the display remained. I wasn’t sure what to replace. I was thinking the power supply — but all of the power was on. As it turned out, even in the middle of a power supply failure the 3000 was working to get out a message. The back side, the core I/O, FW SCSI, and so on, all appeared to have power. That is why I found it hard to believe that the power supply was the problem.
He explained that (back in the day) HP engineered some of the best power supplies in world, lots of checks and verifications. Even though the power supply had actually supplied DC power to the various components, it was not able to verify it.
So the message "Proceeding to turn on DC power" remained on the front panel display, meanwhile the boot process on the console would hang, and if you do a <cntrl-B>, RS it would time-out with a msg:
"FATAL ERROR: System held in reset. POW_ON never came back (APERR 21)"
"Waiting until it's reasserted......"
Bill & Dave's Excellent Machine — even with a power supply failure, it still manages to get a message out (in plain English) attempting to explain the failure.
November 11, 2014
Veterans get volunteered for transition's day
Here on Veteran's Day — I'm a vet of the '70's-era military — I'm remembering there are IT pros with another kind of veteran status. They are people who count more than a couple of decades of experience with the HP 3000, managing their servers since before the time that Windows was the default computing strategy. They've been through a different kind of conflict.
I've learned that the most embattled managers employ a surprising tool. It's a sense of humor, reflected in the tone of their descriptions of mothballing the likes of 25-year-old independent apps during migrations. They have to laugh and get to do so, because their attempts to advance their positions might seem like folly at first look, or even in a second attempt.
Really, an assignment like putting Transact code into an HP-UX environment? Or take the case of working around a financial app software from Bi-Tech -- an indie vendor that "really stopped developing it for the 3000 years ago," according to City of Sparks, Nevada Operations & Systems Administrator Steve Davidek. There's been some really old stuff doing everyday duty in HP 3000 shops. The age of the applications was often in line with the tenure of the project's management.
These pros typify the definition of veterans, a term we'll use liberally in the US today to celebrate their sacrifices and courage. Facing battle and bullets is not on par with understanding aged code and logic. But two groups of people do have something similar at heart. Both kinds of veterans have been tested and know how to improve the odds of success in a conflict. Youthful passion is important to bring fresh energy to any engagement, military or technological. What earns the peace is experience, however grey-haired it looks next to Windows warriors.
With each mission accomplished -- from what looks like the Y2K effort of 14 years ago to embracing a roll-your-own Unix that replaced MPE's integrated toolset -- these veterans moved forward in their careers. "Our knowledge base is renewed with this work," one said after migrating apps that served 34 Washington state colleges. "We're on the latest products."
Recruiting IT talent into small towns — and the 3000 runs in many small cities where manufacturing labor is less costly — meant hiring for Windows experience. Adopting Windows into an organization means leaving proprietary environments even more popular than MPE/iX. Like HP-UX.
Leaving a familiar environment means enduring risks. But a tone of "yeah, that'll happen, but we'll manage through it" is what I hear from the 3000 pros marching into the dark of 2015 and beyond. And if a migration is happening next year or the years beyond, you may want to thank a colleague -- anyone whose IT battles have promoted the knowledge that creates veterans, marching in the ranks of both managers and vendors alike.
Les Vejada worked with HP 3000s for more than 20 years at HP, then moved on to HP's other enterprise platforms until the vendor cut his job. It was as if he'd mustered out of a unit. "I don't work with the 3000 anymore," he told us when he joined the Linked In HP 3000 Community. "I know the 3000 is dying, but it brings back a lot of great memories. I probably would still take something on a MPE machine if offered." Whether it's in-house, or in-community, one motto remains as valid today as it did after any conflict: Hire a Vet.
October 20, 2014
3000's class time extended for schools
The San Bernadino County school district in California has been working on moving its HP 3000s to deep archival mode, but the computers still have years of production work ahead. COBOL and its business prowess is proving more complicated to move to Windows than expected. Dave Evans, Systems Security and Research officer, checked in from the IT department at the district.
We are still running two HP 3000s for our Financial and Payroll services. The latest deadline was to have all the COBOL HP 3000 applications rewritten by December 2015, and then I would shut the HP 3000s down as I walked out the door for the last time. That has now been extended to 2017, and I will be gone before then.
We are rewriting the COBOL HP 3000 apps into .NET and Windows Presentation Foundation (WPF) technologies. Ideal says they can support our HP 3000s until 2017.
And with the departure date of those two HP 3000s now more than two years away, the school district steps into another decade beyond HP's original plans for the server line. It is the second decade of beyond-end-of-life service for their 3000.Evans was checking up on the timeline.
In the original timeline HP published, did HP announce in November 2002 that the HP 3000 was at end of life? That HP 3000 production lines would shut down in 2004, and all HP 3000 support would end 2007?
Very close, but not quite accurate. The 3000's future got its exit notice from Hewlett-Packard in 2001 (almost 13 years ago), and system manufacturing ended in 2003. The first of HP's end of life deadlines was December 2006. Virtually nobody would have figured in 2001 they'd have MPE applications still in service more than a decade after 2006.
But San Bernadino County is giving lessons on how to extend an investment, even while it finishes a migration. By the time those school district servers go offline -- and they won't be the last in the world by any means -- the 3000 product platform will have been in continuous production service somewhere in the world for 43 years.
October 14, 2014
Making a Migration Down the Mountain View
After an exit off the HP 3000, the City of Mountain View is now also saying goodbye to one of its longest-tenured IT pros. Even beyond the migration away from the municipality's Series 957, Linda Figueroa wanted to keep in touch with the HP 3000 community, she reported in a note. "I started working on a Series III back in the 1980s," she said.
But after 38 years with the City, and turning 55, it's time to retire. At a certain time, city employees with as many years as I have get the "when are you retiring?" look. We had 3000s running at the City of Mountain View from 1979 until 2012.
Our first HP 3000 in 1979 was a Series III system (which I just loved; always felt so important pressing those buttons). It had a 7970E tape drive, four 7920 disc drives and a printer. Then we moved to the monster Series 68, and ended up with the Series 957 with DLT tapes — no more switching reel-to-reels! I still have my MPE:IV software pocket guide from January 1981. (I couldn't get rid of it — coffee stains and all.)
When Mountain View took down its HP 3000, a couple of years after the switchover, the City turned off all of its other Hewlett-Packard servers, too. Only its software suppliers have made the transition, proving the wisdom that customers are closest to their applications — and leave the platforms behind. But MPE — from System IV to MPE/iX 6.5 — and the HP 3000 did more than three decades of service at Mountain View.Mountain View first purchased Dell servers to replace the HP 3000 in 2010. "We had Utility Billing and Business License software from Idaho Computer Systems," Figueroa said. "We still run the same software from DataNow (a vendor previously known as Idaho Computer Systems). But we run it now on Dell servers."
"Up until 2012 we had an HP 9000 Unix system running our IFAS accounting software from SunGard Bi-Tech. These computers were also replaced by Dell servers."
The veteran IT manager added that she used Adager, Suprtool, Reflection and several other products on the 3000. "I also used VEsoft's products, Figueroa said. [VEsoft founder] "Vladimir Volokh would make site visits once a year for years, begging me to update our software."
October 08, 2014
Another Kind of Migration
Change is the only constant in life, and it's a regular part of enterprise IT management, too. Another sort of migration takes place in one shop where the 3000 has been retired. Specialized scripts for automation using Reflection are being replaced. Thousands of them.
Micro Focus, which owns Reflection now as well as its own terminal emulator Rumba, is sparking this wholesale turnover of technology. Customers are being sold on the benefits of the Micro Focus product as part of a suite of interlocking technologies. When that strategic decision is taken, as the British like to say (Micro Focus has its HQ in the country) the following scenario plays out.
Glenn Mitchell of BlueCross BlueShield of South Carolina reported his story, after reading our report on Micro Focus acquiring Attachmate.
"As a former WRQ PC2622 user," Mitchell added, "it’s as sad to see my days with Reflection coming to an end as much as my days with MPE ended." As for EHLAPPI, it stands for Enhanced High Level Language Application Program Interface. And with any acronymn that has seven letters, it's a design choice that's got quite a, well, legacy air to it.
I can certainly see many parallels between the latest change at our organization and the migrations many of us undertook from MPE to other platforms.
It has been many years since I was heavily involved with the 3000 and the 3000 community. One of the ties back to those old days has been that we use Reflection 3270 as our mainframe terminal emulator here. I’ve done a number of extensive macros in Reflection VBA to assist our customers and developers, and I understand we have thousands of Reflection VBA and Reflection basic scripts in use throughout the company. (We’re a mainframe-centric organization specializing in high-volume claims processing, including Medicare claims in the US.)
Some months ago, I was told we were dropping Reflection and moving to Rumba by Micro Focus (the old Wall Data product) as a cost-saving measure. As part of that move, all of my macros will need to be converted to use the EHLAPPI interface in Rumba. According to the support staff here, a conversion was going to be required anyway to move to the latest version of Reflection. Well, the support staff has done a good job and many thousands of macros run pretty successfully with some special conversion tools they’ve provided.
Of course, mine don’t, yet.
It's an API that goes back to the early PC days, and allowed a program running on the PC to "scrape" data from a terminal emulator session running on the PC. So it represents a big move backwards in technology from Reflection VBA.
Our guys figured out a way to run our VBA scripts in Excel and trap most of the Reflection API calls (e.g. getdisplaytext) and convert them to equivalent EHLAPPI calls for Rumba. The gotcha is that they've only done the most frequently used API functions, and Rumba doesn't support all of the functions Reflection makes available via API.
Scripting inside of a terminal emulator product represents a deep level of technology. Just the sort of tool a 3000 shop deploys when it can command petabytes of data and tens of thousands of users. When things change with vendor plans, whether it's a system maker or a provider of software, support staff shifts its support to migration tasks.
As an interesting footnote to the changes in the outlook for Reflection -- given that Rumba has been offered as a replacement -- we turn to the a recent comment by Doug Greenup of Minisoft. "Minisoft has NS/VT in its HP terminal emulator," he noted when we described the unique 3000 protocol in some versions of Reflection. "And unlike WRQ, we remain independent. We still have HP 3000 knowledgeable developers and support people." The company's terminal emulator for 3000s, Minisoft Secure 92, has a scripting language called TermTalk.
September 08, 2014
Who else is still out there 3000 computing?
Employing an HP 3000 can seem as lonely as being the Maytag Repairman. He's the iconic advertising character who didn't see many customers because a Maytag washing machine was so reliable. HP 3000s have shown that reliability, and many are now in lock-down mode. Nothing will change on them unless absolutely necessary. There is less reason to reach out now and ask somebody a question.
And over the last month and into this one, there's no user conference to bring people together in person. Augusts and Septembers in the decades past always reminded you about the community and its numbers.
Send me a note if you're using a 3000 and would like the world to know about it. If knowing about it would help to generate some sales, then send it all the sooner.
But still today, there have been some check-ins and hand-raising coming from users out there. A few weeks back, Stan Sieler of Allegro invited the readers of the 3000-L newsgroup to make themselves known if they sell gifts for the upcoming shopping season. "As the holiday shopping season approaches," he said, "it occurred to me that it might be nice to have a list of companies that still use the HP 3000... so we could potentially consider doing business with them."
If September 9 seems too early to consider the December holidays, consider this: Any HP 3000 running a retail application, ecommerce or otherwise, has gone into Retail Lockdown by now. Transitions to other servers will have to wait until January for anybody who's not made the move.
Sieler offered up a few companies which he and his firm know about, where 3000s are still running and selling. See's Candies, Houdini Inc, and Wine Country Gift Baskets are doing commerce with gift consumers. We can add that Thompson Cigar out of Tampa is using HP 3000s, and it's got a smoking-hot gift of humidor packs. (Sorry, couldn't resist.) Then there's American Musical Supply, which last year was looking for a COBOL programmer who has Ecometry/Escalate Retail experience.
Another sales location that could provide gifts for the holiday season is in airports. The duty free shops in some major terminals run applications on MPE systems. HMS Host shops, at least four of them, sell gifts using 3000s. Pretty much anything you'd buy in a duty free shop is a gift, for somebody including yourself.The discussion of who's still using, and feeling a little Maytag solitude, prompted a few other customers to poke up their heads. We heard again from Deane Bell at the University of Washington, where there could be another 10 years of homesteading for the 3000. The first three finished. All in archival mode.
Beechglen furnishes an HP 3000 locally hosted system meeting the following minimum specifications
· Series A500 Server
· 2GB ECC memory
· 365 GB disk space consisting of 73GB operating system and temporary storage for system backups, and 292GB in a software RAID-1 configuration yielding 146GB of usable disk storage
· DDS3 tape drive
· DLT8000 tape drive
There were other check-ins from Cerro Wire, from the California Department of Corrections and Rehabilitation (where one 3000 wag quipped, "the users are not allowed access to files) and one from MacLean Power Systems -- that last, another data point in the migration stats under the column "Can't shut down the HP 3000 as quickly as originally believed." Wesleyan Assurance Society in the UK raised its hand, where Jill Turner reports that "they have been looking to move off for years, but are only now just getting round to looking at this, which will take a while so we will still be using them. Far more reliable than the new kit."
In our very own hometown of Austin, Firstcare is still a user, but nearly all of its medical claims processing has been migrated to a new Linux platform. That's one migration that didn't flow the way HP expected, toward its other enterprise software platforms.
There is Cessna, still flying its maintenance applications under the HP 3000's wingspan. Locating other 3000 customers can be like finding aircraft in your flight pattern. A visual search won't yield much. That's one reason we miss the annual conferences that marked our reunions. This month will be the five-year anniversary of the last "Meeting by the Bay" organized by ScreenJet's Alan Yeo, for example. But the Wide World of the Web brings us all closer.
As a historical Web document that might have some current users on it -- including retail outlets for gift giving -- you can look at the "Companies that Use MPE" page of the OpenMPE website. (That's at openmpe.com these days). That list is more than 10 years old, so it represents the size of the community in the time just after HP's exit announcement. The list is more than 1,200 companies long. And there are plenty of Ecometry sites among the firms listed, including 9 West for shoes and Coldwater Creek for its vast range of clothing. The latter may very well be remaining on a 3000 for now, since retailers' fortunes define the pace of migrations.
And so, in an odd sort of way, patronizing a 3000-based retailer this season might help along a migration -- by increasing revenues that can be applied to an IT budget. It can make for a happier holiday when you can buy what you want, even when that includes a new application and enterprise environment.
September 05, 2014
How to make an HPSUSAN do virtual work
Out on the 3000-L newsgroup and mailing list, a 3000 user who's cloaked their identity as "false" asked about using HPSUSAN numbers while installing the CHARON emulator product from Stromasys. The question, and a few answers, were phrased in a tone of code that suggested there might be trouble from HP if an illicit number was used. HPSUSAN is a predefined variable on a 3000, one that's used to ensure software is not illegally replicated or moved to another system without the software vendor's consent.
People have been talking about HPSUSAN for decades by now, even as far back as the Toronto conference that produced the proceedings cover above. A 19-year-old paper from that meeting -- the last one which was not called HP World -- still has useful instructions on the utility of HPSUSAN. More on that in a moment, after we examine what HPSUSAN does today.
On the fully-featured edition of CHARON for the 3000, a current HP 3000's HPSUSAN number is required. Stromasys installs this number on a thumb drive, which is then plugged into the Intel-based server powering CHARON. There's a 36-hour grace period for using CHARON if that thumb drive malfunctions, or comes up missing, according to CHARON customer Jeff Elmer of Dairylea Cooperative.
But the HPSUSAN process and requirement is different for the freeware, A-202 model of CHARON that can be downloaded from the Stromasys website. As of this spring, users of this non-commercial/production model simply must enter any HPSUSAN number -- and affirm they have the right to use this number. Neither HP or Stromasys checks these freeware HPSUSAN numbers. That model of CHARON software isn't meant to replace any production 3000, or even a developer box.
The freeware situation and installing strategy all makes the newsgroup's answers more interesting. One consultant and 3000 manager suggested that a number from a Dell server would be just as binding as anything from a genuine Hewlett-Packard 3000 server.HPSUSAN identifies an HP-built 3000 server, not the instance of MPE/iX which reveals that number. The U in HPSUSAN stands for Unique, as in System Unique Serially Assigned Number. HP's Cathlene Mc Rae told us this spring that HPSUSAN is a one of a kind identifier for HP-built 3000 systems. What's more, HP's SUSAN doesn't designate an MPE/iX license, even though MPE is licensed via hardware ownership.
Mc Rae explained to us, and to a CHARON prospective user, "MPE hardware and software was created before the technology of virtual systems and emulators, in the 1970s. Licenses were based on hardware ownership."
Nearly 20 years ago, HPSUSAN was not the focal point that it's become since the start of this century. HP 3000s had these numbers swapped and pirated by companies such as Hardware House in 1998 and 1999, and the civil suit and criminal investigations led to low-jack jail time and fines. Some software and service companies even chose to adjust their MPE plans after HP's legal moves. You could be in the right in this kind of circumstance, but not have enough legal budget to prevail in a court against HP. Better to keep a profile low and unquestionably legal.
Of course, that was a different HP than the one which now is scuffling to maintain its sales, as well as watching its Business Critical Servers bleed off double-digit percentage sales dips every quarter. Whatever the legal budget to defend BCS systems like the 3000 was in 1999, these kinds of servers are not HP's focal point. They do remain HP's intellectual property, however. And so, the coded language of today's exchange, starting with the question from "false."
How would one go about getting a HPSUSAN to be able to stand up a HP-3000 VM to play with? Just playing around--not intending anything, but it would be nice to see if I can do it. BTW, my budget for this project is approximately the price of a plain Einstein Bros Bagel with cream cheese.
If you know what an HPSUSAN is, then you should know how to get one. If you only offer a bagel and cheese, you're up a gum tree without a paddle. Since you need to hide behind a 'false' name, you are onto a hiding and burning your bridges at both ends.
What Tom said is true. I would suggest that an HPSUSAN is about the same as the Service Tag on your Dell. I would try using that.
If a 3000 customer has ever owned a server, they've got an HPSUSAN number on file. (It's in your Gold Book, along with all of the other configuration data and software vendor contacts. Sorry, that's 1990s thinking. HP used to issue these notebook binders upon system purchase.)
And if you still have the right to use this number -- for example, if your HP 3000 was scrapped instead of sold -- then there's a great place to start looking for a number to input while CHARON A202 is installed on that Linux-Intel laptop.
HP has not advised its customers about the utility of HPSUSANs from mothballed 3000s. Using Mc Rae's explanation above, 3000s were based on hardware ownership. If a 3000 has been scrapped -- sold for parts or just materials -- nobody else owns that server. The ownership rights don't revert to Hewlett-Packard, do they? This might be one reason for these coded replies. Nobody knows for sure, and like Mc Rae said, "MPE hardware and software was created before the technology of virtual systems and emulators."
Twenty years after the server was created, though, HPSUSAN was a topic that led David Largent to publish an Interex '95 conference paper entitled 101 (More Or Less) Moral Things To Do With HPSusan. You can read it at the 3K Associates archive website. In part, this is how Largent explained what HPSUSAN was meant to do.
The name is actually an acronym for the words "System Unique Serially Assigned Number." So what purpose does it serve? About the same as any serial number does. Software companies have found they can use HPSUSAN as a way to tie a particular copy of their software to a given machine, thus controlling software piracy. They simply request your HPSUSAN and include that in some validation logic in their program that prohibits the running of their software if the HPSUSAN from the system does not match the value in the software.
And so, CHARON requests an HPSUSAN number for the freeware version of its 3000 model. The only thing it requires would be the basic number format, as well as the integrity of the manager installing it. HPSUSAN matters to vendors other than HP, too.
Some of these freeware CHARON installations start out as pilot projects to ensure applications will run correctly. And some of those apps use software which employ HPSUSAN checks. But MPE/iX is not among that software. That's a decision that HP chose. Perhaps it was one small way to ensure that MPE users can keep their 3000 environments alive. First the pilot install, then a proper production-grade install of CHARON. They all need HPSUSANs. Some requirements, though, are more stringent than others. That's what a non-commercial license for a virtual 3000 will buy you -- installation through integrity.
July 28, 2014
Taking a :BYE before a :SHUTDOWN
HP 3000 systems have been supporting manufacturing for almost as long as the server has been sold. ASK Computer Systems made MANMAN in the 1970s, working from a loaned system in a startup team's kitchen. MANMAN's still around, working today.
It might not be MANMAN working at 3M, but the Minnesota Minining & Manufacturing Company is still using HP 3000s. And according to a departing MPE expert at 3M, the multiple N-Class systems will be in service there "for at least several more years."
Mike Caplin is taking his leave of 3000 IT, though. Earlier this month he posted a farewell message to the 3000-L listserve community. He explained that he loved working with the computer, so much so that he bet on a healthy career future a decade-and-a-half ago. That was the time just before HP began to change its mind about low-growth product lines with loyal owners.
We love the part of Caplin's message where he gambled on his expertise and spent the last 15 years staying employed, instead of running from the 3000. We've been doing something similar here. This summer is the 13th we're writing and publishing since HP announced its end-date with the 3000 business. It can be sporting to try to figure who'll be the last to turn out the lights, but there's a good chance we won't be working anymore when it happens to MPE.
Tomorrow, I’ll type BYE for the last time. Actually, I’ll just X out of a Reflection screen and let the N-Class that I’m always logged in to log me out.
I started on a Series II in 1976 and thought I died and went to heaven after working on Burroughs and Univac equipment. The machine always ran; no downtime, easy online development, and those great manuals that actually made sense and had samples of code. I still have the orange pocket guide for the Series II.
I found this list about the same time that getting help from HP became a hit or miss. I always got a usable answer after posting a question, usually in under an hour. So the purpose of this is to say goodbye, but also to say thank you for all of the help over the years.
I was in a headhunter’s office about 15 years ago and he told me that I needed to get away from the 3000 because I’d never be able to make a living until I was ready to retire. I told him that he may be right, but that I was counting on knowing enough to be able to stay employed and that I intended to outlast MPE. I guess I got lucky and won that argument.
So that devoted MPE user has typed his last BYE. But MPE -- at least in some transitional mission at 3M -- has outlasted his days with the server. The community is still full of people who will make their exits before their HP 3000s do. Terry Floyd of the Support Group has said that at some manufacturing sites, there's a good chance the expertise will retire before the hardware does a shutdown. The marvel is to be able to go into retirement operating the same flavor of enterprise server as when you performed your first COLDSTART.
July 21, 2014
Maximum Disc Replacement for Series 9x7s
Software vendors, as well as in-house developers, keep Series 9x7 servers available for startup to test software revisions. There are not very many revisions to MPE software anymore, but we continue to see some of these oldest PA-RISC servers churning along in work environments.
9x7s, you may ask -- they're retired long ago, aren't they? Less than one year ago, one reseller was offering a trio for between $1,800 (a Series 947) and $3,200. Five years ago this week, tech experts were examining how to modernize the drives in these venerable beasts. One developer figured in 2009 they'd need their 9x7s for at least five more years. For the record, 9x7s are going to be from the early 1990s, so figure that some of them are beyond 20 years old now.
"They are great for testing how things actually work," one developer reported, "as opposed to what the documentation says, a detail we very much need to know when writing migration software. Also, to this day, if you write and compile software on 6.0, you can just about guarantee that it will run on 6.0, 6.5, 7.0 and 7.5 MPE/iX."
Some of the most vulnerable elements of machines from that epoch include those disk drives. 4GB units are installed inside most of them. Could something else replace these internal drives? It's a valid question for any 3000 that runs with these wee disks, but it becomes even more of an issue with the 9x7s. MPE/iX 7.0 and 7.5 are not operational on that segment of 3000 hardware.
Even though the LDEV1 drive will only support 4GB of space visible to MPE/iX 6.0 and 6.5, there's always LDEV2. You can use virtually any SCSI (SE SCSI or FW SCSI) drive, as long as you have the right interface and connector.
There's a Seagate disk drive that will stand in for something much older that's bearing an HP model number. The ST318416N 18GB Barracuda model -- which was once reported at $75, but now seems to be available for about $200 or so -- is in the 9x7's IOFDATA list of recognized devices, so they should just configure straight in. Even though that Seagate device is only available as refurbished equipment, it's still going to arrive with a one-year warranty. A lot longer than the one on any HP-original 9x7 disks still working in the community.One developer quipped to the community, five years ago this week, "On the disc front at least that Seagate drive should keep those 3000s running, probably longer than HP remains a Computer Manufacturer."
But much like the 9x7 being offered for sale this year, five years later HP is still manufacturing computers, including its Unix and Linux replacement systems for any 3000 migrating users.
So to refresh drives on the 9x7s, configure these Barracuda replacement drives in LDEV1 as the ST318416N -- it will automatically use 4GB (its max visible capacity) on reboot.
As for the LDEV2 drives, there are no real logical size limits, so anything under 300GB would work fine -- 300GB was the limit for MPE/iX drives until HP released its "Large Disk" patches for MPE/iX, MPEMXT2/T3. But that's a patch that wasn't written for the 9x7s, as they don't use 7.5.
Larger drives were not tested for these servers because of a power and heat dissipation issue. Some advice from the community indicates you'd do better to not greatly increase the power draw above what those original equipment drives require. The specs for those HP internal drives may be a part of your in-house equipment documentation. Seagate offers a technical manual for the 18GB Barracuda drive at its website, for power comparisons.
July 16, 2014
Kell carries key account of 3000 revival
We've come to learn that community icon Jeff Kell is battling a serious illness. While I wish this keystone of MPE wisdom a quick recovery, and the best wishes to his wife, I'd like to share some insights he relayed about the transition from Classic 3000s to the ultimate edition of the server he's worked on and cared for most of his career at the University of Tennessee at Chattanooga.
I'd asked Kell to explain what the HP CEO during that transition era, John Young, might have been talking about while the CEO told Computerworld in 1985 about the strategy of RISC. As the clipping from Computerworld to the left shows, Young was a lot less than clear about what RISC would do for HP's long-term computing plans. A comment in the second paragraph of the clipping -- about networking, one of Kell's most ardent studies -- made me want to reach out to him earlier this summer. Young's conflation of "9000 series terminals emulated the 3000 architecture in some ways, but not really completely" was something Kell could clear up.
I'm not aware of any similarities [Young noted] between 3000/9000 Series except after adoption of RISC, and they used the same processors/hardware. They may have shared some peripheral hardware earlier, but certainly had little in common until RISC. The 3000/9000 had practically nothing in common prior to that other than perhaps HP-IB peripherals.
Network-wise, the 9000-series was following the ARPA/Ethernet track, while the 3000 initially started down the IEEE/OSI architecture. Ethernet was only accepted by the 3000 as an afterthought, it was a checkbox on the NMCONFIG dialogue if you wanted to allow it, and it defaulted to OFF.
So unless Young was talking post-RISC (timeframe is wrong), I'm not sure how he would compare 3000/9000 lines at all. The initial RISC 3000s were in the last half of the 1980s. If I recall correctly, my "migration training" to the "new" 3000s was at the Atlanta response center around 1985 (or a little later) and we were expecting a 930. We ended up with a 950 (since the 930 sucked so badly.) But I do recall many of the details.
"At that time," he said, "we had stretched our Series-IIIs to the limit. HP had "loaned" us a 42 and 48 to "tie us over" until delivery of Spectrum. We had the week at the migration center in Atlanta and spent most of it doing switch stubs for our extensive set of SPL support routines. We finally got a 950 and never looked back, but we had several engineers scratching their heads in the process. We were doing some really peculiar stuff."
Those were "interesting times" indeed. I think at the time we had a Series III with 64 terminals attached (production), a Series III-R (development), a Series 40 or 42 (Library), an academic 44/48, a leftover Series III (academic), and that loaner pair of 42/48 (or 52/58?) to tie us over until Spectrum. We were long overdue for an upgrade, but no hardware was available yet to satisfy the need.
The 3000's direction on networking was most disturbing, taking the OSI standard model in the midst of our evolving Sun/Solaris Internet computers. We had 3000s on our LAN that could only talk to other 3000s on our LAN... while the rest of the server room was on the Internet. It was laughable.
It would be another decade before Posix came to MPE, and it started to play well with the other kids on the block. But unfortunately, a decade too late.
HP executives were taken up with the "Unix" movement... and the 9000s dominated their focus. The 3000s were just along for the ride. And looking back today, that wasn't such a great bet either.
Kell is a classic example of a chapter of living history -- and the lessons we learn from it -- that should be cherished by the community. After nearly 40 years, the decommissioned 3000s at his UTC shop were picked up for recycling. "We're now officially 3000 history," he said, "with nothing left on site."
July 07, 2014
User says licensing just a part of CHARON
Licensing the CHARON emulator solution at the Dairylea Cooperative has been some work, with some suppliers more willing to help in the transfer away from the compay's Series 969 than others. The $1.7 billion organization covers seven states and at least as many third party vendors. “We have a number of third party tools and we worked with each vendor to make the license transfers,” said IT Director Jeff Elmer.
“We won’t mention any names, but we will say that some vendors were absolutely wonderful to work with, while others were less so. It’s probably true that anyone well acquainted with the HP 3000 world could make accurate guesses about which vendors fell in which camp.”
Some vendors simply allowed a transfer at low cost or no cost; others gave a significant discount because Dairylea has been a long-time customer paying support fees. ”A couple wanted amounts of money that seemed excessive, but in most cases a little negotiation brought things back within reason,” Elmer said. The process wasn’t any different than a customary HP 3000 upgrade: hardware costs were low, but software fees were significant.“The cumulative expense of the third party software upgrades was nearly a deal-breaker,” he said. “In the end, our management was concerned enough about reliance on old disk drives that they made the decision to move forward. In our opinion it was money very well spent.”
Just as advertised, software that runs on an HP RISC server runs under CHARON. ‘Using those third party tools on the emulator is completely transparent,” Elmer said. “We had one product for which we had to make a command change in a job stream, and we had to make a mind-shift in evaluating what our performance monitoring software is telling us. Apart from that, it is business as usual.”
July 01, 2014
Northeastern cooperative plugs in CHARON
A leading milk and dairy product collective, a century-plus old, is drawing on the Stromasys emulator’s opportunity.
A $1.2 billion milk marketing cooperative — established for more than 100 years and offering services to farmers including lending, insurance and risk management — has become an early example of how to replace Hewlett-Packard’s 3000 and retain MPE software while boosting reliability.
The Dairylea Cooperative has been using the Stromasys CHARON emulator since the start of December, 2013, according to IT director Jeff Elmer. The organization that was founded in 1907 serves dairy owners across seven states in the US Northeast, a collective that had been using two Hewlett-Packard brand RISC servers for MPE operations.
Dairylea has taken its disaster recovery 3000 offline since December 1. Although HP’s physical 3000 server is still powered up, it’s been off the network all year while production continues. “Once we made the switch to the emulator, we never went back to the physical box,” Elmer said. ‘We can’t see any reason to at this point.”
“However much we may love HP’s 3000 hardware, the disk drives are still older than half of our IS department. Some of our users never knew there was a change.”The Stromasys emulator has been the easiest element to manage in the co-op’s Information Systems group. “For some weeks now I’ve been wanting to get around to making arrangements for the removal of both physical HP 3000s,” Elmer explained, “but the day-to-day distractions of the many other computer systems that don’t run MPE keep filling up the time. Since going to the emulator the only thing we’ve used the HP 3000 physical box for is to store a few files to tape for transfer to the emulator.”
As with any replacement solution, CHARON has required some learning to adjust everyday 3000 operations. We'll have more on that tomorrow.
June 26, 2014
3000 sages threwback stories on Thursday
Two weeks ago in the modest London pub Dirty Dick's, a few dozen veterans and sages of the 3000 system had their personal version of a Throwback Thursday. This is the day of the week when Facebook and Twitter users put out a piece of their personal history, usually in the form of a picture from days long past.
If pressed for a piece of June Throwback Thursday material, I might reach for our very first blog post. Nine years ago this month we kicked off our coverage of new, every-workday reporting. My first story was a tribute to a just-fallen comrade in the 3000 community. Bruce Toback died in that month the Newswire's blog was born. As I said in that first blog article -- "A Bright Light Winks Out" was already a throwback, before the term gained its current coin -- Toback was extraordinary, the kind of person that makes the 3000 community unique. He lived with a firm grip on life's handrail of humor. He died unexpectedly of a heart attack at age 48. As part of a gentle and generous Toback memorial, David Greer hosts pictures of Bruce like the one above. Many of these were taken as Toback became important to the Robelle Qedit for Windows project.
The passing of a special life is a good reason to celebrate what remains for all of us. That's probably what motivated those London veterans to gather at Dirty Dick's Pub this month to toss off stories and toss back drinks. Bob Green of Robelle (pictured here in a throwback picture in the spring of 2001, when he was working from his Anguilla island headquarters) shared some pub photos and a brief report about this month's Throwback Thursday for your community.
“It was great to catch up with 3000 colleagues from around the world: Steve Cooper, Dave Wiseman, Brian Duncombe, Kim Leeper, Brad Tashenberg, the Nutsfords and many more (about 20 in all). We exchanged notes on the current state of the machine -- especially the new emulator -- and discovered what each of us was doing. [Editor's Note: Duncombe (above) had made this trip in a record 48-hour-complete turnaround, from Canada to the UK and back. The intensity still burns bright for some of your community members.]
Green noted, while posting photos of Cooper and Leeper in conversation, or the sweet couples' photo (below) of Jeanette and Ken Nutsford, "An amazing number of people are still doing the same thing: helping customers with their IT concerns. But in reality, most of the time was spent swapping war stories from the past, which was great fun.
"Here are some photos from the party. Everyone is older, but perhaps you will remember some of them." This photo of the Nutsfords, ever the COBOL and HP Rapid standards-bearers, is something of a coup. The couple retired from the world of the 3000 to set off an epic career of cruise line travels, so catching them for a picture requires some foresight. They are circling the globe in a lifestyle that shows there's another, more rewarding kind of migration awaiting the luckiest of us.
June 23, 2014
New search for 3000 expertise surfaces
New openings for HP 3000 production and development jobs are uncommon prizes by now. Contract firms have been known to solicit MPE help while making a migration happen. Application support suppliers need IT professionals who know the details of mission-critical software, too.
But every once in awhile, a company that's still dedicated to using MPE software sends the word out that it's hiring for HP 3000 and MPE specifics. Such is the case from Measurement Specialties. The location is at the company's Hampton Roads, Virginia headquarters. The job listing from Terry Simpkins, Director of Business Systems for the manufacturer which uses MANMAN, Fortran and VEsoft's MPEX and Security/3000 -- among other platform-specific tools such as TurboIMAGE -- describes both classic and specialized enterprise IT skills.
"The leading manufacturer of sensors and sensing systems" is seeking a Business Analyst.
Areas of responsibility include:
- Daily user training and support
- Participate in projects in all functional areas of the business
- Serve as backup support for HP3000 operations and nightly processing
Key skills and capabilities include:
- Strong MANMAN experience and expertise
- Ability to read Fortran and perform some level of programming
- Strong understanding of MPEX scripting and Security/3000 menus
- Ability to handle multiple concurrent projects and tasks
The organization says that "If you are interested in a challenging and exciting opportunity with a dynamic and growing company," please contact
Terry W. Simpkins
Director of Business Systems
Measurement Specialties, Inc.
1000 Lucas Way, Hampton, VA 23666
Office: +1 757-766-4278
Mobile: +1 757 532-5685
June 20, 2014
Time to Sustain, If It's Not Time to Change
In the years after HP announced its 3000 exit, I helped to define the concept of homesteading. Not exactly new, and clearly something expected in an advancing society. Uncle Lars' homestead, at left, showed us how it might look with friendly droids to help on Tattooine. The alternative 3000 future that HP trumpeted in 2002 was migration. But it's clear by now that the movement versus steadfast strategy was a fuzzy picture for MPE users' future.
What remains at stake is transformation. Even to this week, any company that's relying on MPE, as well as those making a transition, are judging how they'll look in a year, or three, or five. We've just heard that software rental is making a comeback at one spot in the 3000 world. By renting a solution to remain on a 3000, instead of buying one, a manager is planning to first sustain its practices -- and then to change.
Up on the LinkedIn 3000 Community page I asked if the managers and owners were ready to purchase application-level support for 3000 operations. "It looks like several vendors want to sell this, to help with the brain-drain as veteran MPE managers retire." I asked that question a couple of years ago, but a few replies have bubbled up. Support has changed with ownership of some apps, such as Ecometry, and with some key tools such as NetBase.
"Those vendors will now get you forwarded to a call center in Bangalore," said Tracy Johnson, a veteran MPE manager at Measurement Specialties. "And by the way, Quest used to be quick on support. Since they got bought by Dell, you have to fill in data on a webpage to be triaged before they'll even accept an email."
Those were not the kind of vendors I was suggesting. Companies will oversee and maintain MPE apps created in-house, once the IT staff changes enough to lose 3000 expertise. But that led to another reply about why anyone might pursue the course to Sustain, when the strategy to Change seems overwhelming.Managed Business Systems, one of the original HP Platinum Migration partners, was ready to do this sustaining as far back as a decade ago. Companies like the Support Group, Pivital Solutions -- they're still the first-line help desks and maintainers for 3000 sites whose bench has grown thin. Fresche Legacy made a point of offering this level of service, starting from the last days when it was called Speedware. There are others willing to take over MPE app operations and care, and some of these vendors have feet planted firmly in the Change camp, as well as staking out the Sustain territory.
Todd Purdum of Sherlock Systems wondered on LinkedIn if there really was a community that would take on applications running under MPE. We ran an article last year about the idea of a backstop if your expertise got ill or left the company. Five years earlier, we could point to even smaller companies, and firms like 3K Ranger and Pro 3K are available to do that level of work. Purdum, by his figuring, believes such backstops are rare.
Although I agree with the need for sustained resources to keep an HP3000 running, I'm not sure that "several vendors" can provide this. We have been in the business for over 23 years, and as a leader in providing hardware and application support for HP 3000s and MPE, I don't see many other vendors truly being capable of providing this.
Purdum asked, tongue-in-cheek, if there was a 3000 resurgence on the way he didn't see coming. No one has a total view of this market. But anecdotal reports are about all anyone has been able to use for most of a decade. Even well-known tool vendors are using independent support companies for front-line support. Purdum acknowledged that the support would be there, but wondered who'd need it.
Customers who use MPE (the HP 3000) know their predicament, and offering more salvation does not help them move into the right direction. I am only a hardware support company (that had to learn all HP 3000 applications) and it disappoints me a little that the companies you mentioned, most of which are software companies, haven't developed software that will allow these folks to finally move on and get off of this retired platform.
I can't change it, I just sustain it... applications and all.
Sustaining mission-critical use of MPE is the only choice for some companies have in 2014. Their parent corporations aren't ready for a hand-off, or budget's not right, or yes, their app vendor isn't yet ready with a replacement app. That's what's leading to software rentals. When a company chooses to homestead, it must build a plan to Sustain. HP clearly retired its 3000 business more than three years ago. But that "final" moving on, into the realms of real change, follows other schedules, around the world. On the world of Tattooine, Lars first changed by setting up a moisture farm, then sustained. And then everything changed for him and Luke Skywalker. Change-sustain-change doesn't have a final state.
June 17, 2014
How a Fan Can Become a Migration Tool
We heard this story today in your community, but we'll withhold the names to protect the innocent. A Series 948 server had a problem, one that was keeping it offline. It was a hardware problem, one on a server that was providing archival lookups. The MPE application had been migrated to a Windows app five years ago. But those archives, well, they often just seem to be easier to look up from the original 3000 system.
There might be some good reasons to keep an archival 3000 running. Regulatory issues come to mind first. Auditors might need original equipment paired with historic data. There could be budget issues, but we'll get to that in a moment.
The problem with that Series 948: it was overheating. And since it was a server of more than 17 years of service, repairing it required a hardware veteran. Plus parts. All of which is available, but "feet on the street" in the server's location, that can be a challenge. (At this point a handful of service providers are wondering where this prospective repair site might be. The enterprising ones will call.)
But remember this is an archival 3000. Budget, hah. This would be the time to find a fan to point at that overheating 17-year-old system. That could be the first step in a data migration, low-tech as it might seem.From the moment the fan makes it possible to boot up, this could be the time to get that archival data off the 3000. Especially since the site's already got a replacement app on another piece of newer hardware, up and running. There's a server there, waiting to get a little more use.
Moving data off an archival server is one of the very last steps in decommissioning. If you've got a packaged application, there are experts in your app out there -- all the big ones, like Ecometry, MANMAN, Amisys -- that can help export that data for you. And you might get lucky and find that's a very modest budget item. You can also seek out data migration expertise, another good route.
But putting more money into a replacement Hewlett-Packard-branded 3000 this year might be a little too conservative. It depends on how old the 3000 system is, and what the hardware problem would be. If not a fan, then maybe a vacuum cleaner or shop vac could lower the temperature of the server, with a good clean-out. Funk inside the cabinet is common, we've seen.
Overheating old equipment could be a trigger to get the last set of archives into a SQL Server database, for example, one designated only for that. Heading to a more modern piece of hardware might have led you into another kind of migration, towards the emulator, sure. But if your mission-critical app is already migrated, the fan and SQL Server -- plus testing the migrated data, of course -- might be the gateway to an MPE-free operation, including your archives.
May 06, 2014
PowerHouse users study migration flights
A sometimes surprising group of companies continue to use software from the PowerHouse fourth generation language lineup on their HP 3000s. At Boeing, for example -- a manufacturer whose Boeing 737 assembly line pushes out one aircraft's airframe every day -- the products are essential to one mission-critical application. Upgrade fees for PowerHouse became a crucial element in deciding whether to homestead on the CHARON emulator last year.
PowerHouse products have a stickiness to them that can surprise, here in 2014, because of the age of the underlying concept. But they're ingrained in IT operations to a degree that can make them linchpins. In a LinkedIn Group devoted to managing PowerHouse products, the topic of making a new era for 4GL has been discussed for the past week. Paul Stennett, a group systems manager with UK-based housebuilder Wainhomes, said that his company's transition to an HP-UX version of PowerHouse has worked more seamlessly -- so far -- than the prospect of replacing the PowerHouse MPE application with a package.
"The main driver was not to disrupt the business, which at the end of the day pays for IT," Stennett said. "It did take around 18 months to complete, but was implemented over a weekend. So the users logged off on Friday on the old system, and logged onto the new system on Monday. From an application point of view all the screens, reports and processes were the same."
This is the lift-and-shift migration strategy, taken to a new level because the proprietary language driving these applications has not changed. Business processes -- which will get reviewed in any thorough migration to see if they're still needed -- have the highest level of pain to change. Sometimes companies conclude that the enhancements derived from a replacement package are more than offset by required changes to business processes.
Enter the version of PowerHouse that runs on HP's supported Unix environment. It was a realistic choice for Stennett's company because the 4GL has a new owner this year in Unicom."With the acquisition of PowerHouse by UNICOM, and their commitment to developing the product and therefore continued support," Stennett posted on the LinkedIn group, "is it better to migrate PowerHouse onto a supported platform (from HP 3000 to HP-UX) rather than go for a complete re-write in Java, with all of its risks. To the user was seamless, other than they have to use a different command to logon. The impact to the businesses day to day running was zero."
The discussion began with requests on information for porting PowerHouse apps to Java. The 4GL was created with a different goal in mind than Java's ideal of "write once, run anywhere." Productivity was the lure for users who moved to 4GLs such as PowerHouse, Speedware, and variants such as Protos and Transact. All but Protos now have support for other platforms.
And HP's venerated Transact -- which once ran the US Navy's Mark 85 torpedo facility at Keyport, Wash. -- can be replaced by ScreenJet's TransAction and then implemented on MPE. ScreenJet, which partnered with Transact's creator David Dummer to build this replacement, added that an MPE/iX TransAction implementation would work as a testing step toward an ultimate migration to other environments.
Bob Deskin, a former PowerHouse support manager who retired from IBM last year, sketched out why the fourth generation language is preserving so many in-house applications -- sometimes on platforms where the vendor has moved on, or set an exit date as with HP's OpenVMS.
Application systems, like many things, have inertia. They tend to obey Newton's first law. A body at rest tends to remain at rest unless acted upon by an outside force. The need for change was that outside force. When an application requires major change, the decision must be made to do extensive modifications to the existing system, to write a new system, or to buy a package. During the '90s, the answer was often to buy a package.
But packages are expensive so companies are looking at leveraging what they have. If they feel that the current 4GL application can't give them what they need, but the internal logic is still viable, they look for migration or conversion tools. Rather than completely re-write, it may be easier to convert and add on now that Java and C++ programmers are readily available.
Deskin added as part of his opinion of what happened to 4GLs that they were never ubiquitous -- not even in an environment like the HP 3000's, where development in mainstream languages might take 10 times longer during the 1970s.
There weren't enough programmers to meet the demand. Along came 4GLs and their supposed promise of development without programmers. We know that didn't work out. But the idea of generating systems in 10 percent of the time appealed to many. If you needed 10 percent of the time, maybe you only needed 10 percent of the programmers.
The 4GL heyday was the '80s. With computers being relatively inexpensive and demand for systems growing, something had to fill the void. Some programmers caught the 4GL bug, but most didn't. There was still more demand than supply, so studying mainstream languages almost guaranteed a job.
Now even mainstream languages like COBOL and FORTRAN are out of vogue. COBOL was even declared extinct by one misinformed business podcast on NPR. The alternatives are, as one LinkedIn group member pointed out, often Microsoft's .NET or Oracle's Java. (Java wasn't considered a vendor's product until Oracle acquired it as part of its Sun pickup. These days Java is rarely discussed without mention of its owner, perhaps because the Oracle database is so ubiquitous in typical migration target environments.)
Migration away from a 4GL like PowerHouse -- to a complete revision with new front end, back end databases, reporting and middleware -- can be costly by one LinkedIn member's accounts. Krikor Gellekian, whose name surfaces frequently in the PowerHouse community, added that a company's competitive edge is the reward for the lengthy wade through the surf of 4GL departures.
"It is not simple, it takes time and is expensive, and the client should know that in advance," Gellekian wrote. "However, I always try to persuade my clients that IT modernization is not a single project; it is a process. And adopting it means staying competitive in their business."
Deskin approached the idea that 4GLs might be a concept as extinct as that podcast's summary of COBOL.
Does this mean that the idea of a 4GL is dead? Absolutely not. The concept of specifying what you want to do rather than how to do it is still a modern concept. In effect, object-oriented languages [like Java] are attempting to do the same thing -- except they are trying to be all things to all people and work at a very low level. However, it takes more than a language these days to be successful. It also requires a modern interface. Here's hoping.
April 23, 2014
Emulator's edition earns closer look in call
First of two parts
The recent CAMUS user group meeting, conducted as a conference call, promised some testing and analysis of the Stromasys CHARON HP 3000 emulator -- as done by an outsider. MB Foster is an insider to the HP 3000 community, but the vendor doesn't have an affiliation with Stromasys as a partner. Not at this point, although there are always opportunities for longstanding vendors to join their customers with such a new solution.
CEO Birket Foster said the company's been asked by its customers if MB Foster products would run safely in the CHARON environment. The question not only has been of high interest to 3000 managers. One similar answer lies in the Digital environment, where CHARON has more than 4,000 installations including some CAMUS members who run MANMAN in a VAX system. All's well over there, they report.
CHARON is so much newer in 3000-land. Principal Consultant Arnie Kwong of MB Foster outlined some of the research results from testing on an Intel i7 server with 64GB of memory and SSD storage, as well as a more everyday 8GB capacity box, albeit an AMD-based system. (Both systems can run CHARON for the 3000 emulation.) Wong said using a private VMware cloud, or private backup machines, are common computing-share practices that deserve extra attention with new possibilities of CHARON. "What will it let me do that's different?" he asked.
One of the assumptions of using cloud infrastructure and these new capabilities is whether the fundamental operating characteristics, business processes and business rules embedded in applications like MANMAN are sufficient for what you're doing now. Having talked to lots of MANMAN customers, all of the industry-standard and regulatory practices can be impacted if we do something major like shifting the platform.
Kwong went on to forecast the use of CHARON in a cloud-based implementation and ponder if that use affects regulatory compliance, as well as "the ability to operate on a global basis, and what new opportunities we can do in that mold." He said he'd confine his comments to instances where a cloud-based infrastructure was already in use at MB Foster customer sites. "But our leading candidate to do this kind of thing isn't a VMware kind of architecture." CHARON, Kwong noted, relies heavily on VMware to do its emulation for HP 3000 operations.Most members of the user group on the call have pieces of their IT infrastructure running in a cloud aspect, such as Google Mail. "They have Internet-based functionalities, global applications that function well. We looked at the HP 3000 applications such as MANMAN that are enabled and helped by having all of that architecture in place." The 3000 is a platform service inside a cloud environment, Kwong said.
Migrating a 3000 to CHARON means "you have to have some systems engineering and systems administration done to bring it up. A key is to look at sizing of the environments and properly sizing data and program sizes and shapes, as far as the size of the application portfolio. You should look at what you are going to be able to effectively maintain."
Testing for such an emulated environment may require more time from technical staff that the time you have available, considering the depth of MPE/3000 knowledge in many sites. "Concurrently, you need to have folks with knowledge of your cloud infrastructure. A key takeaway for this call is you need to pay attention to staff availability of people with a deep technical knowledge, both on the HP side and in your cloud infrastructure."
Kwong said that managers can snapshot production states, to on to things such as a physical inventory cycle. "In a case of global operations, that might not have been easily possible before. Using the virtualization infrastructure offered via CHARON, and storage infrastructure in particular, you can do functions you just weren't able to do in the HP 3000 environment that's tied to physical hardware."
In an evaluation from MB Foster that could lead to implementing CHARON, the company looks at the business cycle activities that need those kind of functions, "and study how we'd map it; for example, could I give one to three days more production time."
One Stromasys representative on the call checked to see if the MB Foster results were off the limited-use Freeware edition, or a full-production installation. Kwong said it was full-production, and the Stromasys rep said the company didn't have a relationship yet with MB Foster. The two said they'd take that issue offline. Regarding the license movement needed to enable CHARON use, Kwong said it wasn't an automatic assumption that everything could move without a major cost, but "it's fair to say that in a lot of cases you'll be able to move without a tremendous cost in relicensing.
Foster said that slides which summarize its results and planned migration processes for the CHARON testing will be available in a forthcoming MB Foster Webinar Wednesday.
April 21, 2014
A week-plus of bleeds, but MPE's hearty
There are not many aspects of MPE that seem to best the offerings from open source environments. For anyone who's been tracking the OpenSSL hacker-door Heartbleed, though, the news is good on 3000 vulnerability. It's better than more modern platforms, in part because it's more mature. If you're moving away from mature and into migrating to open source computing, then listen up.
Open source savant Brian Edminster of Applied Technologies told us why MPE is in better shape.
I know that it's been covered other places, but don't know if it's been explicitly stated anywhere in MPE-Land: The Heartbleed issue is due to the 'heartbeat' feature, which was added to OpenSSL after any known builds for MPE/iX.
That's a short way of saying: So far, all the versions of OpenSSL for MPE/iX are too old to be affected by the Heartbleed vulnerability. Seems that sometimes, it can be good to not be on the bleeding edge.
However, the 3000 IT manager -- a person who usually has a couple of decades of computing experience -- may be in charge of the more-vulnerable web servers. Linux is used a lot for this kind of thing. Jeff Kell, whose on-the-Web servers deliver news of 3000s via the 3000-L mailing list, outlined repairs needed and advice from his 30-plus years of networking -- in MPE and all other environments.About 10 days after the news rocked the Web, Kell -- one of the sharpest tools in the drawer of networking -- posted this April 17 summary on the challenges and which ports to watch.
Unless you've had your head in the sand, you've heard about Heartbleed. Every freaking security vendor is milking it for all it's worth. It is pretty nasty, but it's essentially "read-only" without some careful follow-up.
Most have focused on SSL/HTTPS over 443, but other services are exposed (SMTP services on 25, 465, 867; LDAP on 636; others). You can scan and it might show up the obvious ones, but local services may have been compiled against "static" SSL libraries, and be vulnerable as well.
We've cleaned up most of ours (we think, still scanning); but that just covers the server side.
There are also client-side compromises possible.
And this stuff isn't theoretical, it's been proven third-party...
Lots of folks say replace your certificates, change your passwords, etc. I'd wait until the services you're changing are verified secure.
Most of the IDS/IPS/detections of the exploits are broken in various ways. STARTTLS works by negotiating a connection, establishing keys, and bouncing to an encrypted transport. IDS/IPS can't pick up heartbleed encrypted. They're after the easy pre-authenticated handshake.
It's a mess for sure. But it’s not yet safe to necessarily declare anything safe just yet.
Stay tuned, and avoid the advertising noise.
April 09, 2014
How SSL's bug is causing security to bleed
Computing's Secure Sockets Layer (SSL) forms part of the bedrock of information security. Companies have built products around SSL, vendors have wired its protocols into operating systems, vendors have applied its encryption to data transport services. Banks, credit card providers, even governments rely on its security. In the oldest days of browser use, SSL displayed that little lock in the bottom corner that assured you a site was secure -- so type away on those passwords, IDs, and sensitive data.
In a matter of days, all of the security legacy from the past two years has virtually evaporated. OpenSSL, the most current generation of SSL, has developed a large wound, big enough to let anyone read secured data who can incorporate a hack of the Heartbeat portion of the standard. A Finnish security firm has dubbed the exposed hack Heartbleed.
OpenSSL has made a slow and as-yet incomplete journey to the HP 3000's MPE/iX. Only an ardent handful of users have made efforts to bring the full package to the 3000's environment. In most cases, when OpenSSL has been needed for a solution involving a 3000, Linux servers supply the required security. Oops. Now Linux implementations of OpenSSL have been exposed. Linux is driving about half of the world's websites, by some tallies, since the Linux version of Apache is often in control.
One of the 3000 community's better-known voices about mixing Linux with MPE posted a note in the 3000 newsgroup over the past 48 hours to alert Linux-using managers. James Byrne of Harte & Lyne Ltd. explained the scope of a security breach that will require a massive tourniquet. To preface his report, the Transport Layer Security (TLS) and SSL in the TCP/IP stack encrypt data of network connections. They have even done this for MPE/iX, but in older, safe versions. Byrne summed up the current threat.
There is an exploit in the wild that permits anyone with TLS network access to any system running the affected version of OpenSSL to systematically read every byte in memory. Among other nastiness, this means that the private keys used for Public Key Infrastructure on those systems are exposed and compromised, as they must be loaded into memory in order to perform their function.
It's something of a groundbreaker, this hack. These exploits are not logged, so there will be no evidence of compromises. It’s possible to trick almost any system running any version of OpenSSL released over the past two years into revealing chunks of data sitting in its system memory.The official security report on the bug, from OpenSSL.org, does its best to make it seem like there's a ready solution to the problem. No need to panic, right?
A missing bounds check in the handling of the TLS heartbeat extension can be used to reveal up to 64k of memory to a connected client or server.
Only 1.0.1 and 1.0.2-beta releases of OpenSSL are affected, including 1.0.1f and 1.0.2-beta1.
Thanks for Neel Mehta of Google Security for discovering this bug and to Adam Langley and Bodo Moeller for preparing the fix.
Affected users should upgrade to OpenSSL 1.0.1g. Users unable to immediately upgrade can alternatively recompile OpenSSL with -DOPENSSL_NO_HEARTBEATS.
1.0.2 will be fixed in 1.0.2-beta2
For the technically inclined, there's a great video online that explains all aspects of the hack. Webserver owners and hosts have their work to do in order to make their sites secure. That leaves out virtually every HP 3000, the server that was renamed e3000 in its final HP generation to emphasize its integration with the Internet. Hewlett-Packard never got around to implementing OpenSSL security in its web services for MPE/iX. 3000 systems are blameless, but that doesn't matter as much as insisting your secure website providers apply that 1.0.1g upgrade.
The spookiest part of this story is that without the log evidence, nobody knows if Heartbleed has been used over the past two years. Byrne's message is directed at IT managers who have Linux-driven websites in their datacenters. Linux has gathered a lot of co-existence with MPE/iX over the last five years and more. This isn't like a report of a gang shooting that's happened in another part of town. Consider it more of a warning about the water supply.
In a bit of gallows humor, it looks as if the incomplete implementation of OpenSSL, frozen in an earlier edition of the software, puts it back in the same category as un-patched OpenSSL web servers: not quite ready for prime time.
April 07, 2014
MPE patches still available, just customized
Last week a 3000 manager was probing for the cause of a Command Interface CI error on a jobstream. In the course of the quest, an MPE expert made an important point: Patches to repair such MPE/iX bugs are still available. Especially from the seven companies which licensed HP's source code for the HP 3000s.
This mention of MPE bug repair was a reminder, actually, that Hewlett-Packard set the internals knowledge of MPE free back in 2010. Read-only rights to the operating system source code went out to seven companies worldwide, including some support providers such as Pivital Solutions and Allegro Consultants.
The latter's Stan Sieler was watching a 3000 newsgroup thread about the error winding up. Tracy Johnson, the curator of the 3000 that hosts the EMPIRE game and a former secretary to OpenMPE, had pointed out that his 3000 sometimes waits longer than expected after a PAUSE in a jobstream.
I nearly always put a CONTINUE statement before a PAUSE in any job. Over the years I have discovered that sometimes the CPU waits "longer" than the specified pause and fails with an error.
A lively newsgroup discussion of 28 messages ensued. It was by far the biggest exchange of tech advice on the newsgroup in 2014, so far. Sieler took note of what's likely to be broken in MPE/iX 7.5, after an HP engineer had made his analysis of might need a workaround. Patches and workarounds are a continuing part of the 3000 manager's life, even here in the second decade of the 3000's Afterlife. You can get 'em if you want 'em.A workaround is the more likely of repairs for something that's not operating correctly in MPE, by this year. Patches were a free HP 3000 element, and those that HP created still are free today -- unlike the situation for HP's still-supported servers. The dilemma is that the final round of patches HP built weren't tested to HP's satisfaction. Plus, there's no more vendor work on new repairs.
Enter the third party supporters, the companies I call independent support providers. They know the 3000 as well as anybody left at HP, so long as they're a party to the source code for the operating system. In many cases, a binary patch isn't what a customer wants. Such a thing has to be tested, and a lot of production 3000s are under lockdown today. Changes are not invited.
But in the case of an MPE/iX jobstream PAUSE error, there's always a chance for a fix. HP's Jim Hawkins looked at Johnson's problem and ranked the causes Nos. 1-4. Number 4 was "possible MPE/iX bug."
Sieler said that it looked like this was a genuine MPE/iX flaw. What to do, now that the MPE/iX lab at HP -- which once included Hawkins -- has gone dark? Sieler pointed to patching.
After analyzing hxpause, the executor responsible for implementing the CI PAUSE command, I suspect there is a bug in the MPE/iX internal routine "pausey", which hxpause uses. The bug appears to be triggerable by :BREAKJOB/:RESUMEJOB, but I have not characterized precisely what triggers it. It is, however, apparently the result of the equivalent of an uninitialized variable.
I believe Allegro could develop a patch, should a customer be interested in it.
Patches beyond the lifespan of an HP lab are a touchy topic. A binary patch, as Allegro's Steve Cooper describes this kind of assignment, is likely to live its life in just one HP 3000 installation. It's a creation to be tested, like any patch.
And now it seems that patches are not only a for-pay item, but something to be guarded. HP even pressed a lawsuit against an independent company when the vendor observed that its patches were being distributed by the indie. No money changed hands in the suit settlement, but the support company said it would stop redistributing HP's patches.
This kind of protective culture from systems vendors is endemic by now, according to Source Direct's Bill Hassell. "This is a hot topic, both for customers as well as third party support organizations," he reported. "There have been very strong reactions from customers to recent statements about firmware restrictions." Hassell, well-known as an HP-UX expert among former Interex user group members, pointed to a handful of articles from HP's own blog and the industry press such as ZDNet, or one from PC World.
But the first one Hassell pointed at was the message from HP's own Mary McCoy, VP of Support for HP Servers, Technology Services. It's titled Customers for Life. In essence, the February posting says HP's firmware only gets an upgrade for "customers with a valid warranty, Care Pack Service, or support agreement."
We know this is a change from how we’ve done business in the past; however, this aligns with industry best practices and is the right decision for our customers and partners. This decision reinforces our goal to provide access to the latest HP firmware, which is valuable intellectual property, for our customers who have chosen to maximize and protect their IT investments.
In the face of this, and other HP announcements such as ProLiant patch availability, the customers who are commenting at HP's website are not happy. One noted that "the customer segment who will suffer the most from this revision in HP firmware availability will be the small and medium businesses performing their own in-house IT support." Some say the pay-for-patch mandate is only going to drive them to other vendors for small business servers. HP asserts that every vendor is doing this by now.
Enter the indie patching potential for MPE/iX. Binary patches are much more of a possibility when source code is in the hands of a support company. As far as I know, the source for HP-UX, or any other proprietary Unix, isn't in the wild, and the same can be said for Windows. Linux source is always available, of course. Nobody is going to be tagged as a Customer for Life when they choose Linux.
But that's also true of MPE/iX. Enter an indie support relationship and you get the benefits of that vendor's expertise, based upon the level of their understanding of MPE. Leave that relationship and you're not penalized. You're just on the hunt now for another support vendor of equal caliber.
A support company's caliber is measured by the way it conducts its business practices, not just what it knows how to create or fix. This vendor lock-in is something familiar to a 3000 owner. But it was technology, not business decisions, which enforced such lock-in during the 20th Century. The indie companies have a patch for the current era's lock-in error.
March 27, 2014
Beyond 3000's summit, will it keep running?
If you consider the last 40 years and counting to be a steady rise in reputation elevation for the HP 3000 and MPE -- what computer's been serving business longer, after all? -- then 2027 might be the 3000's summit. A couple of 3000 experts have climbed a summit together, as the photo of Guy Paul and Craig Lalley above proves. What a 3000 might do up there in 20 years prompted some talk about 2027 and what it means.
The two 3000 veterans were climbing Washington state's second highest mountain, Mt. Adams, whose summit is at 12,280 feet. On their way up, Paul and his 14 year old grandson had just made the summit and ran into Lalley, and his 14 year old son, on their way to the top.
The trek was announced on the 3000 newsgroup last year. At the time, some of the group's members joked that a 3000 could climb to that elevation if somebody could haul one up there. "Guy is a hiking stud," said his fellow hiker Lalley. "Rumor has it that Guy had a small Series 989 in his back pack. I wasn't impressed until I heard about the UPS."
After some discussion about solar-powered computing, someone else said that if it was started up there on Mt. Adams with solar power, the 3000 would still be running 20 years later.
Then a 3000 veteran asked, "But won't it stop running in 2027?" That's an important year for the MPE/iX operating system, but not really a date of demise. Such a 3000 -- any MPE/iX system -- can be running in 20 years, but it will use the wrong dates. Unless someone rethinks date handling before then.
Jeff Kell, whose HP 3000s stopped running at the University of Tennessee at Chattanooga in December, because of a shutdown post-migration, added some wisdom to this future of date-handling.
"Well, by 2027, we may be used to employing mm/dd/yy with a 27 on the end, and you could always go back to 1927. And the programs that only did "two-digit" years would be all set. Did you convert all of 'em for Y2K? Did you keep the old source?"Kell added that "Our major Y2K issue was dealing with a "semester" which was YY01 for fall, YY02 for spring, and so forth. We converted that over to go from 9901 (Fall 1999) to A001 (Fall 2000), so we were good for another 259 years on that part. Real calendar dates used 4-digit years (32-bit integers, yyyymmdd)."
At that summit, Paul said that two climbers "talked for a few minutes we made tentative plans to climb Oregon's tallest mountain, Mt. Hood, pictured in the background. We have since set a date of May 16th."
We've written before on the effects of 2027's final month on the suitability of the 3000 for business practice. Kell's ideas have merit. I believe there's still enough wizardry in the community to take the operating system even further upward. The HP iron, perhaps not so much. By the year 2028, even the newest servers will still be 25 years old. Try to imagine a 3000 that was built in 1989, running today.
Better yet, please report to us if you have such a machine, hooked up in your shop.
Why do people climb mountains? The legend is that the climber George Mallory replied, "because it is there." 2028 is still there, waiting for MPE to arrive. Probably on the back of some Intel-based server, bearing Linux -- unless neither of those survives another 14 years. For Intel, this year marks 15 years of service for the Xeon processor, currently on the Haswell generation. Another 25 years, and Xeon will have done as much service as MPE has today.
There is no betting line on the odds of survival for Xeon into the year 2039. By that date, even Unix will have a had its own date-handling issue. The feeling in the Linux community is that a date solution will arrive in time.
March 18, 2014
Customizing apps keeps A500 serving sites
HP's A-Class 3000s aren't that powerful, and they're not as readily linked to extra storage. That's what the N-Class systems are designed to do. But at one service provider's shop, the A500 is plenty powerful enough to keep a client's company running on schedule, and within budget. The staying power comes from customization, that sticky factor which is helping some 3000s remain in service.
The A500 replaced a Series 987 about a year ago. That report is one point of proof that 9x7 systems are still being replaced. It's been almost two decades since the 9x7s were first sold, and more than 15 years since the last one was built. The service company, which wants to remain unnamed, had good experience with system durability from the 3000 line.
We host a group of companies that have been using our system for over 20 years. So, we’re planning on being around for a while. One of these customers may migrate to a Windows-based system over the next few years, but I anticipate that this will be a slow process, since we have customized their system for them over the years.
The client company's top brass wants to migrate, in order to get all of its IT onto a single computing environment. That'd be Windows. But without that corporate mandate to make the IT identical in every datacenter, the company would be happy staying with the 3000, rather than looking at eventual migration "in several years' time." It will not be the speed of the server that shuts down that company's use of an A500. It will be the distinction that MPE/iX represents.There are many servers at a similar price tag, or even cheaper, which can outperform an A500. HP never compared the A-Class or N-Class systems to anything but other HP 3000s. By the numbers, HP's data sheet on the A-Class lineup lists the top-end of the A500s -- a two-CPU model with 200 MHz chips -- at five times the performance of those entry-level $2,000 A400s being offered on eBay (with no takers, yet.) The A500-200-200 tops out at 8GB of memory. But the chip inside that server is just a PA-8700, a version of PA-RISC that's two generations older than the ultimate PA chipset. HP stopped making PA-RISC chips altogether in 2009.
HP sold that 2-way A500 at a list price of just under $42,000 at the server's 2002 rollout. In contrast, those bottom-end A400s had a list price of about $16,000 each. Both price points didn't include drives, or tape devices. Our columnist at the time, John Burke, reported on performance upgrades in the newer A-Class systems by saying
There is considerable controversy in the field about the A-Class servers in particular, with many people claiming these low-end boxes have been so severely crippled (when compared to their non-crippled HP-UX brothers) as to make them useless for any but the smallest shops. Even if you accept HP’s performance rating (and many people question its accuracy), the A400-100-110 is barely faster then the 10-year-old 928 that had become the de-facto low-end system.
I see these new A-Class systems as a tacit agreement by HP that it goofed with the initial systems.
The power of the iron is just a portion of the performance calculation, of course. The software's integration with the application, and access to the database and movement of files into and out of memory -- that's all been contributing to the 3000's reputation. "I’ve been working on the HP since 1984 and it’s such a workhorse!" said the service provider's senior analyst. "I've seen other companies that have gone from the 3000 to Windows-based systems, and I hear about performance issues."
Not all migrations to Windows-based ERP, for example, give up performance ground when leaving the 3000 field. We've heard good reports on Microsoft Dynamics GP, a mature set of applications that's been in the market for more than a decade. Another is IFS, which pioneered component-based ERP software with IFS Applications, now in its seventh generation.
One area where the newer products -- which are still making advances in capability, with new releases -- have to give ground to 3000 ERP is in customization. Whatever the ERP foundation might be at that service provider's client, the applications have grown to become a better fit to the business practices at that client company. ERP is a set of computing that thrives on customization. This might be the sector of the economy which will be among the last to turn away from the 3000 and MPE.
March 13, 2014
Can COBOL flexibility, durability migrate?
In our report yesterday on the readiness for CHARON emulation at Cerro Wire, we learned that the keystone application at that 3000 shop began as the DeCarlo, Paternite, & Associates IBS/3000 suite. That software is built upon COBOL. But at Cerro Wire, the app's had lots of updating and customization and expansion. It's one example of how the 3000 COBOL environment keeps on branching out, in the hands of a veteran developer.
That advantage, as any migrating shop has learned, is offset by finding COBOL expertise ready to work on new platforms. Or a COBOL that does as many things as the 3000's did, or does them in the same way.
OpenCOBOL and Micro Focus remain two of the favorite targets for 3000 COBOL migrations. The more robust a developer got with COBOL II on MPE, however, the more challenge they'll find in replicating all of that customization.
As an example, consider the use of COBOL II macros, or the advantage of COBOL preprocessors. The IBS software "used so many macros and copylibs that the standard COBOL compiler couldn't handle them," Terry Simpkins of Measurement Specialities reported awhile back. So the IBS creators wrote a preprocessor for the COBOL compiler on the 3000. Migrating a solution like that one requires careful steps by IT managers. It helps that there's some advocates for migrating COBOL, and at least one good crossover compiler that understands the 3000's COBOL nuances.Alan Yeo reminds us that one solution to the need for a macro pre processor is AcuCOBOL. "It has it built in," he says. "Just set the HP COBOL II compatibility switch, and hey presto, it will handle the macros."
Yeo goes on to add that "Most of the people with good COBOL migration toolsets have created COBOL preprocessors to do just this when migrating HP COBOL to a variety of different COBOL compilers. You might just have to cross their palms with some silver, but you'll save yourself a fortune in effort." Transformix is among those vendors. AMXW support sthe conversion of HP COBOL to Micro Focus as well as AcuCOBOL.
Those macros were a staple for 3000 applications built in the 1980s and 90s, and then maintained into the current century, some to this very day. One of the top minds in HP's language labs where COBOL II was born thinks of macros as challenges to migrations, though. Walter Murray has said
I tried to discourage the use of macros in HP COBOL II. They are not standard COBOL, and do not work the same, or don't work at all, on other compilers. But nobody ever expects that they will be porting their COBOL. One can do some very powerful things with macros. I have no argument there.
COBOL II/iX processes macros in a separate pass using what was called MLPP, the Multi-Language PreProcessor. As the name implies, it was envisioned as a preprocessor that could be used with any number of HP language products. But I don't think it was used anywhere except COBOL II, and maybe the Assembler for the PA-RISC platform.
Jeff Kell, whose 3000 shutdown at the University of Tennessee at Chattanooga we've chronicled recently, said macros were a staple for his shop.
In moving to COBOL II we lived on macros. Using predictable data elements for IMAGE items, sets, keys, status words and so forth, we reduced IMAGE calls to simple macros with a minimum of parameters.
We also had a custom preprocessor. We had several large, modular programs with sequential source files containing various subprograms. The preprocessor would track the filename, section, paragraph, last image call, and generate standard error handling that would output a nice "tombstone" identifying the point in the program where the error occurred. It also handled terminal messages, warnings, and errors (you could put the text you wanted displayed into COBOL comments below the "macro" and it filled in code to generate a message catalog and calls to display the text).
It's accepted as common wisdom that COBOL is yesterday's technology, even while it still runs today's mission critical business. How essential is that level of business? The US clearinghouse for all automated transfers between banks is built upon COBOL. But if your outlook for the future is, as one 3000 vet said, a staff pool of "no new blood for COBOL, IMAGE, or VPlus," then moving COBOL becomes a solid first step in a migration. Just ensure there's enough capability on the target COBOL to embrace what that 3000 application -- like the one at Cerro Wire -- has been doing for years.
March 12, 2014
Wiring Up the Details for Emulation
For two-plus years, Herb Statham has been inquiring about the Stromasys CHARON HP 3000 emulator. He first stuck his hand up with curiosity before the software was even released. He's in an IT career stop as Project Manager for Cerro Wire LLC, a building wire industry supplier whose roots go back to 1920. Manufacturing headquartered in Hartselle, Alabama, with facilities in Utah, Indiana and Georgia.
Statham is checking out the licensing clearances he'll need to move the company's applications across to this Intel-powered solution. The privatization of Dell turns out to be a factor in his timetable. Dell purchased Quest Software before Dell took itself private. By the start of 2014, Dell was still reorganizing its operations, including license permissions needed for its Bridgeware and Netbase software. Cerro Wire uses both.
I’m after some answers about moving over to a virtual box" Statham says. "I know CHARON's emulating an A500, but that [Intel] box [that would host it] has four processors on it. I’ve heard what I’m going to have to pay, instead of hearing, 'Okay, you’re emulating an A500, with two processors.' They’re looking more at the physical side.”
This spring is a time of change and new growth for legacy software like Netbase, or widespread solutions such as PowerHouse. While the former's got some room to embrace license changes, the latter's also got new ownership. The PowerHouse owners Unicom Systems have been in touch with their customers over the last few months. The end of March will mark the projected wrap-up on Unicom's field research. At Cerro, the Quest software is really the only license that needs to be managed onto CHARON, according to Statham.Cerro Wire's got an A500 now as a result of several decades of 3000 ownership. The company is fortunate enough to have control over its main applications, software based on the DeCarlo, Paternite, & Associates IBS/3000 suite. At the company HQ in Hartselle, Alabama, Statham said turning to the 3000 early meant source code was Cerro's to revamp and extend.
"We have highly customized it, and we’ve written applications around it," Statham says. "When we bought it, source code was part of it. Some of the programs that were written for it now do a lot more than they used to do. Some have been replaced altogether."
The company replicates its data from the Hartselle center to an identical A Series server, including a dedicated VA 7410 RAID array, in Indiana. Netbase was a replication groundbreaker for the 3000 from the late 1980s onward, so it's essential to keeping the MPE/iX applications serving Cerro.
Statham has no pressure from Cerro management to replace the applications that are successful at running the company. With ample spare parts, independent support and storage consulting, and his own source in hand, he needs only the green light from Dell to move forward. Specifics on pricing and performance are still in play from Stromasys, at least from his vantage point. A 1.5 version of CHARON HPA/3000 was announced late last year, promising increased performance. But meeting the speed needs of an A-Class would be no challenge for the CHARON lineup.
This veteran of 3000 deployment and management has little desire to send his company toward an application replacement that might end up with Cerro "spending millions of dollars." There are many years left for MPE/iX, and his company is an all-HP shop, with the exception of a couple of Dell monitors on Statham's desk. He can see a long future for the app the company has fine-tuned to its business.
The CALENDAR intrinsic roadblock is the only thing he can forecast by now. He's not sure how HP might react to an independent fix for that issue, a date challenge that's still 13 years away.
"If we could ever get this 2027 thing out of the way, you could run your applications indefinitely, so long as you’ve got someone to support them," he says. "My only concern is HP themselves, in the event that someone said they had a patch to the operating system — and so you didn’t have to worry about the year, because there was some type of workaround."
But Stromasys became an HP Worldwide Reseller Partner last year, so perhaps even that question could be resolved. What nobody can be sure of, at the moment, is if Dell might want CHARON to be hosted on its server hardware, now that it owns Netbase.
February 26, 2014
Comparing Historic 3000 Horsepower Costs
Over the last few weeks we've checked in with Jeff Kell, the system manager at the University of Tennessee at Chattanooga. The university powered off its last two HP 3000s not long ago, and along the way has mounted dozens of Unix and Linux CPUs and virtual servers to replace that pair of MPE machines. We asked him what he believed the school's IT group had spent on MPE over 37 years -- and limited the question to the capital costs of systems. (Ownership cost is much harder to calculate across four decades.)
Kell, who founded the HP 3000 listserve and newsgroup, as well as chaired the SIGSYSMAN group for Interex over the years, said "We have had comparable expenses with each iteration of the 3000's life-cycle." Across those decades, the university owned Classic HP 3000s based on CISC technology, then early PA-RISC servers -- new enough in that generation to be considered "Spectrum" 3000s -- then later-model PA-RISC units, and finally the ultimate generation of HP 3000 hardware.
"In short, it was an expenditure in the low six figures, once every decade," Kell said.
We ran Series II, then Series IIIs, and the tags were low six-figures in the 1970s. We then got some 950s in the late 1980s (we had some early Series 950 deliveries) at about the same price point. Then the 969 in the 1990s, again about the same. And finally, the A/N-Class during this century.
Comparisons to two points seem worthy. The pricing for the value of high-end 3000 computing remained constant; at the time of the late 1980s, for example, a Series 950 was the most powerful 3000 available. Then there's the comparison to the expenditure of acquiring the hardware to support dozens of servers, virtual and otherwise. The low six figures won't buy much toward the high end of business critical computing gear over a decade, using today's commodity pricing. The newest servers might seem cheaper, but they don't give durable service for 10 years per installation, like the ones at Kell's shop did.It was not all smooth sailing on value for expenditures, Kell added. The A-Class server line was performance-challenged, even though it was rated a bit faster than the previous, K-Class 3000 hardware known as the Series 900 line.
"We had some performance issues with the A500 after we started offering our "online" applications: self-service, and we tried web-based apps, too -- but that was early on and challenged," Kell reported in his 3000 debriefing. Even at that moment in time, there was belief expressed for the ability of HP 3000 hardware to rise to the need, so long as it was more powerful 3000 hardware. Given the performance issues with the A-Class, he explained, "there was some political incentive to address the problem when we got the N-Class, which was a dominating force until the end of our 3000 days. It never blinked."
In short, the longest lifespan for any server still available with a Hewlett-Packard 3000 badge belongs to the N-Class. This is illustrated by the drive to match the horsepower of the top three models in that lineup, an effort which kept Stromasys CHARON engineers well-engaged during 2013.
February 21, 2014
Just how fast is that A-Class, anyway?
By Brian Edminster
Earlier this week, there was a report of an A-Class HP 3000 going wanting on eBay. It was being offered for $2,000 with no takers. The system at hand was an A400-100-110, the genuine bottom of the A-Class line.
While I'd argue that a $2,000 A400 with a transferable MPE/iX licence is a steal, there seems to be a lack of appreciation for the wide variance in speeds in what is considered a A-Class' system.
I believe the system that was being offered as a bare bones A400, as indicated by its system number "A400-100-110." The first character (A) is the class; the next three numbers (400) are the family; the next three are the number of CPUs (100, meaning one); and the last three are the HP rated speed in MHz of the PA-RISC CPU chip. (In this case, it's a PA-8500) This system on eBay also happened to be missing a tape for creating/booting from a CSLT, so if your boot drive failed -- or you needed to make configuration changes that required booting from tape -- you would be out of luck without buying a little more hardware.
This particular A400 system, according to the AICS Relative Performance chart mentioned in the article, runs at a 17. That's about 1.7 times faster (CPU-wise) than the original 917/918 systems. In IO-intensive applications, I have found it felt closer to 2 times faster. I have also worked on an A400-100-150, which CPU speed-wise is a 37. (That system also happens to allow installation of 2GB RAM vs. the 1GB limit on an A400-100-110).
So in short, we can have a greater than 2:1 performance potential between two servers that are both ostensibly A400 A-Class systems. And that's not even taking into account the advantages of multiple CPUs for performance in complex multi-user environments.A400s and A500s have been available in both 1-way and 2-way models, while the N4000s are available in 1-way, 2-way, 3-way, and 4-way configurations. Prior generations of PA-RISC systems could be configured with as many as 4, 6, or even 12 processors. [Ed. note: I recall that several of those higher numbers were available only to HP-UX users.]
Performance benefits aside, multiple CPU systems have been, in my experience, more resilient to CPU failures. This is by virtue of having multiple CPUs. I've had multi-CPU systems where a single CPU failed, and if I had not noticed a minor difference in batch throughput, my online users wouldn't even have noticed. I simply scheduled a service call for the next day -- after warning my users of a previously unplanned service outage, and making sure the backups ran for the night.
It took longer for the hardware to self-test and MPE/iX to reboot than it took the service engineer to replace the bad CPU. Total un-planned down-time was about an hour. Not bad.
It was not quite hot-swap easy, like many modern RAID disk arrays. But that HP 3000 was plenty resilient enough to "Take a licking, and keep on ticking" -- as they once said of Timex watches.
Brian Edminster is the curator of the MPE Open Source repository and website www.MPE-opensource.org, as well as founder of an independent consultantancy.
February 19, 2014
Finding Value in An Exiting MPE Box
A few weeks ago, Jeff Kell of the University of Tennessee at Chattanooga asked around to see if anybody wanted his decommissioned N-Class server. It's way above the power range of the A-Class servers, and even includes some storage options not usually found in a decommissioned 3000.
But the interest hasn't been strong, according to our last update from Kell. He put out his offer -- basically trying to keep the system from becoming more than spare parts, he said -- on the mailing list that he founded two decades ago. We refer that resource as the HP 3000 newsgroup, but it's a LISTSERVE mailing list of about 500 members.
We've heard several reports like this for HP 3000s being turned off, but none of them involved an N-Class system. There's a Series 969 on offer for free -- yes, take it away is all that Roger Perkins of the City of Long Beach asks. While that 969 is more powerful than an A-Class, it's still leagues behind an ultimate-generation N-Class 3000.
This begs the question of what value your community would assign to any used system, regardless of size. Horsetrading on hardware is an IT manager's pastime, when searching for newer for more powerful systems. But it's becoming clear there's a reset going on in the market.
Kell's offer on the newsgroup was straight to the point.
Kell also mentioned his own A-Class onsite, an A500, DATs, two DLTs, a few internal drives, and a dual connect VA fiber channel array. "It has an external SCSI rack that had some issues we never quite resolved; it won't boot today since the mirrored disks have issues). But the VA array was healthy. Assuming the software transfers these days, both these systems have MPE/iX 7.5, Mirror/iX, ToolSet, and TurboStore/7x24."
We have tentative arrangements to have our last two 3000s decommissioned, but was curious if there was any interest in the hardware/systems. Hate to sound like a sales pitch, but we're basically happy with shipping, plus a certification the drives are wiped.
We have an HP 3000-N4000 4-way, DATs, 2 DLTs, a few internal drives, and a VA fiber channel array (dual connect). It's perfectly fine.
That's a very suitable datacenter keystone to build a homesteading practice around. In fact, that's what the university had in mind when it bought those servers.
We tried our first "migration" in 1997 off the 3000 to Banner software, which was a gigantic Oracle monster, one that the UT system had essentially licensed for all campuses. But compared to our legacy application's customizations -- we did just about anything we were asked -- Banner was too restrictive. There was a revolt, and we ended up only implementing Financial Aid and student Account Receivables. So knowing that we had to stick on the 3000, we got that N-Class as our "homestead" machine. The A-Class was just a warm standby. We ran periodic snapshot backups and popped them over to the A-Class for restore, and did a full sync on weekends.
We ran that way for another decade, when we had Round 2 of the Banner conversion. We had roughly four generations worth of HP 3000s, maybe even actually five. After our delays for the Series 950 we purchased, HP provided us with a temporary Series 52/58 (development/production) systems to tide us over until the delivery -- our Series IIIs were beyond maxed out.
February 07, 2014
Code-cutter Comparing Solutions for 3000s
When a 3000 utility goes dark — because its creator has dropped MPE/iX operations, or the trail to the support business for the tool has grown faint — the 3000 community can serve up alternatives quickly. A mature operating system and experienced users offer options that are hard to beat.
One such example was Aldon Computing's SCOMPARE development tool, once a staple for 3000-based developers. It compared source files for more than 15 years in the HP 3000 world. Eventually Aldon left the MPE business. But there are a fistful of alternatives. Allegro Consultants offers a free MPE/iX solution in SCOM, located at
At that Web page, scroll down to SCOM. Other candidates included a compare UDC from Robelle, GNU Diff, diff in the HP 3000's Posix environment, and more. If you're willing to go off the MPE reservation -- and a lot of developers work on PCs by now -- there's even a free plug-in for Notepad++, that freeware source code editor which relaces Notepad in Windows. You can download that plug-in as an open source tool at SourceForge.net
When the subject first surfaced, Bruce Collins of Softvoyage offered details on using diff in the HP 3000's Posix.
run diff.hpbin.sys;info="FILE1 FILE2"
The file names use HFS syntax so they should be entered in upper case. If the files aren't in the current account or group, they should be entered as /ACCOUNT/GROUP/FILE
Donna Hofmeister offered a tip on using Robelle's compare UDC:
Regarding Robelle's compare. Being a scripting advocate, I strongly recommend adapting their UDC into a script.... and take a few seconds to add a wee bit of help text to the script, to make life more enjoyable for all (which is the reason for scripting, yes?)
Other environments that might be operating in the 3000 datacenter provide alternatives. Former HP engineer Lars Appel brought up a Linux option in the KDE development environment:
If using KDE, you might also find Kompare handy...
http://en.wikipedia.org/wiki/Kompare (see screenshot)
On MPE, as others mentioned, there is still the Posix diff in two flavours: the HP-supplied in /bin and the GNU version that lives in /usr/local/bin. The former allows two output formats (diff and diff -c); the latter also allows “diff -u”.
Oh, regarding /bin/diff on MPE... I sometimes got “strange” errors (like “file too big”) from it when trying to compare MPE record oriented files. A workaround was to use tobyte (with -at options) to created bytestream files for diff’ing.
Appel has noted the problem of comparing numbered files, like COBOL source files, when one or both files have been renumbered.
With Posix tools, one might use cut(1) with -c option to “peel off” the line number columns before using diff(1) for comparing the “meat”. Something in the line of ... /bin/cut -c7-72 SourceFile1 > BodyText1.
February 04, 2014
Making Domain Magic, at an Efficient Cost
Five years ago, HP cancelled work on the DNS domain name services for MPE/iX. Not a lot of people were relying on the 3000 to be handling their Internet hosting, but the HP decision to leave people on their own for domain management sealed the deal. If ever there was something to be migrated, it was DNS.
But configuring DNS software on a host is just one part of the Internet tasks that a 3000-savvy manager has had to pick up. One of the most veteran of MPE software creators, Steve Cooper of Allegro, had to work out a fresh strategy to get domains assigned for his company, he reports.
We have been using Zerigo as our DNS hosting service for a number of years now, quite happily. For the 31 domains that we care for, they have been charging us $39 per year, and our current year has been pre-paid through 2014-08-07.
We received an e-mail explaining exciting news about how their service will soon be better-than-ever. And, how there will be a slight increase in costs, as a result. Instead of $39 per year, they will now charge $63 per month. A mere 1900% increase! And, they won't honor our existing contract either. They will take the pro-rated value of our contract on January 31, and apply that towards their new rates. (I don't even think that's legal.)
In any case, we are clearly in the market for a new DNS Hosting provider. Although I am not a fan of GoDaddy, their website. or their commercials, they appear to offer a premium DNS Hosting service, with DNSSEC, unlimited domains, etc. for just $2.99 per month. Sounds too good to be true.
Cooper was searching for experience with that particular GoDaddy service. GoDaddy has been a default up to now, but acquiring a domain seems to need more tech savvy from support. The 3000 community was glad to help this other kind of migration, one to an infrastructure that MPE never demanded. The solution turned out to be one from the Southern Hemisphere, from a company whose hub is in a country which HP 3000 experts Jeanette and Ken Nutsford call home.Cooper said that some 3000 vets suggested "rolling my own," self-hosting with his external DNS. Here's a few paragraphs addressing those two topics:
We have a dual-zoned DNS server inside our firewall, but we do not have it opened to the the outside world. Instead, only our DNS hosting service has access to it. The DNS hosting service sees itself as a Slave server and our internal server as the Master server. However, our registrars point to that external DNS hosting service, not our internal server, so the world only interrogates our DNS hosting service when they need to resolve an address in one of our 31 domains.
Why don't we open it up to the world? Well, we get between 200,000 and 3,000,000 DNS lookups per month. I don't want that traffic on our internal network. There are also DDoS attacks and other exploits that I want no part of. And, since some of our servers are now in the Cloud, such as our mail, webserver, and iAdmin server, I don't want to appear to disappear, if our internet connection is down. Best to offload all of that, to a company prepared to handle that.
When I need to make a change, I do it on our internal DNS server, and within a few seconds, those changes have propagated to our DNS hosting service, without the need for any special action. The best of both worlds.
Now, on to the issue from earlier in the month. Our DNS hosting service, Zerigo, announced that they were raising rates by 1900%. And, our first attempt at a replacement was GoDaddy. Although the information pages at GoDaddy sounded promising, they made us pay before we could do any testing. After three days of trying to get it to work, and several lengthy calls to GoDaddy support, they finally agreed that their service is broken, and they can't do what they advertised, and refunded our money.
The biggest problem at GoDaddy is that I (as the customer) was only allowed to talk to Customer Service. They in turn, could talk to the lab people who could understand my questions and problems. But the lab folks were not allowed to talk to me, only the Customer Service people. This is not a way to do support, as those of us in the support business know full well.
After more research, I hit upon what appears to be a gem of a company: Zonomi. They are a New Zealand based company with DNS servers in New York, Texas, New Zealand, and the UK. And, they let you set up everything and run with it for a month before you have to pay them anything. We were completely switched over with about an hour of effort.
Now, the best news: they are even cheaper than our old DNS hosting service used to be. If you have a single, simple domain, then they will host you for free, forever. If you have a more complex setup, as we do, the cost is roughly US $1 per year, which beats the $63 per month Zerigo wanted to charge. The first ten domains cost $10 per year, then you add units of five more domains for $5 per year.
The only risk I can see is if they go out of business. In that case, I could just open our firewall and point our domains to our internal server, until I could find a replacement. So, that seems reasonable.
That problem is solved. On to the next fire.
January 29, 2014
University learns to live off of the MPE grid
One of the most forward-looking pioneers of the HP 3000 community shut off its servers last month, ending a 37-year run of service. The University of Tennessee at Chattanooga IT staff, including its networking maven Jeff Kell, has switched over fully to Linux-based computing and an off the shelf application.
UTC, as Kell and his crew calls the school, has beefed up its server count by a factor of more than 10:1 as a byproduct of its transition. This kind of sea change is not unusual for a migration to Unix and Oracle solutions. HP 3000s tend to be single-server installations, or multiples in very large configurations. But to get to a count of 43 servers, IT architecture has to rethink the idea of a server (sometimes just a blade in an enclosure) and often limits the server to exclusive tasks.
After decades of custom-crafted applications, UTC is running fully "on Banner, which has been SunGuard in the past," Kell said. "I believe it's now called now Ellucian. They keep getting bought out." But despite the changes, the new applications are getting the same jobs done that the HP 3000s performed since the 1970s.
The heyday of the HP 3000 lasted until about 2009 or so, when UTC got all of the Banner applications up and running, Kell reports. Banner -- well, Ellucian -- has many modules. Like a lot of migrating sites that have chosen replacement software off the shelf, the transition was a stepwise affair.
It's Linux / Oracle replacing it. The configuration was originally Dell servers (a lot of them), but most of it is virtualized on ESXi/vCenter, fed by a large EMC SAN. They got some server hardware refreshed recently, and got Cisco UCS blade servers. I'm sure they're well into seven figures on the replacement hardware and software alone. I've lost count of how many people they have on staff for the care and feeding of it all. It's way more than our old 3000 crew, which was basically six people.
The 3000 was still pretty heavily used until 3-4 years ago, when they got all of Banner up and running. The 3000 continued to do some batch transfers, and our Identity Management. The 3000 was the "authoritative" source of demographics and user accounts, but they are now using Novell's Identity management -- which bridges Banner/Oracle with Active Directory, faculty/staff in Exchange, and the student accounts in Google Mail.
We had dedicated 3000s in the past for Academics. They later jumped on an IBM system, then Solaris for a time, and now Linux. We also had one for our Library catalog and circulation (running VTLS software), but they later jumped on the Oracle bandwagon. More recently, the whole module for the library has been outsourced to a cloud service.
Even in the days when the 3000 ruled at UTC, there were steps in a transition. "I think we peaked at seven 3000s briefly while we were in transition -- the days we were moving from our old Classic HP 3000 hardware to the then-new Series 950 RISC systems," Kell said. "After the delays in the 3000 RISC system deliveries and the promises HP had made, they loaned us a Series 52 and a 58, so we could keep pace with production while waiting on PA-RISC."
January 16, 2014
Replacing parts a part of the 3000 lifestyle
We'd like to hear from the community about 3000 parts: the ones that will push them away from MPE, as well as the parts that will keep decade-old servers running. Check in with me at firstname.lastname@example.org.
Customers who continue to rely on HP 3000s place great store on parts. Spare parts, the kind that tend to wear out sooner than others like disk drives, or the ones which can force a company into disaster recovery like a CPU board. The veterans in the community know that there's no support without a source of parts. And the demise of 3000 installations, like a well-run junkyard, can be a source.
However, a dearth of spare parts forced one 3000 customer into entering the world of HP 3000 emulation. Warren Dawson had systems that were aging and no clear way to replace what might fail inside them. Dawson's in Australia, a more remote sector of the 3000 empire. But his need became the spark that moved HP's iron out and replaced it with Intel-based hardware. Commodity became the follow-on costume that Dawson's information now plays in.
While there are portions of the HP 3000's high-failure parts list that can be replaced with third party components -- drives come to mind -- a lot of the 3000's body is unique to Hewlett-Packard's manufacture. Another company in Mexico, a manufacturing site, moved its applications off MPE because it figured that replacing 15-year-old servers was a dicey proposition at best.
This leads us to our latest report of HP 3000 parts, coming from a switched-off site in California. Roger Perkins has a Series 969 that he's working to give away. Like everybody who paid more than $50,000 for a 3000, he'd like to believe that it has value remaining. But on the reseller market, he might be fortunate to get a broker to haul it off.
Those who do are likely to take the system for its parts. What's more, the HP 3000s that are going offline are not the only resource for replacement parts. Other HP servers can supply this market, too. Finding these parts is the skill that homesteading managers must master.One of our bedrock sponsors, Pivital Solutions, makes a point of ensuring that every support customer has a depot-based spare parts source. Whatever you'd need to get back online, they've got on hand. Not something to be hunted down, ordered and then shipped in a few days. Steve Suraci of Pivital asked questions back in 2012 that still need answers, if homesteading's risk is to be fully considered. And sometimes the parts aren't even inside a 3000. They just have a MPE version that's got to be hunted down, if a support provider doesn't depot-stock parts. Hostess Brands had a Series 969 back then, one which needed an fiber router.
How many HP 3000 shops are relying on support providers that are incompetent and/or inept? The provider was willing to take this company's money, without even being able to provide reasonable assurance that they had replacement parts in a depot somewhere in the event of failure. There are still reputable support providers out there. Your provider should not be afraid to answer tough questions about their ability to deliver on an SLA.
The easy questions to answer for a client are "Can you supply me support 24x7?" or "What references will you give me from your customers?" Harder questions are "Where do you get your answers from for MPE questions?" Or even, "Do you have support experts in the 3000 who can be at my site in less than a day?"
But Suraci was posing one of the harder questions. "Here are my hardware devices: do you have spares in stock you're setting aside for my account?" Hardware doesn't break down much in the 3000 world. But a fiber router is not a 3000-specific HP part. Hewlett-Packard got out of the support business for 3000s for lots of reasons, but one constant reason was that 3000-related spare parts got scarce in the HP supply chain.
There are other support companies that guarantee parts availability. But many sources of support services to keep 3000s online wait to acquire customer parts as needed. Some of them pull components like power supplies out of the plentiful HP-UX servers from the early decade of this century. HP called those boxes K-Classes, servers that were both Series 800s as well as Series 900s.
A Series 969 server like the one turned off by Perkins and pushed to the curb serves a need for homesteaders. A reseller first has to take it out of a datacenter, clean up and test what's inside the cabinet, then do triage on what's worth keeping in the 3000 food chain. Not many places have enough storage to run the equivalent of an auto salvage yard. You know, the kind of place where a steering wheel bearing you need is deep inside a junked Dodge Dart.
Depending on the model of HP 3000, many have value in their spare parts. An owner who's getting rid of a 3000 shouldn't expect much compensation for a system they're selling off for parts. But the operators in the 3000 community who are both selling used systems as well as supporting these servers need a supply of components. How much they need depends on the limitations of available warehouse space.
Governments are beginning to insist on responsible recycling. Purchasing a computer in California now includes a recycling fee built into the sale at retail and consumer spots like Best Buy. But Goodwill Industries' Reconnect takes on many computers, regardless of their working status.
There's a lot to consider when keeping an HP 3000 running as a mission-critical component. MB Foster summed up the elements well in a Sustainability Plan document you can download from their website. Way back in 2010, Foster asked some good questions that a Sustainability Plan should answer.
Okay, so let’s look at the impact of a crash on Friday afternoon when the HP 3000 was backed up last Saturday (you do verify your backup tapes, right?) You have a full backup from last Saturday and daily backups from Monday through Thursday. The spare parts are not on site, and you have to contact your provider to get the parts and a skilled technician to the site, and then you can start restoring your hardware and application environments. How long will it take to restore all the data, applications and the whole system?
Way, way back in 2005 -- yes, more than eight years ago -- one of the bigger sources of HP hardware said the savvy customers were arranging for their own inventory of spare parts. Genisys' Robert Gordon said that customers who know the 3000 have their own spare parts options to rely upon.
"They're either going to go to third party maintenance, or they're going to self-maintain," Gordon said. "I think a lot of people are technically savvy on the 3000; they know it's not rocket science, and they're going to buy spare board kits. So we're going to see that business pick up. We'll see a lot 3000 sales in the year 2006." There are fewer 3000s to sell in the marketplace today. But that doesn't matter as much as locating the ones which are still around.
January 09, 2014
Eloquence: Making a Bunny Run Elsewhere
An email poll over the last week asked 3000 owners and their suppliers what was in store for their systems this month. One reader in Long Beach, Roger Perkins, has a 3000 they've shut down at the City of Long Beach and wants to find "somebody who's interested in taking that out for us. I don't know if it's worth any money, but I was hoping we wouldn't have to pay anyone to take it out." Perkins left his number for a recommendation on recycling a 3000: 562-570-6054.
Our experience with this situation is that individuals -- fellow 3000 owners -- will be interested in the machine for parts, provided they don't have to bear too much freight costs. But there's something more unique than a collection of slower CPU boards and decade-plus-old discs on hand. The city has an MPE/iX license attached to its 3000. It's a system element that's not being sold any more, and essential to getting a virtualized 3000 online.
But little will change in that sort of transition transaction, except the location of a boot drive. In contrast, at Genisys Total Solutions, Bill Miller checked in to report that a change in databases has extended the reach of the application software for financials that has been sold by Genisys since the 1970s.
Though we have migrated all of our software to a Windows platform running Eloquence, we still have an HP 3000 that has been in operation for close to 13 years and has not failed at all during that time. We still support a handful of HP 3000 clients, who also seem to think the HP 3000 is the Energizer Bunny and see no reason to move from it.
Our main business is selling and supporting our applications on the PC platform. We have found Eloquence (as is IMAGE) to be a reliable and easy to maintain database.
December 30, 2013
2013 emboldened 3000 changes for both migration and homesteading practices
As a service to readers who crave summary and broad strokes, we hearby sketch what the year 2013 meant to the 3000 community. It's too much of a cliche to say that the previous 12 months were driven by change. That's been an essential element for the community since 2001. But a dozen years has now spread changes onto the migrating community member, as well as those who have made their mission one to homestead.
The HP 3000 CHARON emulator from Stromasys showed more promise this year, but some of its impact lay in the way it held migrations in check without even being deployed. Another factor came from the economy. By year's end the markets were flying at an all-time high, but the recovery has its blind spots, according to some 3000 users. Couple the proposed savings in keeping MPE apps virtual with with an uncertain future for HP's replacement solutions, and the movement away from the 3000 slowed.
Even with that evidence, some shutdowns of systems stood out. A major installation of 3000s that had been serving the airline industry saw their work moved to .NET replacements, as Open Skies became New Skies. We also saw Hewlett-Packard closing down its own internal HP 3000 operations at long last, powering off the final four systems, just 12 years after advising its customers to do the same.
The year also offered a chance to see what remained on the field a decade after the community marked the World Wide Wake of 2003. The server got its first iPad app when a terminal emulator emerged for iOS, even as other experts found other ways to get an MPE console onto a tablet. And the exit of expertise continued throughout our 3000 world, even as some stalwart resources remained online.HP set the pieces in place long ago for its 3000 strategy to evolve away from the need for physical hardware. The Apps on Tap strategy that led to the Open Skies offering -- where networked 3000s serve up apps to customers who don't have servers onsite -- is now being echoed in Software as a Service.
Sites that moved off HP 3000 installations for ecommerce software watched their vendor get acquired, then see the open version of their software slip into a 140-product lineup. It was an example of how migrations became a part of life at those 3000 sites that had already left MPE behind. Even among the sites where server migration hasn't occurred, data migration is already afoot. Customers are now looking at a migration off of Windows XP for their users, and some are facing the same reluctance and lack of budget they saw for 3000 diaspora.
Hewlett-Packard had its share of problems to overcome, from shuffling the pathways to MPE documentation online to keeping its enterprise mission critical business from evaporating. Each of the four quarters of revenues for its BCS group posted a 20 percent sales decline from the previous year's numbers. It was a continuation of a 2012 trend. The company's CEO and CFO called the Unix server business a formerly growing venture. Then there was the announcement of curtailing another HP business OS, OpenVMS, starting in 2015 when new Integrity systems won't run on the environment. Things got so critical for BCS and its bretheren that HP reorganized the whole enterprise server operation into a single unit, then removed its executive VP from the job.
Emulator news emerged from two fronts. Stromasys built out its management for the CHARON product and opened the doors on its North American rollout with a May Training Day event. The latter was the first 3000-specific event in almost two years. In the snows of February in Europe, a similar event for CHARON recalled HP's final organized event for the 3000, nine years earlier. Early in the fall, a group of freeware developers was trying to create a not-for-commerce version of what it called a simulator of HP 3000 hardware. Successful booting remained elusive.
In the meantime, the offering of an emulator had customers checking HP's rules and processes for license transfers, some three years after the company shut down all other 3000 operations. It helped to be able to ask for the right process, and ask the right person.
Another trend emerged in the longevity of the 3000 expert. Outlasting the 3000 server was a duel that some experts were giving up. One company in LA made a shift to Windows because its IT staff for the 3000 was aged 67 and 72. But among those who continued to keep the MPE lamp lit, techniques to continue 3000 operations still emerged. Replacing HP's disks with third party alternatives got detailed to swap in fresh hardware for decade-old drives. Moving store to disk files with attributes intact is possible with newer open source archiving software.
The year showed that change itself has changed for the community. The long run of the HP 3000 unreels into the dark of the as-yet-unlit future. There was even a careful examination of the costs of remaining on the 3000 for 5-10 years.
December 13, 2013
Euro 3000 allies find a foothold in meeting
Yesterday we made reference to a 2001 Q&A interview with Stan Sieler, the Allegro co-founder interviewed in our latest print issue. Across the top of that page is a 2001 web ad for an entity called Millware Corporation. It was a company whose Dave Wiseman was pushing out a web-enabled dashboard, based inside a free terminal emulator. ScreenJet's Alan Yeo was one of the Millware partners, too. And Yeo has remained a vital force in meetings in 2007, 2009, and the HP3000 Reunion of 2011.
The truth about the HP 3000 community is that remains connected. Yours has always been a social group, long before there was such a concept as social networking via Twitter, Facebook, and the others. Last week the old-style networking was afoot, thanks for Millware's old founder taking a first step.
Dave Wiseman sent the word to more than 50 HP 3000 community members in Europe to gather on December 5, and despite serious storms about Europe on meeting day, he got 10 to make the trek to London. He reports:
I’ve included my invitation to as many of our overseas friends as I can, so that if they are even thinking of coming to Europe, this might form a focus time to come over.
A huge thank you to all who made the effort to get to London last week to meet. It was great to see all our old friends and everyone clearly enjoyed the meeting. Amazing to see that apart from going grey or bad, most of us were still recognisable. As far as I am aware, everyone made it safely home, although I had to stay in London, as all trains were cancelled due to storm damage on the line.
Despite the storms in Germany and what ended up as relatively short notice, we still had around 10 of us from as far afield as Berlin, Lyons, Wurzburg, Sheffield and various other places around the UK. With a large sprinkling of beer and a few bottles of wine after, we revived many fond memories of conferences past. Alas, none of us took any photos -- which shows that we’re just not the modern generation who would have all this posted on Twitter before they’d eaten!
Our thanks must also go out to Ian Kilpatrick who generously paid for the meal and the drinks so please visit his website at WickHill.
We all resolved to have another meeting in the not too distant future, and so I would ask you all to answer the following questions for me and I’ll happily organise another meeting.
Again, if you can think of anyone who might be interested in attending, let me have their details for future mailings. Especially I do not have many contacts form the other European vendors --- so if you are in touch with the vendors in your country, you know what to do.
1. Would you like to come if you can?
3. London/Outer London hotel/Amsterdam/somewhere in the sun? Suggestions?
4. Preferred day of the week?
I would probably try to organise the next one in a location that has meeting rooms, as well as a bar
December 11, 2013
In a slowing market, things can shift quickly
Our November printed edition of The 3000 Newswire carried a headline about the success that the Stromasys CHARON emulator is experiencing in your community. However, one of the green lights we noted in that article turned red during the time between writing and delivery into postal mailboxes.
Ray Legault has checked in to report that the project to virtualize HP 3000s closing down in a soon-to-be-closed disaster recovery site has been called off. The close-out doesn't appear to reflect any shortfall in the value of the CHARON element. But carrying forward applications has proved to be complicated.
In particular, the costs for license upgrades of third-party software came in for special mention. This isn't standalone application software, like an Ecometry or MANMAN or even an Amisys. That sort of app isn't in wide use in 3000 customer sites, and to be honest, off the shelf solutions never were. The software license that needed a transfer wasn't from HP, either. MPE/iX has a ready, $432 transfer fee to move it to an industry-standard Intel system. No, this well-known development and reporting tool was going to cost more than $100,000 to move to a virtualized HP 3000.
"Our project was cancelled due to other reasons not related to the emulator," Legault said. "Maybe next year things will change. The apps not having a clear migration path seemed to be the issue."So file this blog report under a correction, or perhaps an update. But I wouldn't want to be the bearer of incorrect news, and there's something virtuous about filing a correction, anyway.
Where we made our mistake was in observing activity in license transfer inquiries, then getting information about a pending CHARON purchase, but not seeing a confirmed PO. That's a document we rarely see here at the Newswire. These days, it's a rare thing for anyone to share a number as specific as, say, $110,000 for a license fee.
For a company which has eight remaining applications to push into the year 2018, and needs to keep those apps hotsite-recoverable, an emulated 3000 with low hardware costs makes good sense. But there are license budgets to resolve in order to proceed with this sort of transition activity.
Most important, we haven't heard any reports yet of any vendor who flat-out refuses to license their MPE/iX software for CHARON. Perhaps when a vendor has to watch a $110,000 sale disappear, it could spark a different approach to the business proposition of serving a slowing marketplace. Maybe next year, things will change.
In the news business when there's a mistake, we were always taught to close the correction with "We regret the error." In this case, everybody regrets the shift in strategy.
November 25, 2013
Calculating the 3000's Time to Purchase
On an informal call with a 3000 vendor today, he delivered some sound advice about purchasing. "In the end, it's really about how they buy -- not how you sell." This makes a difference to everyone at this time of year, when fiscal year-end closing is less than six weeks away.
Sometimes a buyer of IT products or a service will want to make a purchase, but then the learning curve gets twisted. The manager might have an outdated estimate of how long it takes to get something into a status for a PO. This can be especially true for an HP 3000. Even when the system is on the path away from mission-critical, en route to migration, buying something related to a 3000 can be a distant memory.
This is not to be confused with renewing support contracts. Those are renewals, not outright new purchases. The time needed to get to a PO can include the processing time at related vendors, in some cases. For example, there's the licensing which is part of making a transition to the only emulator for HP 3000s. Software suppliers, or HP, require time to approve transfers. You only learn how much time your organization, or your vendor, needs by purchasing something. Or attempting to, near the end of your fiscal year.At Dairylea Cooperative, transferring the 3000's MPE/iX license to an emulator took almost a week, Jeff Elmer reports. The HP employee Erick learned about the process from Stromasys, the maker of the emulator.
Once Erick was on board, it was just a matter of signing another document and processing a credit card purchase of $432. It took 3 days from the point when they said it could be done to when I had the appropriate documents via e-mail. (It took 3 days to convince them to do it, so the process overall was 6 days. I hope convincing them to do it is no longer necessary.)
Over at Boeing, the internal workings of the purchasing process for emulator hosting hardware will be tested. "I think we are too late in the year to get the hardware," said the 3000 manager there.
One vendor said they figured on 16 months to get to the PO point for their product. Another said their mission to close a purchase was nearly complete -- but the customer's legal counsel still had to weigh in on the deal. It's a good idea to review the timeline for purchasing if you're just getting back to supplying your 3000 with something, even if it's just assessments for transition or sustaining services.
That's especially true if you're left with money in your budget you'll need to spend, or lose it for next year. Only a buyer can put a Rush on an IT purchase. We're in the Season of the Rush now, the same part of the calendar when HP announced its 3000 exit. It was no surprise when nothing related to migration purchasing happened during the following year.
November 18, 2013
MANMAN and a 3000 in new Ohio action
Just when you thought the HP 3000 and MPE were done with new installations, along comes a manufacturer to put another system online.
If you break it down, this kind of event needs a few elements to succeed today.
1. A license structure for software (apps and utilities) that is low-budget. Extending third party licenses, for example, rather than buying new ones.
2. In-house expertise to manage and maintain a new system -- or if not in-house, then in-organization
3. A requirement for inexpensive HP hardware for the install. Because if you're going to put something online that has an HP badge on it today, you'll want component redundancy. Think spare CPUs and CPU boards.
The 3000 install was mentioned during last week's CAMUS manufacturing RUG conference call. Measurement Specialties has been a MANMAN manufacturing app and 3000 supporter for so long that ERP Director Terry Simpkins was even used by HP to testify about the integrated 3000 solution. In print. In an ad. Remember print ads for computer systems? HP even bought a few in the 1990s.
Simpkins wasn't at his usual spot during the CAMUS call because he was in Ohio, we were told, working on another outpost in the MSI network. There's more than a dozen worldwide, with many outside of North America. There were years when Simpkins was in China for weeks on end.Some MANMAN customers have a clear path to put as much application up as they like. These forward thinkers got a source code license from ASK Software when such a thing was available from the MANMAN creators. Computer Associates and the succeeding MANMAN owners cut off the source purchasing.
A 3000 owner who maintains their own applications, written in house, is in a similar situation to a site installing MANMAN from source code. In fact, they have a lot in common. One of the reports from users on that RUG call was that most of the efficient operations in MANMAN come from mods. No, not the rockers from the Sixties in Britain. It's short for modifications, of course, custom programming either built in-house or bought from a consulting and support house.
Back in 2011, the MSI IT Director Bob Andreini had a staff of 32 to help him manage operations. Simpkins was responsible for MSI's ERP implementation and support, with a primary focus on MANMAN.
Simpkins has been asked in the past why new operations in MSI go online running MANMAN, sometimes resulting in a 3000 coming online. His answer: "We are using HP 3000 systems for general ledger, accounts payable, inventory control, purchasing, production scheduling, order entry, and invoicing." Way back in 2008 there were 11 locations around the world, "and we have a substantial investment in its continued operation."
Measurement Specialties has been a self-maintainer of its 3000 hardware for more than a decade. They've done their own independent support. Simpkins been a clear speaker along the lines of Teddy Roosevelt for as long as I've known him. 13 years ago HP was trying to assert that IT managers were not looking at platforms anymore when they deployed apps, just the software. Here's the exchange we had in a Q&A for the Newswire.
HP likes to tell us in the press that IT managers at your level don’t make deployment decisions around platforms anymore, that applications are the only thing that matters. What do you believe?
I think that’s bullshit. If I’m looking for an application and I find two that run equally well, and one of them runs on a platform I already have expertise with, I’m all over that one. I don’t believe that we’re all in a heterogeneous environment, or that we want to be in one. I’m not afraid of a heterogeneous environment, but why do I want to add complexity to my life if I don’t have to?
November 07, 2013
Staying on Schedule in a Move to Windows
Yesterday we reported on an airline service provider who's made the move from HP 3000s to Windows .NET systems and architecture. While there's a great advantage in development environment in such a transition -- nothing could be easier to hire than experts in Visual Studio, nee Visual Basic -- companies such as Navitaire have to arrange a new schedule. To be precise, the job handling features of MPE/iX must be replaced, and Windows won't begin to match the 3000's strengths.
Enter a third party solution, or independent software as we like to call it here in the 21st Century. In 2010 MB Foster built a scheduler for Windows sites, and yesterday we heard a customer from the Windows world size up the MBF Scheduler tool. This was an IT shop where a HP 3000 has never booted up. But NaturMed, a supplier of supplements and health education, is a user of the JDA Direct Commerce (formerly Ecometry-Escalate Retail) software on its Windows servers. The company's never seen an MPE colon prompt, but it needs that level of functionality to manage its jobs.
"We've helped Ecometry with the move of many customers off the 3000 and onto Windows," said CEO Birket Foster. "If senior management has simply decided that Windows was the place to be, we could help automate the business processes -- by managing batch jobs in the regular day and month-end close, as well as handling Ecometry jobs and SQL Server jobs." Automating jobs makes a Windows IT shop manager more productive, like creating another set of hands to help team members out. For a 3000 shop making a transition, something like an independent job handler means they'll be able to stay on schedule with productivity.Companies that use Windows eventually discover how manual their job scheduling process becomes while hemmed in with native tools for the environment. Credit card batches must be turned in multiple times a day at online retailers, for example. The site that sparked the MBF-Scheduler design didn't have a 3000's tools, either. It just had 14,000 jobs a day running.
"It seems like this is extremely powerful," one Windows shop said of the product after looking it over, "and we could benefit from this."
Job listings, also known as standard lists (STDLISTs), are common to both the 3000 and Windows environment, and the software was built to provide the best of both 3000 and Windows worlds, Foster said. The software's got its own STDLIST reviewer, one that's integrated with a scripting language called MBF-UDAX. Ecometry sites working on HP 3000s usually rely on a tool as advanced as Robelle's Suprtool for job scheduling.
Foster's Scheduler includes filtering buttons in job reports by user, by job name, by status and by subqueue. A recent addition to the product introduced a custom category that managers can use to select or sort jobs. While running thousands of batch jobs a day, some are in distinct categories. Customers like the idea of managing factory floor jobs separately from finance jobs, for example. Managers
Measurement Systems, the manufacturer which runs a dozen HP 3000s in sites across North America, China and Europe, uses the MBF Scheduler. The product manages a complementary farm of MBF Scheduler Windows servers to move jobs among servers throughout Measurement Systems' 3000s. Terry Simpkins there has been devoted to Infor's MANMAN implementations well beyond the vendor's ability to support the application. Like other customers around the community, Simpkins and his team have compared the Scheduler to MPE's mature tools, and favorably. Sites like this don't need a separate Unix or Linux server for job scheduling, which is the usual way to keep Windows IT on schedule.
Windows schedulers serve HP 3000s, but also server Windows-only IT environments where some MPE/iX operations will be headed. At Measurement Specialities, for example, the IT pro who handles scheduling never sees the HP 3000. But enterprise server-born concepts such as job fences are tools which are at his command.
November 06, 2013
Open Skies flies to a .NET transition
Mark Ranft has been reporting on choices being made by his Pro 3k consultancy to move airline transaction processor Navitaire off a farm of 35 HP 3000s, carefully and with precision. The application -- which began its life as IMAGE-MPE software in the 1990s -- has become New Skies, a shift from its Open Skies roots. Windows .NET is the platform of the future.
What remains of the 3000 farm is going up for sale, he noted in a posting at the HP 3000 Community of LinkedIn. Asked why Windows and its .NET architecture is a suitable replacement for the MPE/iX operations that served major airlines, Ranft said that Windows, like MPE or Linux or HP-UX, is just a tool.
"The enterprise architect must understand the strengths and the weaknesses of the platform and design the application around them, Ranft told us when the migration was underway, some five years ago. "Sometimes this may mean you have large pools of mid-tier systems/application servers to make up for the lack of resiliency in the operating system. This could be compared to using the RAID concept for disk arrays. However, I fear that most enterprises will find the licenses, care and feeding of the numerous mid-term systems needed is far from being inexpensive. Keep in mind that MPE was never exactly cheap."
.NET has been popular for years, a way to apply the Windows environment with more complete application architecture for enterprises. But some of the latest advice about .NET seems to factor in the slowing speed of the Microsoft juggernaut. One writer has even called .NET a failed Microsoft business line, but IT managers who use the product say it's a good choice for Windows implementations.
Ranft has reported that the enterprise once known as Open Skies ran more than a dozen of the largest HP 3000s that Hewlett-Packard ever sold. Five years ago he said
We have 21 HP 3000s. Eighteen of them are the largest, fully-loaded N4000-4-750 systems you can get. We have migrations to Windows in various stages, but there is also a very real need for legacy data access after the migration. The alternative is to migrate all the data and all the archival history, and that can be costly.
.NET has been on the radar screens for 3000 migration since at least 2004. Back when Managed Business Systems was one of the four HP Platinum Migration partners, Rich Trapp said at an HP World presentation the environment may be involved if an organization chooses:
To port their applications, since some porting tools convert the existing applications into .NET. Tools from Unicon specialized in .NET while they were being used by 3000 sites doing a migration. Once inside the .NET framework, further enhancements may involve .NET development.
For the limited number of companies that choose to re-build their applications, they may be re-written or re- engineered in a .NET development environment.
To replace 3000 apps with off-the-shelf packages, which can mean adopting an implementation in a .NET development environment. After replacement, customizations and interfaces may best be written in .NET.
The development for all of these choices takes place in Microsoft's Visual Studio IDE. Like many transition choices from standard 3000 tools -- VPlus, COBOL II, 4GLs, DEBUG and the like -- stepping into Visual Studio means a serious increase in power, as well as a learning curve if a 3000 pro hasn't developed VB skills yet.
.NET adoption means that a staff will need to be up to speed on Visual Basic .NET, C# and SQL Server -- or another .NET-compatible database. There's also a need to get a scheduler working in the Windows world. Fortunately, a 3000-like scheduler has been available for Windows since 2010, from MB Foster. MBF-Scheduler has gained advanced reporting tools, explicit and fine-grained filters, and the same robust functionality as the MPE/iX job handling tools.
The primary difference between development in an HP 3000 environment and a .NET environment is the HP 3000 is geared toward procedural design, while .NET is geared toward object oriented design. Procedural design establishes procedures or steps as a sequence of commands, acting on data structures. With object oriented design, developers model real-world situations and business scenarios as objects that perform actions, have properties, and trigger events.
Even nine years ago, when .NET was much less entrenched, Trapp said that ".NET provides efficient development, well-structured applications; a large number of interfacing techniques and interfaces; and a large quantity of existing, re-usable source code."
October 30, 2013
Marking Moments on Wake Anniversary Eve
In about six hours or so, the HP 3000 community might pause to commemorate one of its last collective acts. Ten years ago the World Wide Wake, organized by event ringleader Alan Yeo, invited members in dozens of locations throughout the world to lift a glass and salute the end of HP's manufacturing of the HP 3000 computer. MPE/iX would be recrafted and revised for another five years, but Oct. 31, 2003 was the last day customers could order a new HP-badged 3000.
At the time we invited a director of the Interex User Group, Denys Beauchemin, to offer a confirmation about the success of the system and record the aftermath of HP's departure. He did so in our Open Mike column in the November printed issue of the NewsWire. (It would be almost two years before we'd start up this blog.) It's fun to track the predictions in that column. Beauchemin, heading up a group that itself would remain open just another 20 months, collected sentiments from community notables including the late, great Wirt Atmar, who would pass away a little more than five years later.
Wirt outlived HP's 3000 business, right down to the closing of its MPE labs at the end of 2008. Unless you're reading this from the blazing-fast Google Fiber of the afterlife, you've also outlived the end of HP's 3000 saga. For HP computer users whose systems are facing an end of manufacture, the following is educational. It's memorable for migrators to revisit that time of reflection, too, and see if anything resonates in today's platform ownership.
Please leave a comment below to share your own story of the 10 years that have followed this anniversary. Or email one to me to tell your tale of what has followed the Wake.
By Denys Beauchemin
On All Hallows Eve of the year 2003, an historic event took place without fanfare and virtually ignored by the vast population at large. Only the cognoscenti will mourn the passing into computer history of the HP e3000, née HP 3000. This magnificent machine, which would be marking its thirty-first year of existence next month, is instead disappearing from the list of HP computer products. End of Sales for the HP 3000 is now upon us.
I was first introduced to the HP 3000 in 1977 somewhere in New Hampshire. At that time I was working in Montreal on an HP 21MX designing and programming applications in a timesharing bureau. I immediately took a liking to the HP 3000, transitioned jobs to be able to work on one and joined the users group for the first time. Over the years wherever I worked, there was always an HP 3000 in my environment. The HP 3000 has been part of my career almost from the beginning. Its passing fills me with melancholy, and whilst I had not been doing as much with it these last several years, I could always count on it being there, adding new capabilities along the way. This is true no more.
I asked a few luminaries of this long-lived computing environment to reflect on the machine, its passing and perhaps to shed some light on this event and what its effect might be.
“A great IT platform: reliable, affordable, flexible, easy to operate, and easy to program. And every release compatible with the previous for over 30 years. Perhaps some future OS team will adopt these same goals.” — Bob Green, Robelle
“The HP 3000 has been one of very few computers with a very important property: it lets people get things done. Because of that, it’s been my primary professional focus for the last 24 years, and hopefully for many years to come. Its cancellation was the straw that broke the camel’s back in my regard for, and trust in, HP as a company.” — Stan Sieler, Executive Vice President, Allegro Consultants. [Ed. note: Sieler marked his 30th anniversary at Allegro this month.]
“One of the worst things a hardware company (which subsequently develops some excellent software) can do to that software is to support it as if it were hardware. The 3000 was a victim of such treatment. RIP.” — Fred White, Co-creator of IMAGE
“My association with Hewlett-Packard began in 1963, when I was first introduced to extraordinary quality of HP instruments. Our official association with MPE began in 1976, and it too represented to me the very highest ideals of quality engineering. MPE was a magnificent operating system, simple, stable and extraordinarily efficient. The death of MPE concerns me greatly about the future of HP itself, not because MPE was ever a substantial contributor to HP’s bottom line, but because its death is indicative of the kind of company that HP is now casting itself as: a manufacturer of commodity products, having wedged itself in between Dell and IBM, a virtually unsustainable niche. I have come to believe that the most likely scenario now for the future of HP is for HP to be bought by Dell in three to seven years, just for the printer division, with the remainder of the organization either sold off or disposed of. If true, that’s a sad end for a company with which I’ve proudly had a life-long association.” — Wirt Atmar, AICS Research, Inc.
“When HP announced that it was no longer in HP’s best interest to continue with the HP 3000, my reaction was one of joy. I believed that — once HP was out of the HP 3000’s way — MPE-IMAGE would be able to prosper ‘under new management’. HP, unfortunately, had other ideas. Be it as it may, I feel a tremendous amount of loyalty towards MPE-IMAGE users and, as HP’s MPE-savvy people dwindle, I keep adding more and more items to my to-do list. I love IMAGE and I continue to work, on a full-time basis, searching for ways to make the lives of TurboIMAGE users as rewarding as possible.” — F. Alfredo Rego, Adager.
“The HP 3000 has been my business companion for 26 years, providing continuity for my COBOL application development. It enabled my company to become an international solution provider and its tragic demise is a reminder of my own mortality on this earth. May the spirit of MPE live on forever in the user community it leaves behind. I believe that inside every HP 9000 there is an HP 3000 waiting to be released after October 2003.” — Jeanette Nutsford, Computometric Systems Ltd, New Zealand/UK/USA
“I came from an IBM mainframe background and then started working on the HP 3000 at HP as a Systems Engineer on the Series II in 1976. I knew I had gone to heaven when I could use a terminal to do compiles and queries in a very short time and on-line with a very user friendly operating system, MPE. Times were good then in the user community because everyone was in a learning mode and helped each other. Times have changed and we must now move on to new challenges. I really miss the good old days but am glad to have met a great circle of friends along the way!” — Paul Edwards, Paul Edwards & Associates.
October 22, 2013
3000 stays above water at manufacturer
Ed. Note: The HP 3000's ability to remain running over more than 25 years has kept it in service at MacLean-Fogg. IT Director Mark Mojonnier updated us on the current status and future plans for their MPE/iX server. At times, the computer simply needed to keep its (disk) head above water.
We've been running HP 3000 systems since 1983. The company was originally part of Reliance Electric out of Cleveland years ago. In 1986, Reliance sold a piece of that business to MacLean-Fogg company in Mundelein, IL. The new company, Reliable Power Products, bought its first HP 3000 Series 48 in 1987. We had a flood in the building later that year and had to buy another one. The disk drives were high enough out of the water to survive, so when the new one arrived, we warm-booted it (with the old disk packs) and it picked up right where it left off.
At the time we bought our first HP 3000, there was a single manufacturing location to support. Now, there are 11 manufacturing facilities in North America we support. The business has grown from $25 million to about 10-15 times that now. Same base software -- just a lot more functional these days. It evolves constantly.
Since those first days, we have had an Series 925, Series 957, Series 969, Series 989, and now an N4000-750 (for production) and a N4000-500 for DR. We run home-grown ERP software written in COBOL. We run about 200-250 users pretty much all the time. The system runs 24/7/365, basically unattended. We have developed all sorts of notification software that pages, texts, emails, and calls when the system sees $STDLIST for unexpected things that went bump in the night (or day). There are two of us that write the software, manage the OS (not much to do there), and handle the day-to-day activities.
When we bought our latest HP 3000 systems, we found that these were actually new machines that had simply been in storage for many years. These new ones simply blow the doors off that Series 989. Fiber vs. single-ended SCSI is no match on throughput. Our 2-3 hour overnight processes dropped to 1 hour. Just being able to backup to LTO instead of DLT made a big difference.
We moved from that Franklin Park address simply because the building would have a tendency to flood just about every year. In 1987, we had 18 inches of water. This was called the flood of the century. 18 inches rising up the back of a Series 48 doesn't leave much dry space. Then, starting in 2008, we had a flood almost every year through 2013. Now, 6-8 inches of water in the office doesn't go over too well with furniture and office equipment. But those floods never hurt the HP 3000s.
After about five of these floods, the company decided to sell the building. They moved the factory to Tennessee and the corporate offices to South Carolina. The 3000 now resides in Mundelein, where the two of us continue to keep the 3000 running.
However, as most 3000 sites go, the system is being phased out over the next few years. We are installing EPICOR. It runs on a cluster of Windows and SQL servers. The plan is to phase it in, and phase out the HP 3000, all over the same time period. The two HP 3000s will remain around for a few years after that to hold archived data.
I hate to see them go, but I've been working with them since 1983 (30 years and counting) and hope to see another four to five years. It's been quite a ride. I hope it continues for a few more years.