Earlier in August a 3000 manager who relies on the Stromasys virtualized 3000 was searching for failures. Well, he was asking about the causes for failures. He wanted to know more about failures of SFTP transfers on his MPE/iX system. (We'd call it a 3000 but there's no more HP iron there at Ray Legault's shop). He gave the rundown on the problems with MPE/iX.

We send about 40 files each day most of these in the early morning. Sometimes we would have zero to fives connection failures each morning. I noticed that these failures seem to occur when two SFTP jobs ran at the same minute. I then added a "JOBQ=FINLOG" to the job card of every SFTP job I had and set the job limit to 1. This was two weeks ago and we have not had a failure yet.

Brian Edminster, who still hosts open source software for MPE/iX, checked in to offer an answer to why those SFTP jobs were failing.

I'd be willing to bet that Ray's issue at Boeing with SFTP connect failures is due to the Entropy Generator running dry. Connections take lots of entropy data — and the one that comes 'out of the box' with the SFTP client doesn't generate very much without some modifications.

If you need to make more than one connection a minute (job limit 1), this modification will likely become necessary. Let me know if you'd like some pointers on how to do this. It will require some revisions to the SFTP software. The Entropy Gathering Daemon which Mark Klein's SFTP port uses is written in Perl. It is not terribly difficult to modify to include new data sources to "stir into the pool" that is drawn from by the SFTP client.

Edminster's MPE-Opensource.org website has an SFTP quickstart bundle of all packages required to install OpenSSH on MPE/iX including SFTP, scp, and keygen.

Podcast: Ending the Reruns

Podcasts have become more popular than ever. We started recording and sharing stories about 3000s back in 2005 when blogs were just taking off along with the audio content that people think of as free. It's free to listeners, and the good companies sponsoring the NewsWire take care of the expenses. Thanks to the backing of firms like Pivital Solutions (support service) and Stromasys (emulation) and Hillary Software (file sharing software that's 3000-savvy) we can bring audio about MPE to you.

I call it MPE Audio because it's told by voices, my own and those from experts in the field. Some of them gathered at this summer's 3000 Reunion. A chalk talk out there in the Bay Area, across the street from the former HP campus, examined what homesteaders need to succeed. In this case success is overcoming the age of HP's 3000 iron. And storage. And so on.

There's a new wrinkle in the watch-out category. Not that the old disks have started running more reliably. It's just that other media is a failure point too. DDS has gotten older, along with managers who know MPE. Companies are treasuring the latter. The former is turning into trash.

The cozy corner of your world that's the HP 3000 Community on LinkedIn has many reports about where users are working. One story that's slid onto our desk from that community illustrates the work a 3000 owner faces when a software vendor leaves them behind. For many sites with few resources, this is the situation that triggers a migration.

This is a story of what happens when the Hewlett-Packard hardware outlasts vendor applications. It's not that the app has stopped working, not at all. But the manager of the 3000 knows his employer has limited time to plan for a future while using software that can't be modified, because there's no source code — and not much documentation, even though the software's been customized.

3000s stay in service long enough to get to the point where the most durable piece of the solution is MPE/iX. Hardware can die off and be replaced by Charon from Stromasys. Expertise can be hired on contract for administration and operations. Development, though, is the hardest asset to maintain. When expertise dwindles for a third party piece of software — in this case, the IBS/3000 (Integrated Business System) financials package from DeCarlo, Paternite and Associates — a migration is usually in order.

The 3000 manager said he came on board at the company and learned about IBS being a key business component. "I Googled it to see if there was a user forum, vendor site, or any information. All I found was Irritable Bowel Syndrome."

Even if there was still vendor support available, it would be of limited use because there are so many in-house modifications and enhancements. The app has served us well, but is not sustainable for the long term.

He reports that the migration project will have to survive bad decisions, woefully inadequate management support, bad consultants, and a lack of expertise. "New leadership is in place with more realistic expectations. I hope we can hit a 2019 target." This manager of an A500 system is counting down to an end of 2018 retirement — and doing his best to ensure the system can outlast him.

Open source software developed a reputation for keeping HP 3000s online and productive, even in the face of industry requirement changes and new government regulations. Applied Technologies founder Brian Edminster has shared reports of a 3000 installation processing Point of Sale transactions, a customer which faced new PCI compliance demands. He was tasked with finding a solution to the new credit card compliance rules late in one December — with a January deadline.

“What we were struggling with was not that uncommon,” he explained. “The solution of choice was a version of the package OpenSSH, an open source implementation of a secure shell.”

OpenSSH offers publicly exchanged authentication, encrypted communication for secure file transfers, a secure shell command line, port forwarding. “It’s amazing how much you get," Edminster said, "and it’s available for many operating systems.” He's got a website devoted to the open source tools for the 3000.

In the summer months of 1996, HP was plugging 3000s in where mainframes were serving. Jim Murphy, program manager for the mainframe replacement project, told the 3000 community that IBM mainframes from 30 years earlier were getting the boot because HP was building servers better than the Big Blue iron.

The company was finally using 3000s to do the work they were built to do. An order fulfillment system called Heart was driving every sales fulfillment. Payroll for HP in North America was also performed using MPE/iX.

1996 was a hard year for the 3000 in some places. The spots where HP's reps felt that only a Unix solution — mistakenly called an open system — would win a sale were no-3000 zones. As a separate division, GSY's 9000 group never wanted to give any ground to HP's commercial computer line. At times, 3000 sites would be encouraged to get a open computer from HP. Plenty of the mainframe replacement in HP involved HP-UX systems.

By the time the August 1996 conference gathered in Anaheim, California, Murphy had a paper in the Interex '96 proceedings. HP IT Program to Eliminate Mainframes explained to a conference full of 3000 owners and managers that it was all HP systems inside the corporate data center by May 17, 1996. The 3000 was a key element in HP's modernization.

The role of the HP 3000 in HP's mainframe elimination process is important from two perspectives. First, as the number of data centers within HP rose, the reliance on IBM-style mainframes did not: the HP 3000s carried a fair amount of the increasing processing loads. Second, as IT began rewriting IBM-based COBOL applications for the 3000 platform, many of the re-writes included moving to client/server architectures. This meant HP IT was becoming familiar with client/server as early as the late 1980s.

The paper is archived at the OpenMPE website.

The Anaheim conference was notable for another big announcement. The World's Largest Poster was unfurled in the winds of a nearby high school's football field. "MPE Kicks Butt" was the slogan on those acres of paper. Inside the HP IT datacenter, the 3000 had kicked sand into the face of some of the company's most critical mainframe systems.

The Stromasys Charon software is a fact of life in the homesteading community by this year, after almost six years of field service. Lately the emulator users have been offering insights on how they're using their servers.

It's a lot like any HP 3000 has been used for the last 44 years, in some ways. Transferring files. Queueing up jobs. A few of the emulators shared their advisories not long ago.

Ray Legault at Boeing talked about his experiences with file transfers, especially an SFTP client and the SFTP "Connection refused" errors. As the Charon developers like to say, if the MPE/iX software behaves the same on the emulator as it does on 3000 hardware, even if MPE registers an error, then Charon is doing its faithful emulation job.

"We are running on a Stromasys Charon A500-200 and a A500-100 virtual machine which executes on a HP ProLiant DL 380 Gen8 3.59 GHZ CPU, with 6 cores and 64 gig of memory," Legault said.

We send about 40 files each day most of these in the early morning. Sometimes we would have zero to fives connection failures each morning. I noticed that these failures seem to occur when two SFTP jobs ran at the same minute. I then added a "JOBQ=FINLOG" to the job card of every SFTP job I had and set the job limit to 1. This was two weeks ago and we have not had a failure yet.

Another emulator user, Tony Summers of Smith & Williamson in the UK, shared queueing advice and a massive job checker (HOWMANY) that's working well for him.

Just a few weeks ago, a 3000 manager using an A-Class server checked in on how he might connect the SC-10 arrays from Hewlett-Packard to his A500. As a West Coast service provider carried the manager toward that hardware (it can be done) it seems like a good time to review the use of storage arrays with MPE/iX systems.

Our founding net.digest editor John Burke covered this ground in the years after HP announced it was cutting off its 3000 operations. While the HP label is still anathema to some, the hardware prices are sometimes too compelling. Here's Nike Arrays 101, advice still worthy on the day you're moving around arrays connected to a 3000.

By John Burke
Newswire Classic

Many 3000 homesteaders are picking up used HP Nike Model 20 disk arrays. The interest comes from the fact that there is a glut of these devices on the market — meaning they are inexpensive — and they work with older models of HP 3000s. However, there is a lot of misinformation floating around about how and when to use them. For example, one company posted the following to 3000-L:

We’re upgrading from a Model 10 to a Model 20 Nike array. I’m in the middle of deciding whether to keep it in hardware RAID configuration or to switch to MPE/iX mirroring, since I can now do it on the system volume set. It wasn’t in place when the system was first bought, so we stayed with the Nike hardware RAID. We’re considering the performance issue of keeping it Nike hardware RAID versus the safety of MPE Mirroring. You can use the 2nd Fast-Wide card on the array when using MPE mirroring, but you can’t when using Model 20 hardware RAID.

So, with hardware RAID, you have to consider the single point of failure of the controller card. If we ‘split the bus’ on the array mechanism into two separate groups of drives, and then connect a separate controller to the other half of the bus, you can’t have the hardware mirrored drive on the other controller. It must be on the same path as the ‘master’ drive because MPE sees them as a single device.

Using software mirroring you can do this because both drives are independently configured in MPE. Software mirroring adds overhead to the CPU, but it’s a tradeoff you have to decide to make. We are evaluating the options, looking for the best (in our situation) combination of efficiency, performance, fault tolerance and cost.

First of all, as a number of people pointed out, Mirrored Disk/iX does not support mirroring of the System Volume Set – never did and never will. Secondly, you most certainly can use a second FWSCSI card with a Model 20 attached to an HP 3000

Ten years ago this month the LinkedIn group for HP 3000 users opened for communications. In a posting from August of 2008 we took note of 30 members in a new group devoted to a server that hadn't shipped a new unit in five years.

There's no more new servers today. But by now those 30 members have turned into 669 and growing. It's been a pleasure to curate the group (admit new members) over this decade and spark some conversations, too. A few weeks ago I asked what people were still doing with their HP 3000s. Some of the newer members are old hands at the server. Edwin Clements, who just became a group member this week, worked at Harland Clarke back in 2012 as a COBOL specialist.There's new resources in the group, too. Matt Barker, the CMO of Stromasys, is a group member. 

HP 3000s remain on duty in surprising places. That's the Harland Clarke disaster recovery design up there at the top of this post. A pair of HP 3000s were working in the direct marketing end of one of the world's largest check manufacturers.

The membership of the group is something special, since it's hand-tooled. Anyone can request a spot, but only the clear 3000 users and experienced vendors have a place there. A LinkedIn group is often overrun with careerists whose skills don't match the discussions. Almost 80 pending members are on the outside looking in. If your resume doesn't include MPE, you're probably not a member.

Lately the LinkedIn group has been identifying itself with stories of durability. Some members are continuing to work with the server. Others have experience waiting if the opportunity surfaces to use it. It's a good place to look for someone you might've lost touch with.

In early 2028 I’ll be 70, and some MPE/iX apps could be 40 years old. I can hope retirement is in my rangefinder by then, but at the moment it looks like I’ll be writing until I can’t make sentences anymore. (Gotta remember, first noun, then verb.)

Well before that year, though, the roadblock of 3000-MPE date handling will be cleared. The companies most likely to have a comprehensive workaround are the ones which have licensed MPE/iX source. Or, the companies which are allied with the source licensees. Fixing the use of the CALENDAR intrinsic doesn't need to be a source-level repair. Having source access, though, only makes the fix more robust.

The 3000 owners and managers are all about robust. That's why they're still making use of a computer the vendor hasn't sold since 2003. Many of the companies who don't self-maintain are relying on support services now working more than seven years since HP left that support business, too.

The established independent support companies will be glad to collect money for building a 2028 solution, customer-by-customer. They should be paid. There are some IT managers out there in the 3000 world who see leaving their existing systems in a future-proof state, software-wise, as part of their job whenever they get to retire. Those are the real Boy Scouts, I’d say. On the other hand, you will hear arguments they’re not doing their jobs by leaving their companies running MPE/iX, even today.

The heart wants what it wants, though. If a company hasn't got heart for a migration right now, then the adminstrative work to be done is preparing for a forever journey for MPE/iX. Or at least until I'm 80, when the Unix 2038 roadblock appears.

Nobody should be building a 2028 fix unless they’re going to be paid. This issue is important to the Stromasys customer base. Not all: some Charon 3000 emulator installations are holding a place for a migration that's underway.

The community's elders care about the future. So long as the old managers can get a new expenditure approved, the game's afoot, as Sherlock Holmes would say.

Newswire Classic

Twenty years ago HP took steps forward, into the realm beyond 2028, when it released a set of COBOL-related MPE/iX intrinsics. The community is now looking into the next decade and seeing a possibility of hurdling the Dec. 31, 2027 date handling roadblock. In this Inside COBOL column from the late 1990s, Shawn Gordon took readers on a quick tour of the new intrinsics — new to 1998, at least — that would make the 3000 easier to program for the future. He even wrote a sample program employing the improved data handling.

In 2018 the information might seem more history lesson than operational instruction guide. But when a long-running mission critical app needs repairs, knowing the full set of date capabilities might help. Gordon even mentions that using the official intrinsics will help maintain programs written 20 years earlier. Enough time has passed by now that any new programs at the time of the article would be 20 years old.

3000 managers have always had a sharp focus on coding for long life of applications. 

By Shawn Gordon

Since Year 2000 is rapidly approaching, I'll review the date intrinsics that HP gave us in MPE/iX 5.5 starting with PowerPatch 4.

As I've done a lot of Y2K consulting it seems everyone has written their own date routines. Most I have seen will break by Y2K. My goal in my consulting was to implement an HP-supplied solution, making it easier to support YYMMDD as well as YYYYMMDD date functions during the conversion process.

My only negative comment about these intrinsics is that I wish they had been created with the introduction of the Spectrum series of HP 3000s (PA-RISC systems). I could have used them then, too.

Six new intrinsics are available. All of the parameters for all new intrinsics are now 32-bit. This means they will work for as long as anyone reading this will ever care. I feel it’s important to standardize on these new HP-supplied intrinsics. They will make it a lot safer than trying to maintain some piece of code that was probably written 20 years ago. With code that old, it’s likely that nobody remembers how it works.

Here’s the lineup of intrinsics:

1. HPDATECONVERT: converts dates from one supported format to another 
2. HPDATEFORMAT: converts a date into a display type (I usually use this instead of HPDATECONVERT)
3. HPDATEDIFF: returns the number of days between to given dates 
4. HPDATEOFFSET: returns a date that is plus or minus the number of days from the source date
5. HPDATEVALIDATE: verifies that the date conforms to a supported date format
6. A new 32-bit HPCALENDAR format (HPCALENDAR, HPFMTCALENDAR).

The Newswire's articles can sometimes be evergreen, even in the hottest months of the year. This week we got an email about a 2001 article that introduced Linux to our readers. A companion to the article from 17 years ago, A Beginners Guide to Linux, includes one outdated link, along with timeless advice.

Linux was already a juggernaut on corporate IT whiteboards and it had a strong following in the field, too. Shawn Gordon wrote a pair of columns about Linux as a 101 course for 3000 experts. The first article was published in the first weeks after HP's exit announcement about the 3000 business. Gordon, who founded a software company built around Linux applications, connected the dots.

To be honest, you can go another seven years quite easily with your existing 3000 system, which is a long time for a system these days. But if you were looking for a change anyway, now is the time. So what does this all have to do with Linux?

Linux seems to be the great equalizer. It runs on watches, set-top boxes, PDAs, Intel chips, PowerPC chips in Macs and IBM systems, Itanium chips, IBM mainframes — the list goes on and on. IBM and HP both are moving their customers towards it, and IBM has done some fantastic work helping Linux on scalability.

In our HP 3000 space we mostly know the players and we are comfortable where we are. Jumping over to Linux required that I learn a lot about things I never cared about before — like the GPL, GNU, Linux, RMS, ESR, and other things that I will explain in a bit. One of the bits that has been floating around a lot on the various 3000 discussion lists is Linux.

The update for the two-part article comes from a company that has a free Linux education website. Alex Nordeen, Editor of Guru99.com, hopes to get your web visits.

Despite having both hardware and application vendor deserting them, companies who chose the Ecometry e-commerce software to run on HP 3000s keep making sales. Fluent Edge Technologies has served Ecometry sites for more than 20 years. Cliff Looyenga checked in on the LinkedIn HP 3000 group to mention that five or six companies in his client list continue to use MPE/iX for Ecometry.

"Ecometry is currently owned by JDA," Looyenga said. "They no longer provide any support for the HP 3000 version. They don't even advertise the current Windows version. In my opinion, Ecometry is dying a slow death."

"On the positive side," he added, "they are still enhancing and collecting support revenues for the Windows version. For users of the HP 3000 version, support comes from ourselves, Snapshot Design, Hire Experience, and Odin Technologies."

Here's the best part of the report. "We have continued to see support demand decrease," he said, "as more clients are moving off Ecometry altogether and going with other vendor solutions."

The fate of the 3000-using companies has had many seasons since 2001. Losing the vendor's support for hardware, for MPE/iX, for applications: these are events that trigger opportunities for replacement expertise. There are four suppliers of Ecometry support today, more than 16 years after HP declared the 3000's ecosystem doomed.

"I still have clients running on the HP 3000," Looyenga said. "One of them is running the Charon emulation software. None are planning to get off anytime soon."

The coming change for HP 3000 date-keeping is a product of the computer's operating system. The hardware will run as it always has, no matter how far ahead calendars and dates reach into the future. Even into the year 2028 and beyond. For some users, they're heard about this as the "2027 problem."

The CALENDAR problem is in the OS, not the hardware. The old intrinsic was only built to record accurate dates until the end of 2027. Any resolution will involve work within applications' use of intrinsics, among other software revisions. Replacing CALENDAR with HPCALENDAR is part of the solution. Charon sites will have to prepare for it too, because they are running faithful virtualizations of the PA-RISC hardware — and use MPE/iX

Hardware vs. software in 2028 is a common misunderstanding in the 3000 community. Everyone figures the Hewlett-Packard 3000 hardware, pushing 20-25 years in service in most places even today, won't be able to keep up. It's not the PA-RISC chips that will make a mistake identifying the correct year, though. It's MPE/iX.

There's a way around this aging software intrinsic: replacement. HP built a perfectly serviceable improvement of calendar intrinsics, HPCALENDAR, when it became obvious the 3000 community was going to go well past the Year 2000. HPCALENDAR isn't wired into the OS's roots, though. SHOWTIME, for example, is always going to report an incorrect year starting the first day of 2028.

Applications that can be revised to use HPCALENDAR will stream jobs on correct dates. Native job-streaming service in MPE/iX will work if a command uses a request such as "three days from now." In general, the more closely a piece of MPE/iX software relies on CALENDAR, the less likely it will be to deliver accurate dates starting in 2028.

Source code revision will be the most direct solution in some cases. Support companies are assembling certification services for Year 2028 operations. It's a place the 3000 community has worked through before now. We all remember how Y2K didn't halt 3000s. Developers, vendors, and support experts all say that 2028 is nothing as serious as Y2K was — so long as customers are aware of it and prepare.

We put together an FAQ about 2028 last year after vendors and users starting talking about the CALENDAR impact. It's worth a look and perhaps a forward to your support team if their answers are "that's a 3000 thing with HP's hardware" or they don't know how the year 2027 will end for MPE/iX. In December of that year, dates that are supposed to be reported at 2028 will say 1900. There's a strategy to repair that.

Illustration by Melanie Demmer

A few years ago the world's astronomers kicked Pluto out of the list of planets. 3000 owners and managers might know how that feels. Pluto was a perfectly acceptable planet for 80 years, like the 3000 was a small yet notable server in HP's enterprise solar system.

Like the 3000, Pluto didn't change to trigger its exile from the list of nine planets. Astronomers started to revise their specs for being a planet. Pluto wasn't big enough and was too far away from the sun to get the dispensation Mercury gets. But this celestial object is still in orbit around that sun. It's a little like the MPE/iX apps that are running at Boeing. 

Charon is the largest moon of Pluto's, so big it shares a gravitational field with the planet. (No, wait, it's a dwarf planet, Pluto is.) Charon is so big, compared to Pluto's size, that the two objects face one another with the same aspect constantly. Charon is in tidal lock, as one scientist explains it. That moon reminds me of the Charon software that powers those apps at Boeing today. Its emulation of the 3000 keeps it in lock with the PA-RISC chips that continued the orbit of MPE/iX at the world's largest aircraft maker.

The fate of Pluto and its moons graces the pages of a new children's book, A Place for Pluto. The book is illustrated by Melanie Demmer and written by Stef Wade, both making their debut in kids lit. Pluto does find a place by the end of the book, but I won't give it away. I have a grandson and a granddaughter who will enjoy the book soon.

Those kids are close enough that I don't need any time spent sitting in a Boeing product to see them. While I pawed through recent messages from 3000 users and fans, though, I thought of being exiled and how there's a place for everybody if they keep working and looking.

"Boeing is going as virtual as possible," system administrator Ray Legault of Boeing wrote, "using X86 hosts with .net and Java coded applications where possible. The applications' owners are responsible for funding the re-writing or retiring or migrating of applications, not IT. They are looking into Pivotal Cloud foundry, which requires .net or java coded applications.

Legault said the virtual HP 3000, powered by Charon "will be gone within two years when the Finance organization migrates all Finance applications to 1SAP." That's the moment when the MPE/iX moon will fade from the horizon at Boeing. By that time, it will have been carried through the skies by Charon for more than five years. That's a longer MPE orbit than HP imagined. By 2020 and that last Finance transaction at Boeing, it will be 19 years since HP announced the end of its 3000 business.