February 28, 2006
Change comes from different directions for Unix customers
Change in the computer business happens in little bits. A lot slower than mathematical formulas can predict. In our weekly podcast (5 MB MP3 file) we talk about how predicting the future on the basis of the past can be a trap. An HP rep explains how change works for the Unix customer, as well as your OpenVMS brethren. HP understands a customer base that wants to stay where it is forever. Now, anyway, since the vendor is trying to sell Integrity and Itanium servers anywhere it can.
Hear about how HP’s Unix customers will become Integrity users, putting PA-RISC to work right alongside Itanium in a server’s frame. If you wish your future might have been derived from a past of loyalty to HP, nobody can blame you.
February 27, 2006
Free HP 3000 CPU Upgrade in a Heartbeat
By Gilles Schipper
A recent query on the 3000-L newsgroup about converting DTC configuration from OpenView-based to host-based brought up some memories — as well as lessons learned about significant CPU-intensive excesses as the result of improper DTC setup and configuration. In the early 1990s, with the advent of HP 3000 host-based telnet, one of our customers no longer needed their DTC Telnet access card (TAC) to enable network online access to their HP 3000.
And since the TAC was the primary reason they utilized Openview DTC Manager, they asked if it was possible to eliminate the additional hardware, software, and management overhead associated with Openview DTC Manager.
The answer, of course, was yes, definitely and absolutely — or so we thought.
We made meticulous plans to reconfigure the DTC-associated devices to use host-based DTC configuration, instead of the current OpenView-based configuration.
Coincident with the enabling of that new configuration, we could discard all the OpenView-required extraneous hardware — such as the Openview Windows for Workgroups 3.11 workstation, as well the required Windows, OpenView and associated DTC software. The telnet card in the DTC could also be decommissioned.
At the same time, we decided to convert the DTCs from thinlan BNC connections to 10base-T RJ-45 connections. The project was to be implemented one weekend after all users had been notified well in advance.
Cutover time came and we proceeded to make the planned changes. We copied our newly modified nmcfgnew file onto NMCONFIG.PUB.SYS and subsequently performed a graceful shutdown of the HP 3000.
Then, we needed to partially disassemble each DTC48 in order to change the large jumper switch therein to enable the MAU port instead of the BNC port.
After re-assembling each of the four DTCs, we attached a transceiver to each of them and attached to the LAN via Cat-5 cables to a hub.
After rebooting the HP 3000, we immediately noticed a problem.
The activity light on LDEV 1 was abnormally high and we noticed very sluggish response time, even though only the console was signed on and no batch jobs were executing.
Having no idea what the problem was — and also absent any tools such as Glance to shine a light on the situation — we began to revert to the pre-conversion configuration, software and hardware.
This meant reverting back to OpenView-based configuration, as well as BNC DTC connections.
Only a week later, with some analysis of the NM log files, were we able to establish what was going on. Long story short, the problem was related to the transceivers and the fact that SQL heartbeat was disabled for all of them. The result was that the CPU was being inundated with an overwhelming amount of IO requests in order to log the missing heartbeat event in the NM log file.
This unnecessary and voluminous IO was enough to bring the system to its knees — even absent any other activity.
In today's HP 3000 environment, this serious CPU wastage problem can be overlooked, because faster CPUs could render the problem relatively less noticeable.
But I would venture to guess that there is a lot of the "wasted IO" that is affecting a large number of HP 3000s out there.
Fortunately, there is a very simple way to recognize whether the problem exists, and also a simple cure.
If your DTCs are connected without transceivers, you will not be subject to this problem.
Otherwise, to determine if you have the problem, simply type the following command and look at the reply that follows:
ACCOUNT= SYS GROUP= PUB
FILENAME CODE ------------LOGICAL RECORD----------- ----SPACE----
SIZE TYP EOF LIMIT R/B SECTORS #X MX
H000000A* 1W FB 5 66010 1 256 1 *
H000000B* 1W FB 0 66010 1 0 0 *
H0909A5A* 1W FB 5 66010 1 256 1 *
H0909A5B* 1W FB 0 66010 1 0 0 *
H13ECEEA* 1W FB 5 66010 1 256 1 *
H13ECEEB* 1W FB 0 66010 1 0 0 *
H15F669A 1W FB 5 66010 1 256 1 *
H15F669B 1W FB 0 66010 1 0 0 *
HASTAT NMPRG 128W FB 347 347 1 352 1 8
HAUTIL NMPRG 128W FB 424 424 1 432 1 8
HP32209B PROG 128W FB 15 15 1 16 1 1
Notice the OPEN files (the ones with the associated asterisk suffixing the file name) that are 1W in size.
There are two such files associated with each configured DTC, file name starting with the letter H, followed by six characters that represent the last six characters of the DTC MAC address, followed by the letter A or B. The EOF for these files should be 0 and 5 for the respective "A" and "B" files.
Otherwise your CPU is being subjected to high-volume unnecessary IO, requiring CPU attention. The solution is to simply enable SQL heartbeat for each transceiver attached to each DTC.
This is done via a small white jumper switch that you should see at the side of each transceiver.
Voila, you've just achieved a significant no-cost CPU upgrade.
There is also another method of eliminating this excessive CPU overhead that involves using NMMGR to uncheck as many logging events as you can for each DTC, revalidating and rebooting.
But the SQL-heartbeat enable method is a surer bet.
For those HP 3000 users concerned about the impending HP support termination – or those looking for a more personalized support offering — please consider GSA Inc.
We have been offering HP3000 System Administration Support Services throughout North America since 1983 – longer than any other third-party HP support provider.
You can visit us at www.gsainc.com for more information.
February 24, 2006
Don't be afraid to speak out on the future
For one more week, OpenMPE is holding its election of its board members. This year the vote is likely to top numbers of participants for any of the last three years. It's an interesting figure, considering that so many seem to dismiss the work this advocacy group has done.
Be a part of the forward thinkers among your 3000 community. Vote to decide who will be thinking about the needs of the 3000 user in 2007, 2008 and beyond. Have a listen to our podcast of last week, all about OpenMPE and its continued push toward your tomorrows. Take 10 minutes to hear about the issues, then get to the OpenMPE Web site and sign up. It's free.
The voting ends March 3, next Friday. At least one new board member is going to join the effort. Decide who it will be.
Even if you're migrating, the advocacy of OpenMPE can make a difference in the comfort and value you get from your HP 3000 until you turn it off. We note, by the way, that some companies who are migrating — or who already have — are not turning off their 3000s, but leaving the systems available for archive and history purposes.
In a brief but illuminating exchange, we heard from Pam Losey at a Wisconsin school district. "We have migrated off the 3000 but it is still here," she reported. "We access it from time to time for historic reoprting purposes. We moved to a Windows 2000 Advanced Server (HP ML370) running Oracle."
How many others can raise their hands and point to a 3000 still not moved out of the IT shop, even if it's moved away from their IT strategy? Isn't that hardware, still powered up and online, worth a few minutes to join OpenMPE and cast a vote?
We think so. But that's because our job is looking into the future. Open your eyes and see more, with OpenMPE.
February 23, 2006
HP works to get the word out
It's hard to run a small outpost within an $85 billion a year technology company. That might be the lesson which HP 3000 customers are getting as they watch HP try to extend its 3000 operations. Or just trying to get the customer base to pay attention to lab work requested long ago, but now finally finished.
On the effort to extend its 3000 support business by two years, customers say HP doesn't seem to know what it has decided about the server's 2007 and 2008. These are mistakes in HP's communication and process, according to staff inside the 3000 group — not a revision of HP's latest timeline.
Then there's the problem of getting the customers to change anything about their 3000s. Several long-sought-after improvements haven't been tested much, HP says. The lack of beta test sites keeps these enhancements out of an MPE/iX PowerPatch, although you can get them otherwise. If you're an HP support customer. Which seems to bring the customers back to the first dilemma, verifying support.
Jeff Vance, an HP engineer whose job includes communicating with the 3000 customer base, posted two messages in as many days to help clarify the support schedule and attract interest in those enhancements. Vance, who held a post on the OpenMPE board until the vendor thought better of the conflict of interest, has been pushing out more than his coding on new Command Interpreter (CI) features. He's also pushing out clarity during a fuzzy time for HP's 3000 operations.
He also delivered a little good news about enhancements. The FTP Phase I security enhancements (patch FTPHDG4 for MPE/iX 7.5) have "a good chance of of being included in the 7.5 PowerPatch." The same is true of new CI User Functions (patch MPEMXW1A for MPE/iX 7.5). FTP and CI improvements were well down on the list of new features out of the 2003 and 2004 Systems Improvement Balloting, however.
More popular requests have been coded, but not very well tested by the customers who asked for them. Perhaps the two-year delay might have cooled customer interest, but these enhancements are in patch jail, if you will, waiting to be sprung by customer testing. Vance said in his message to 3000 customers:
We in vCSY are trying to pull together another PowerPatch for 7.5 but there are several patches lacking sufficient customer exposure to be considered for inclusion in the PP. Only General Release (GR) patches are included in our PowerPatch builds. I am trying to get all of the recent SIB patches into this PP, but I can’t because several of them have not had adequate beta customer usage.
Vance noted that OpenMPE board member Donna Garverick seems to be the only customer so far who's tested these enhancements. So for now, these SIB patches remain in beta-test status. This means they can be requested by customers who are buying support from HP, but are not currently in the PowerPatch. Vance summarized the pending patches:
MPEMXV9C - Network Printing. This patch was re-worked due to a bug where we assumed all printers that support PCL also support PJL. Well, certain printers, including the HP 405, support PCL but do not support PJL, and this is addressed in this latest version of the NW Printing patch. It remains in BT status since no one has requested this patch.
MPEMXT3 - Large Disk family of patches. We have one beta tester as of today. These patches allow MPE to access up to 512GB per spindle.
MPEMXV0A - CI Functions - devinfo, spoolinfo, volinfo. One or maybe two beta testers. It would be great to get one more site to request and use these patches so they can move to GR status.
The patches which are available for beta-testing are summarized on a single HP Web page, jazz.external.hp.com/src/patches
Patch testing is limited to customers with current support contracts; HP won't send them out to any other kind of HP 3000 site. Support contract responses — notes on contract renewals that cut off IMAGE support at 12/31/2006 — still have mistakes. Vance did his best to assure customers HP's support is extended on MPE/iX and IMAGE right through all of 2008:
Just for the record, HP is extending IMAGE support to “at least 2008,” per our earlier announcement. We all apologize for mistakes we’re making in incorrectly renewing some HP support contracts.
If you find any discrepancies in your support renewal, please bring them to the attention of HP, either via your HP rep, or I can forward your issues to the proper support folks at HP.
If you want to send e-mail to Vance, he's at email@example.com. Just don't shoot the messenger. At the moment HP's messages are not in sync — something that can be said of customer interest and the lab's enhancements, too.
February 22, 2006
Migrate toward a return on that Interex investment
Encompass, the surviving user group that wants a relationship with you HP 3000 customers, extended its free membership offer recently. This was an offer first floated in the fall, around the HP Technology Forum, one supposed to expire Dec. 31. Not unlike HP's extension of its 3000 support, Encompass is holding the door open to its user group for an extra period — in this case, three months.
The free offer that expires March 31 applies to members of Interex. We're not sure if that's folks who were current members, left holding the bag when Interex suddenly shut down last summer, or if its anybody who had a recent membership. In any case, the application form to join Encompass for free for a year starts off with a field for your Interex membership ID. It is, however, not a required field. (We had trouble getting the compete form to load up in Firefox 1.5 and the Mac's Safari browser. Microsoft's Internet Exploder loaded it up pronto, though — if you want to check out your virus defenses at the same time you fill out the form.)
Encompass wants 3000 members, but it's still unclear what the user group will be able to do for the customer who is not planning to migrate. President Kristi Browder said that migration from the 3000 is an eventuality for the 3000 customer base, in her opinion. When we interviewed her at that Tech Forum, she also said, with a great deal of tact, that Encompass can help the 3000 customer who is a realist about the future of their system.
In our interview with Browder, we posed the homesteading question: Is Encompass comfortable with having legacy-using members among its membership?
"Yes," she said. "But it’s part of our charter to move people, too. As you saw in the OpenVMS SIG, there was a lot of passion in that room. One of our jobs as a partner to HP is to keep them as an HP customer but assist them in migration. Sometimes when you’re doing a migration you run a mixed shop, and it takes awhile.
"That’s one of the things we learned from the DEC days. We did multiple migrations: VAX to Alpha, Alpha to Itanium. We’re pretty good at migrations."
On the subject of migrations, Encompass is becoming more Unix-driven in its first decade of service (the group formed up in 2000 after Compaq had acquired Digital. Scratch an Encompass leader and you're likely to find a former member of DECUS, although that too is changing.) Many of its customers still use OpenVMS in mission-critical operations, however, an operating system now fighting to keep its place in HP's strategic picture. If nothing else, Encompass members will know how the HP 3000 advocate feels about keeping HP's attention.
Migration customers of the 3000 would be well-served to take Encompass up on its free membership offer; it's hard to beat the price. That is, if you're a former Interex member. If not, well, the user group experience still has something to offer here in the 21st Century, even in a world where the Web connects us all so much better.
February 21, 2006
Migrate with outside resource, inside savvy
Transoft has reported that it is migrating MWS Wire, a supplier of copper and specialty wire, off its HP 3000. The migration was sparked by HP's exit from the 3000 community, according to IT manager Tomm Carlson.
In a report on the migration provided by Transoft, Carlson took note of the winning combination when he was selecting a route off the HP 3000. Total outsourcing did not appeal to MWS, just like it might not appeal to a lot of HP 3000 customers — a group for whom do-it-yourself has become a mantra over the past two decades and more.
That kind of DIY approach to IT management often springs from the origins of a company's applications. So many HP 3000 customers are running home-grown apps. That was the case at MWS. "We had too many lines of code to re-write our applications," Carlson said. "It would have taken too long, cost us too much and would have been a huge risk for the business."
HP's spark to migrate brings a study of such risks. HP advises its customers that the vendor considers it risky to stay on the 3000. But moving applications of 15 years good service has potential pitfalls, too. Shifting to a new application — a less complex choice in some environments — can mean replication of business logic.
"We did look at other software packages," Carlson said, "but we didn’t want to lose the years of business logic and enhancements we’d made to the applications that are the basis of our business. We needed to maintain the stability we’d enjoyed in our IT environment for so many years."
Transoft is using its Legacy Liberator to help MWS migrate their applications to Microsoft Windows. The target platform will replace COBOL II and IMAGE from the 3000 with MicroFocus COBOL and Microsoft SQL Server for the database. Transoft says the Liberator option will be putting the MWS systems "into a modern platform much faster and with less business risk than re-writing the code or replacing it with packaged software."
“The migration of the MWS Wire mission critical business applications to Windows means that it has one common modern computing environment for all of its IT functions," said Transoft CEO Paul Holland. "Their migration is, therefore, a strategic step ensuring IT continues to meet the needs of the business, a decision driver that is well beyond how long the HP 3000 will continue to be supported.”
The Legacy Liberator tools provide a way to move the 3000's MPE apps to Windows. In the case of MWS Wire, Legacy Liberator is being used to move their VPlus screens to VB.net and the IMAGE database to SQL Server.
The bottom line, though, was Transoft's approach to the project: Providing a way to share the work between the MWS staff and its own experts.
"Although we have limited resources," Carlson said, "we didn’t want to completely outsource this project, because we have a very particular environment and wanted to maintain control. Moreover, we felt the knowledge we’d gain by going through the migration process would put us in a much stronger position to manage the new environment. The mix of outside expertise and insider knowledge was the best solution."
February 20, 2006
Tracking the Traits of Persistence
By Birket Foster
There are links to persistence that run through my life. After all, my family motto is a simple “Persevere.” (My grandmother gave me a reminder of this motto for my 21st birthday: a signet ring.) If you search on Birket Foster in Wikipedia you will find info on my great, great grandfather Birket Foster (aka Myles Birket Foster II) who was a artist and a member of the Royal Watercolour Society. His father, Myles Birket Foster, invented commercial beer bottling.
(At MB Foster, we are trying to top commercial beer bottling, but in the field of data transformation.)
The definition of persistence varies depending on the context. Beyond the trait of perseverance there are whole new meanings in the world of computing in areas like Java Beans and object-oriented computing.
When one door closes another one opens; but we so often look so long and so regretfully upon the closed door, that we do not see the ones which open for us.” — Alexander Graham Bell.
I believe there were several opportunities that occurred when HP announced the discontinuance of the HP 3000. There were and are opportunities, whether the customer’s response was to leave the platform or to “homestead.” There are risks associated with both, but managed properly, anything is possible.
“Never doubt that a small group of thoughtful, committed people can change the world. Indeed, it is the only thing that ever has.” — Margaret Mead
OpenMPE: a persistent passion
OpenMPE is a passion for a small, dedicated group. Immediately after HP’s discontinuance announcement, it was apparent that it would mean that the current version of the OS and utilities provided by HP would be frozen in time. To persist on the platform, some changes would be needed. Through OpenMPE, founded by Jon Backus, it became possible to do advocacy on behalf of HP 3000 users. The result has been that HP has made some changes to actions they were taking, as well as made changes to the OS and utilities. Examples of these changes are in the 3000’s new FTP features, and the ability to use the first 500GB of a disc added to the 3000.
Now OpenMPE is trying to put in place a funded business plan that would let it become the steward for the MPE/iX operating system and utilities source code. Some basic planning shows that it is doable, provided there is enough interest in funding the project. If you haven’t read of this project you should take a look at www.openmpe.org the official home site and catch up. If you want the HP 3000 to last beyond HPs support window, you ought to be talking to OpenMPE and financially supporting this project. In this world there are spectators and players – only the players get to direct the ball.
“The difference between a pipe dream and a great idea is persistence.” — Anonymous.
The thing that allows me the freedom to persevere as a volunteer at OpenMPE is that MBFoster provides software and services to both homesteaders and to migrators. I would be remiss if I did not provide my team at MB Foster, who are persistently looking at lowering the barriers to moving data between platforms, a lot of credit for the evolution of the abilities of the HP 3000 to play well with the world of Windows, Unix and Linux.
Persist into the future
So here we are with four years passed of the five that HP made available in 2001– and lo and behold, HP recognized that there are still many customers using HP 3000. So many, in fact, that HP saw a business proposition, an opportunity for HP to continue the profitable business of supporting the customers.
HP 3000s have persisted for a 30-year lifespan — incredible in this day and age of built-in obsolescence. In customer visits we often have them bring out the budget numbers for the support of the various platforms. Many senior managers are very surprised at the low total cost of ownership that the HP 3000 platform has demonstrated, compared to the resources dedicated to supporting Windows desktops.
Although you might decide to stay on the 3000 for some time to come, you will be well advised to keep up on the technologies that will shape the applications of the future. In general, most of the future will be based on Web delivery of services. To keep up on what is going to happen, you can check up on the World Wide Web Consortium (www.w3.org), the standards body where Web service standard are determined. At their site you will find the latest RFC (Request for Comments) for such technologies as Web Services and XML (eXtensible Markup Language). By understanding the concepts behind these standards, you can plan how to evolve and incorporate the technology into your application strategy.
We know that the world will evolve and that the risks and investments of customers will and must change over time. We appreciate the opportunity to serve your organization in whatever capacity will help you reach your goals. At MB Foster we would like to apply for the role of “trusted advisor” to assist on the journey your team is on.
I hope that each of you will find an occasion to build a plan for how you will evolve your applications for the next five years. In the plan you must have a recruit and retain strategy for your application talent, both users and technical, and you must keep it aligned with the evolution of your business. Then you should plan to make the platform you that run your applications on sustainable and extensible — so it will be a persistent factor in the success of your organization.
At MBFoster we have persisted in the evolution of our data access and delivery tools. We first built Oracle loader files in 1985, using our DataExpress product. Since then we have established ODBC and then JDBC access to data on the HP 3000. We continued the evolution to include Unix, and then Linux. With the persistence of our customers saying they need things simpler and faster, we now deliver data between all of these platforms using UDASynch, UDAConnector and UDACentral. Thanks to our customer base persisting on the HP 3000, we have been encouraged to evolve our UDA series to include the ability to allow BEA Weblogic, J2EE, Websphere and other Web Services Architectures to view the HP 3000 as a group of services — and through the process of persistence, to deliver those systems’ value to other parts of the enterprise.
Birket Foster (Birket@mbfoster.com) is chairman and CEO of MB Foster, a software company offering cross-platform data access and delivery solutions in the HP 3000, HP 9000, Unix, Linux and Windows markets. He travels a lot and hopes that he has made a difference for NewsWire readers over the years. If you have any comments, questions or suggestions on this article, please feel free to contact him at 800-267-9377 (800-ANSWERS), extension 204.
February 17, 2006
Play a role in the future this spring
Spring brings election season, and something seems to be growing in your community’s back yard. It is interest in OpenMPE. Our weekly podcast (4 MB MP3 file) takes note of the growth, even though HP put off the post-HP future of your 3000 for another two years. The advocacy group that cares for that future has more folks voting in its board election than last year.
It’s just one more thing that’s been hard to figure about OpenMPE. Have a listen for eight minutes to our view of the state of the only group working for the homestead customer's needs this year. And get out there and vote, once you become a member of OpenMPE. Membership is free, just like the deal that Encompass is offering to former Interex members. Trek out to the OpenMPE Web site and play your role in the future. Have a say in who will be talking for you to HP this year.
February 16, 2006
Hurd's first quarter succeeds, except in support
HP reported its 2006 first quarter results late yesterday, after the markets had closed, numbers that had investors buying up the stock this morning. HP says it's on its way to its first $90 billion sales year. But the only dark spot on an otherwise glimmering rebound picture shows HP 3000 customers why they've gotten two extra years to migrate. At least it gives good financial motive for HP's move.
While the enterprise storage and servers segment saw a healthy 5 percent jump in revenues, the other end of the 3000 customer's HP lifeline looked a bit frayed. HP Services revenue took a slight 2 percent decline over 2005's Q1. HP noted that if you accounted for the value of currency, revenues were actually up by 3 percent.
For argument's sake, let's call HP support revenues absolutely flat. It still represents a data point that HP wants to avoid. See, even with sales of support down, the group still reported $12 million more in profits. Support is a high-profit business, especially for servers like yours with a legendary uptime record among HP products.
By my analysis, the numbers represent the reason your calendar for leaving the platform (if you're leaving) just got pushed back to end of 2008. By anyone's accounting, support for the 3000 customer must be among the most profitable in the Services stable. Turn that off at the end of this year, like HP intended to do, and it would walk away from millions of dollars of income.
And so your relationship with HP is extended, if you're buying your support from the vendor. Enterprise business at HP, where the vendor hopes you will spend your money by migrating, showed a profit increase of $260 million, the second straight quarter of significant black ink for a group selling servers to replace your 3000s.
Details for those of you moving toward HP-UX and Integrity servers: Both saw stronger sales for the period. HP-UX revenue rose 2 percent from 2005's Q1. Integrity revenues nearly doubled from last year's Q1, and now make up 30 percent of Business Critical Server revenues. HP's fallen short of its "Itanium makes up half" goal, but the servers are up from about 16 percent one year ago. You can hardly buy a PA-RISC server anymore from HP, unless you insist. But apparently there's still a lot of insisting.
You can have a look at HP's financial details, if you like that sort of thing, in a PDF file from the company Web site.
Of course, printers contributed more than half of HP's profit for the period, another verse in a song so dear to your hardware supplier's heart. HP's CEO Mark Hurd — who the Wall Street Journal said was getting his first report card with this quarter's numbers — said the great quarter came from efficiency and streamlining your company.
"Growth was balanced across most of our businesses and geographies," he said in HP's press release, "Cash flow was strong and we were disciplined in controlling costs. While hard work remains ahead of us, our efforts are starting to show results."
HP posted its second-ever $22 billion sales quarter, following 2005's Q4, with profits of $1.7 billion. The period represented the first time that HP has had to tell analysts that sales results would be on an uptick if measured in constant currency. In reported dollars, the growth rate fell for the second straight period.
Forecasts for the company's Q2 came in ahead of expectations, as did reported profits for Q1. The result saw HP's share price rise up beyond $32 — a mark the company's stock has not seen since the summer of 2001, when HP decided to curtail its HP 3000 business. Just a coincidence there, but better times for most of HP's customers, including those still purchasing HP support.
February 15, 2006
Vendors repeat warnings about LFDS corruption
Three vendors with some of the deepest knowledge of HP 3000 databases — HP, Adager and Robelle — have posted fresh messages which update and warn customers about Large File DataSets (LFDS). This change in the 3000's IMAGE/SQL database can cause corruption in datasets greater than 4GB. You might be using LFDS, if you've got a large database, but not be aware of the risk. Or even be aware that your largest datasets now are in this risky format.
That's because LFDS is the new default for large datasets created in IMAGE C.10.00, the database version shipping with MPE/iX 7.5. HP used Jumbo datasets as the default in earlier versions of IMAGE. The vendor gave us an update yesterday on the project to repair this corrupting bug. This is an issue for customers migrating as well as homesteaders. Anyone who continues to run a 3000 with a large database and the most recent MPE/iX release should be aware of the risk.
Messages have also been posted in the last 24 hours by Adager's Alfredo Rego and Robelle's Bob Green about LFDS. Combined with the detailed technical report by Allegro's Stan Sieler in our February NewsWire and on this blog the past two days, these experts present a massive set of reasons to avoid using LFDS, even accidentally. It seems to me like the IMAGE community has waited long enough and is raising its collective voice. Even now, HP says it has pre-released another patch to continue tests of its work on the problem, a project begun in 2004.
Liz Glogowski, HP’s vCSY division escalation manager, replied yesterday to our request for an update.
A pre-release copy of the patch and a very preliminary copy of a supporting technical vendor document were delivered to a trusted third party for testing on February 2. Additional testing is also continuing at HP in parallel. Following these alpha tests, we will assess the situation and determine our release strategy. These are currently in progress but we do not have a firm completion date for them. We will give you another update when we have completed our plans.
The testing can be a time-consuming process, according to reports we've received. Since these are tests on very large databases, only the very largest HP 3000s can begin to complete a suite in a period less than several days. This level of system — a 4-CPU HP 3000 in the 750MHz N-Class tier, is usually working in a customer site, not in a third party vendor's test bay.
Adager's report was as complete as the vendor's reputation for resolving problems, full of experience, technical detail and even some wry humor. Alfredo Rego wrote:
Several MPE-Image colleagues have called me and emailed me, asking for “my personal opinion” regarding LargeFile Datasets (LFDS).
As with everything, there are pluses and minuses with LFDS. I highly recommend that, before even considering installing LFDS, you should have a serious consultation with your HP software support person(s).
WARNING: The version of DBSCHEMA released with MPE/iX 7.5 (released by HP in August 2002) produces incorrect LFDS. Without realizing it, you may have already created bad LFDS.
IMAGE version C.11.0 (not yet released by HP) corrects this LFDS problem.
With IMAGE version C.11.0, the relationship between IMAGE blocks and the file RECORDS that encapsulate them has been subtly altered for LFDS.
Regular (non-privileged) access to IMAGE data entries (via DBGET, DBPUT, DBUPDATE, and so on) continues to function without any change for LFDS.
MultiRecord NoBuf access to IMAGE blocks (which contain IMAGE media entries — which, in turn, contain IMAGE data entries — plus block bitmaps) is subtly different with LFDS.
If we were to extrapolate the schedule and budget that Adager Corporation has invested (and will have to continue to invest, to tie up a couple of loose ends that we need to update to be able to support the latest HP version), and if we were to multiply Adager’s LFDS schedule and budget “times the number of commercial products and home-grown solutions that use MultiRecord NoBuf access to IMAGE blocks”, we would come up with several person-years and several million U.S. Dollars.
Bottom line: The fact that Adager supports LFDS turns out to be irrelevant, because people do not use MPE-Image just for the fun of running Adager. People use MPE-Image to be able to run APPLICATIONS, such as Suprtool, that process data to produce information. If these applications (many of which use MultiRecord NoBuf access to IMAGE blocks for performance reasons) break down with LFDS (because of the subtle differences in the relationship between IMAGE blocks and the file RECORDS that encapsulate them), then you can NOT use LFDS.
So, besides having a serious consultation with your HP software support person(s), you must have a serious consultation with your non-Adager suppliers (and with your in-house programmers, should you have home-grown solutions that use MultiRecord NoBuf access to IMAGE blocks for performance reasons). What are the total combined schedules and budgets required to make these non-Adager systems LFDS-savvy?
It’s all a matter of time and money.
My personal opinion: I wish I had more time and more money.
Robelle's Bob Green wrote on the company's blog this morning to add more details as well as another advisory to steer clear of LFDS:
With the advent of MPE/iX 7.5 a new option for datasets greater than 4Gb became available in mid-2002 with IMAGE version C.10.00. The option, called Large File Datasets (LFDS), puts all data into a single Large File if the data set size is greater than 4Gb. This was done with very little fanfare and basically flew in under the radar for most customers and even tool vendor providers such as Robelle.
LFDS became the new default for datasets over 4Gb. Previously, datasets that needed to be larger than 4Gb were implemented using "Jumbo datasets," a series of files built in the POSIX space in 4Gb chunks. Suprtool has supported Jumbo datasets for the last ten years, since they were created.
We started to look at how Suprtool would react to LFDS, but then reports came that there was corruption when using these datasets and that HP was re-visiting/fixing the code. Some months later we then heard that a new set of patches were being released in July/August 2005 timeframe and again started to look at the impact on Suprtool. However, we now have heard that there are new potential corruptions with the July/August 2005 set of patches and HP has again reviewed the design and implementation of LFDS.
Due to the previous corruption issues and that HP has the enhancement under review we cannot state that Suprtool supports this enhancement as yet. We will keep you posted on what we learn, but at this time we certainly would not recommend using LFDS.
You will get a LFDS if you specified a size thru Dbschema that was less than 128Gb. See the details below from HP's communicator article on this feature:
By default, any data set size less than 128GB is created as a single file data set, while a data set size greater than 128GB is created as Jumbo data set. The user can force creation of Jumbo data sets, if data set size is greater than 4GB, with a $CONTROL JUMBO option in the database schema. Each jumbo chunk file would be a maximum of 4GB and can have up to 99 chunks. If the user specifies $CONTROL NOJUMBO, which is default, any data set greater than 4GB but less than or equal to 128GB will be LFDS, while data set size greater than 128GB cannot be created.
We recommend that you always use $CONTROL JUMBO for all databases. Please note that LFDS cannot co-exist with Jumbo data sets within one database.
Adager is also capable of generating Large File datasets. However, you must set a special JCW in order for Adager to select this option over Jumbo datasets.
Adager also has a utility that will convert from Jumbo datasets to LFDS. However again, I personally would only use this for testing purposes — and we are not endorsing that anyone use the LFDS feature in production.
February 14, 2006
IMAGE's Large File DataSets: The Problem and How to Fix It
[Editor’s note: HP has been working on repairing IMAGE/SQL’s Large Files DataSets (LFDS) since 2004. Using LFDS, the default in MPE/iX 7.5, can result in corrupt HP 3000 databases.]
By Stan Sieler
The basic LFDS problem: The engineers who implemented LFDS on IMAGE/SQL were unaware that all native mode languages that supported 64-bit pointers only supported them in a “single 4 GB space” mode. Please note that nearly every engineer in HP and outside of HP was probably also unaware of this compiler limitation.
Prior to the introduction of Large Files in MPE/iX 6.5, the biggest virtual address range you could get was 4 GB, by doing a “long mapped” open of a 4 GB file. The resulting address was a 32-bit Space ID, and a 32-bit offset. No compiler needed to worry about the Space ID changing while manipulating a pointer, because no address range could ever start in one space and continue to the next higher space (i.e., the Space ID would never change when doing address arithmetic).
When Large Files were added to MPE/iX, that changed.
A 9 GB Large File is assigned three consecutive Space IDs... so you have 9 GB of consecutive virtual addresses associated with the 9 GB of the file.
But that introduced difficulties. Imagine if you have an 8-byte record in a file, and the virtual address of the start of that record is $ff00.$fffffffc. i.e., it starts 4 bytes before the end of a 4 GB chunk (in our example, Space ID is $ff00, and the offset is $fffffffc). The first four bytes are at $ff00.$fffffffc, $ff00.$fffffffd, $ff00.$fffffffe, and $ff00.$ffffffff. The next four bytes are at $ff01.$00000000, $ff01.$00000001, $ff01.$00000002, and $ff01.$00000003 — so the record crosses a 4 GB boundary. (The “$” indicates a hexadecimal value; the format of a 64-bit address is a 32-bit Space ID (e.g., $ff00), and a 32-bit offset (e.g., $0 or $ffffffff.)).
Unfortunately, in all of our compilers, ordinary attempts to access those 8 bytes will result in getting the first four bytes correctly, and then the addressing will “wrap” and the wrong second four bytes are fetched: $ff00.$00000000, $ff00.$00000001, $ff00.$00000002, and $ff00.$00000003. Note the incorrect Space ID!
Some people might ask, wouldn’t the “record split over a 4 GB boundary” problem be familiar, and have been encountered during the implementation of Large Files for MPE/iX 6.5?
The most common problem, and the easiest to fix, was spotting code like:
move_bytes (num_bytes, source, destination);
The “move_bytes” would fail, because it did not understand the 4 GB boundary problem. The remediation for such calls was simple:
move_fast_64 (num_bytes, source, destination);
where move_fast_64 was a new MPE/iX routine that properly handled 64-bit addresses.
When I reported the LFDS data corruption problem, I suggested a solution along the above lines. Far less common within the operating system was code that accessed items within records that crossed 4 GB boundaries. Code like:
buffer_type = array [0..127] of integer;
var ptr : ^ $extnaddr$ buffer_type;
i, j : integer;
...code to set ptr to point into a Large File
i := ptr^ ;
j := ptr^ ;
If “ptr” is pointing into a Large File, then the Pascal compiler’s address calculations for dereferencing “ptr^” would be incorrect ... the code would work almost all of the time, but when the first byte of a range pointed to one Space ID, and the last byte points into the following Space ID (as in the $ff01.$fffffffc example above), the wrong data would be accessed.
The remediation in that case would be something like:
temp_buf : buffer_type;
...code to set ptr to point into a Large File
move_fast_64 (sizeof (buffer_type), ptr, temp_buf);
i := temp_buf ;
j := temp_buf ;
Luckily, the majority of pointer use in any PA-RISC program is of 32-bit pointers (which don’t have this problem), not 64-bit pointers.
Still, it’s tedious to track down all uses of 64-bit pointers and then determine if they need remediation (most didn’t need work).
The first attempt at fixing the LFDS corruption problem presumably involved changing most “move_byte” calls to “move_fast_64” calls. Unfortunately, I suspect that IMAGE had within-a-record pointer access in some places that were used less often (and not caught until mid-2005 testing of the first patch, reported by the 3000 Newswire) and that those uses proved to difficult to patch — resulting in a different approach to fixing the problem in the next patch.
Around mid-2005, I suggested a simpler fix: change the “allocate a new entry” code in IMAGE to simply not allocate any entry that crosses a 4 GB boundary. It would have zero effect on tools like Suprtool (i.e., code that uses the file system to read IMAGE datasets), and marginal effect on database tool vendors (or anyone using the file system to write into a dataset). I received no response from HP — perhaps they realized, as I did recently, that my proposal would limit the “hashing area” of LFDS master datasets to 4 GB.
Status, and Future
The currently shipping LFDS has a data corruption problem, and should not be used.
HP is working on a patch, one which approaches the problem quite differently. At present, it looks like the patch would require new releases of most database tools, both those that modify databases and those that extract (and/or update) information. HP is discussing this with a number of vendors.
Should you switch to LFDS in the future? Probably not.
Oddly enough, Jumbo datasets still provide for a larger dataset than a Large File would. Jumbo’s current limitation is 400 GB, compared to a maximum file size of 128 GB for a Large File. A one line change to IMAGE, transparent to users and vendors, would increase the Jumbo dataset limit to 4 TB (a five line change would increase the limit to more than 4 PB (peta-bytes)). Of course, those limits ignore other IMAGE limits that further limit dataset sizes (e.g., a maximum of 16,777,215 blocks, and a maximum block size 5,120 bytes, resulting in a limit of 80 GB).
A second advantage of Jumbo datasets is enhanced exportability. Some foreign operating systems do not allow you to create files larger than 4 GB. That means that you might not be able to transfer (e.g., FTP), an LFDS to a particular Unix site. With jumbo datasets, you would be transferring a series of files of up to 4 GB, which is much easier (and has better error recovery capability).
The sole real advantage of a properly working LFDS, compared to Jumbo datasets, would be the ability to enable DDX or MDX for one — an advantage rendered less interesting in light of existing database tools.
Stan Sieler, who led the design of major enhancements to IMAGE/3000 while working at HP, is executive vice president of Allegro Consultants, Inc., an HP User Group Hall of Fame member, developer of dozens of HP 3000 commercial and freeware products, and a co-author of “Beyond RISC! An Essential Guide to HP Precision Architecture.”
February 13, 2006
Large File DataSets: Background, Status, and Future
[Editor’s note: HP has been working on repairing IMAGE/SQL’s Large File DataSets (LFDS) since 2004. Using LFDS, the default in MPE/iX 7.5, can result in corrupt HP 3000 databases, as we have reported in the NewsWire. As of this month, HP was working with database tool vendors in testing a new fix for the problem; a 2005 attempt failed in alpha testing. Database tools vendors have documented corruption at some customer sites. Vendors recommend avoiding the use of Large File DataSets in IMAGE/SQL. Stan Sieler, executive vice president of Allegro Consultants and a 3000 veteran who led the design of major enhancements to IMAGE/3000 while working at HP, offered his insights on the problem.]
By Stan Sieler
Background: Before Large Files
In the beginning, or at least in 1994, IMAGE datasets were limited to 4 gigabytes (GB). This was because a single datatset was a single file, and MPE limited file sizes to 4 GB. Allegro Consultants, Inc., was contracted by HP to design and implement a solution to this limitation. At the time, along with other technical experts, we suggested that the solution lay in enhancing MPE/iX to support larger disk files — but we were told that that enhancement was uncertain, and certainly not likely to appear in the near future. So, we chose to implement “Jumbo datasets”, using a collection of 4 GB files to logically comprise a single dataset. Given the limitations of IMAGE filenaming, we chose to name the “chunks” with HFS names (e.g., set 23 of SALES would be SALES23, SALES23.001, SALES23.002, etc.)
Jumbo datasets arrived, and worked well — then and now.
As it turns out, the ability to use a Jumbo dataset for DDX (Dynamic [detail] Dataset eXpansion) or MDX (Master Dataset eXpansion) was originally planned to be added in a follow-on project. Unfortunately, that project was never funded.
Over the years, it was made clear to us that DDX/MDX for jumbo datasets wasn’t a critical enhancement. Utilities like Adager made it easy for users to expand their datasets quickly, and disk drive capacities/prices made it easier to add tremendous amounts of disk storage (allowing the user to always have sufficiently large datasets).
Background: Large Files
Eventually, Large Files (files larger than 4GB) were added to MPE as of MPE/iX 6.5 — but without any changes to IMAGE to support their use.
HP later decided to enhance IMAGE to support Large File DataSets (LFDS, or a dataset consisting of a single Large File), and released the first version in 2002. In March of 2004, I discovered a very serious data corruption problem in LFDS, which I reported to HP, and the 3000 NewsWire has also covered. The result was recommendations from both vendors and HP: avoid using LFDS, use Jumbo datasets instead.
At several points in the last two years, I and some other vendors have tried to get HP to drop LFDS entirely, but without success. We were concerned that users could invoke LFDS by accident, and encounter the data corruption problem.
February 10, 2006
Secrets, and hidden commands
There are two kinds of Hidden Value on HP 3000s: The commands that only the veterans know, and the processes that have been plumbed to bypass the blind alleys. Out on the Internet newsgroups and mailing lists, there's a growing number of these gems, the wheat that stands out from the chaff of Off Topic (OT) postings.
A recent online discussion brought up this problem:
I use cut and paste with EDIT/3000 to enter data to batch files. It works well except that I am limited by the size of the scratch file:
*36*SCRATCH FILE IS FULL. KEEP, THEN TEXT AGAIN.
Can I change the size of this file so I can paste more at a time?
A simple command to EDITOR/3000, a program included with every MPE system, solved the problem. Steve Thompson replied, "Immediately after entering Editor, enter "set size=######" to give yourself more space."
For other tasks, like finding forgotten passwords, and keeping them fresh and the 3000's data secret, more elaborate answers have surfaced.
A system manager pitched his plight. "My operator, in his infinite wisdom, decided to change passwords on manager.sys. Of course he forgot, or fingerchecked... I don’t know. At any rate I need some help. Any suggestions, other than a blindfold and cigarette?"
Several versions of help involved the use of utilities from security experts VEsoft. "Do you have the GOD program on your system? If so, it has PM capability, and so it can give the user who runs it SM capability. So it will allow you to do a LISTUSER MANAGER.SYS;PASS=
(That's why GOD should be secured, by the way. A randomized lockword will do the job, visible only to users who have SM capability. When VEsoft installs MPEX, for example, it installs a randomized password to MGR.VESOFT, and to GOD.PUB.VESOFT.)
Paul Edwards, ever a source for HP 3000 training, ran through the backstop methods every system manager should practice to avoid such a dilemma.
1. You do run BULDACCT prior to each full backup so you can look in BULDJOB1 for the passwords, don’t you?
2. You have another user on the system with SM capability and a different password as a backup in case this happens, don’t you?
3. Your operator used LISTUSER MANAGER.SYS;PASS just after changing the password to verify the accuracy as spelled out in the Operations Procedures section in your Systems Manager Notebook, didn’t he/she?
4. You do have a Systems Manager Notebook as discussed in detail in my papers on Homesteading on my Web site, don’t you?
Then Duane Percox of K-12 app vendor QSS opened up a clever back door:
If your operator can log onto operator.sys:
Using your favorite editor or other utility search for the string: "ALTUSER MANAGER SYS" You will notice: PAS=
, <passwd> which is your clue
Then there's the question of how to keep passwords fresh. A development manager posed this problem:
"What management wants is for a user to be forced to change their password on a regular basis. Also, certain rules must be applied to the new password. We don’t have VEsoft’s Security/3000, although we do have MPEX, and I am unable to get the funds to buy it. I therefore have two options. 1) Write something myself, or 2) See if there is anything in the Contributed Software Library that will do the job or can be modified to supply the required solution.
Bill Lancaster of Lund gave the manager a summary of his situation. "You won’t be able to accomplish this using MPE’s password scheme. You’ll either need to find the money for Security/3000 or write/creatively acquire a passwording system that sits on top of MPE and can apply all the rules you want. The absolute best option is to get Security/3000."
Homegrown and bundled solutions followed. HP's Jeff Vance offered this:
There is a pseudo random password generator available on Jazz which knows MPE’s password rules. See: http://jazz.external.hp.com/src/scripts/index.html and scroll down to the link “RANDNAME”. There are also UDCs on Jazz which force a password to be supplied when using NEWUSER, NEWACCT and NEWGROUP CI commands. These required passwords can be random (uses the script above) or user entered with a minimal length enforced. See: http://jazz.external.hp.com/src/scripts/udcvol/index.html
Then he added as an afterthought, a strategy to program your own password system:
I haven’t thought about it much, but it seems you could have a password file (maybe a CIRcular file?) for each user on the system. This file would have their last N passwords, and the modified date of the file would be the date their password was most recently changed.
A logon UDC could detect if the password file for that user exists. If not create it and require a new password right then. If the password file exists then get it’s modified date and compare that to today’s date. If greater than X days then in a loop prompt for a new password. Validate the entered password against previous N passwords and your other rules. Maybe run a dictionary checking program to make sure the password is not common, etc.
Update the user-specific password file with their new password, and then logon the user.
From the user community, Donna Garverick weighed in with this advice:
The solution that your management is asking for is going to cost more for you to develop/implement/maintain than it’ll take for you to get Security/3000. If you have no choice other than to develop a product, then I’d certainly model it after what VEsoft has already done. That is:
Based on a system-wide UDC, examine all sessions (it is just sessions, yes? By the way, a DSLOGON from inside a job is still a session....) against a ‘database’ (By the way, just how secure is this database? A real database needs passwords... Who’s going to maintain that? A flat file could be lockworded... but that’s not a slamdunk answer.) which is looking for the ‘age’ of the password (By the way, are you going to provide an advance warning period?).
If it is time to change the password, get the ‘new’ password from the user... but writing the rules is a pain, and keeping track of reused passwords is just annoying. Auditors in the states love when you can say the password is one-way encrypted. Dunno what your management is saying for encrypting an MPE password.
Then came a solution rolled up by Paul Christidis
Some years ago I had developed a set of command files that could be used to require users to have passwords. Later on, mostly as an exercise, I enhanced the process to age passwords and to automatically assign ‘random’ passwords as they expire. The random passwords are comprised of alternating consonants and vowels, they can have a minimum and maximum length and optionally a random digit can be inserted.
The entire ‘process’ is comprised of a system batch job (should be running always), a command file that is invoked by a log-on UDC and communicates with the batch job, a ‘control’ command file that starts and stops the batch job, a command file to determine the password age and a command file to generate the random password. Below are the comments from the batch job. They explain some of the details.
!# Author: Paul H. Christidis
!# Date: 06/17/96
!# Remarks: This job 'listens' at a message file for any requests to
!# determine if a user has a password. Once that determination is
!# made it passes back to the session an indicator to that effect.
!# A command of STOP causes the job to terminate.
!# The request comes via the execution of a command file or a System
!# wide UDC and it is comprised by the file name where the reply should
!# be placed and the user's name and account.
!# This job does NOT return the user's password, it only writes in
!# message file specified by the client the command:
!# setvar user_password true/false
!# The client then executes the command and tests the setting of the
!# variable 'user_password' to decide what action to take.
!# DATE: 06/08/2004 *** WHILE RETAINING THE ABOVE BEHAVIOR ***
!# The job logic was changed to assign a new password after 30 days. A
!# file in the posix space is built using the user's name. Then at each
!# logon a command file is used to determine the file's age, using its
!# date, and when it is older than 30 days another command file is used to
!# generate a random password. Said password is sent back to the session
!# and the user is informed about his new password.
!# If the 'job/session' name is not to be used in creating the posix
!# that will be used to age the password, then the value of the CI
!# 'pw_UseSess' should be set to 'FALSE'.
!# DATE: 06/10/2004 [Added alternative aging values functionality]
!# Alternative aging limits are kept in an ASCII file "pswrdage"
!# of 'setvar' commands, for each MPE account, MPE user or Session name.
!# should adhere to the following format:
!# SETVAR SYS_MANAGER_XTIDIS_pwage 45
!# SETVAR SYS_MANAGER_pwage 40
!# SETVAR SYS_pwage 35
!# The above have the following implications:
!# The user's "xtidis,manager.sys" password expires in 45 days
!# The user's "manager.sys" password expires in 40 days
!# Any other user of the "sys" account has their password expire in 35
!# While the 'default' 30 days applies to every other user on the
!# NOTE 1: Suffix of '_pwage' is required.
!# NOTE 2: A negative or zero setting equals to NO password aging.
!# NOTE 3: The order MUST be 'ActName_UserName_JobName_pwage'.
!# Date: 06/11/2004 [Added code to 'force' a password change]
!# When the CI variable "ForcePwChange" is set to TRUE in the session
!# executes the command file, the 'passed' code is changed and the batch
!# forces the password change (Unless it was already changed on the same
Dave Powell coded up some of the fine print nicely in his contribution:
It ought to be possible to do everything for free, using just MPE. Editing the new password is the ugly part, but if you randomly assign it that issue goes away. The next issue is that the :password command seems to like to be purely interactive, as in:
echo oldpass >> tempf
echo newpass >> tempf
echo newpass >> tempf
:password < tempf
Command not allowed in noninteractive environment. (CIERR 2500)
PASSWORD WAS NOT CHANGED.
That leaves the altuser cmd, which needs AM cap. If you don't want a background job (like Paul's suggestion), you can have the command file (called by your logon UDC) use the echo command to build job with AM cap, which it then streams, kind of like (untested, but I have working examples of cmd-files building other jobs):
ECHO !!JOB ACCTMGR/PASS.SOMEACCT; HIPRI >> TF
ECHO !!ALTUSER !HPUSER;PASS=!NEW_PASS >> TF
ECHO !!SETVAR STREAMED_BY WORD(HPSTREAMEDBY,"()",2) >> TF
ECHO !!TELL !!STREAMED_BY Your new password is !NEW_PASS >> TF
ECHO !!EOJ >> TF
ECHO Please note your new password when it appears
PAUSE 99; JOB = !HPLASTJOB
PURGE TF, TEMP
If I haven't screwed up the fine print too badly, this code in the middle of the password cmd-file runs the job that changes your password, then waits for the job to tell you what the new password is. The single exclamations before HPUSER & NEW_PASS mean that values that are variables to the session and command-file become hard-wired values for the job.
Before all this your cmd-file checks the date, gets a random password, etc., as posted by others. After it the cmd-file writes (or just builds) a file that serves as a timestamp. But I am not too comfy with putting the passwords into a plain-text file, so I might skip that part (remember, the user needs both read and write access to it). Put the command file in a group that users have xeq access to, but not read/write access.
February 09, 2006
New Alliance sends an Itanium valentine
On a podcast almost as late to market as some Itanium releases, we examine for about 7 minutes (4 MB MP3 file) the newly-minted Itanium Solutions Alliance. HP and Intel have attracted 15 partners to promote the Itanium processor as the leading choice by decade's end for mission-critical enterprise computing. Yes, that's the heartland of the HP 3000 customer, just as Itanium is the only long-term choice for using HP-UX as a target migration environment. You need to cheer for the Alliance. $10 billion of investment from this group of companies is supposed to ensure a more widespread adoption for the processor you'll be using if you migrate to HP's Unix. It should also increase the number of manufacturing solutions beyond a handful for discrete manufacturers.
February 08, 2006
Helping OpenMPE to help itself
Over the next few weeks OpenMPE will search for a few new volunteers to man its board of directors. Donna Garverick of the board posted a notice on the 3000 newsgroup and OpenMPE mailing list yesterday, soliciting fresh blood for the organization.
Some board members who've served for two years are frustrated with HP's engagement with the group. OpenMPE got scant advance notice of HP's support extension plans, according to one director. The drive to get a source code license agreement got stalled by two extra years when HP pushed back its exit date to the end of 2008.
This year's election will take place over starting next Monday and running into March, just as in prior years. The three weeks of voting ends March 3. Five of the board's nine seats are up for re-election, with just one likely to be vacated. OpenMPE members are the only customers who can vote for directors. Last year's vote gathered fewer than 60 ballots. Some directors say that's an indication of how far OpenMPE needs to go to get the community's attention. Membership in OpenMPE, after all, is free.
Working on an all-volunteer staff plan, with virtually no operating budget, OpenMPE has done an admirable job of making HP speak up on its plans for a customer base that had an incomplete migration path from the vendor in 2001. But HP's responses to OpenMPE haven't included much in the way of guarantees. Even the recent promise of a third-party MPE/iX license, extracted after two years of insisting from OpenMPE, is conditional: if there's anybody interested in such a license, and if HP hasn't wrapped up its 3000 support, a license of some parts of MPE/iX will be available.
Like much of the 3000 installed base, OpenMPE hasn't been able to find much leverage with HP in its negotiations. Directors have despaired of that ever changing. John Burke, an open critic of the non-disclosure arrangement HP demanded, has served on the board for two years under such an NDA and won't continue with a second term.
OpenMPE's future might not be bright at the moment — but the volunteers who meet on a conference call each week remain determined to make a difference in HP's endgame for the 3000. Luring new energy to an effort strung along on a four-year path, with at least three more years to go, won't be easy. But the entire mission of making a future for the 3000 after 2006 seemed hopeless, too. Heck, now there's two more years in HP's 3000 effort. The unexpected might have to play a part in making OpenMPE's 2006 significant. If you have interest in playing a role by volunteering for the board, contact Garverick at firstname.lastname@example.org.
February 07, 2006
Another 3000 enhancement, if you test
HP has put some beta-test-caliber code up on its Jazz Web server that implements three new functions of the Command Interface (CI). Jeff Vance, ever HP's source of this level of improved 3000 interface, posted a note to the 3000-L newsgroup and the OpenMPE mailing list last week:
MPE/iX patches MXV0(7.5), MXV1(7.5), and MXV2(6.5) implement the 3 new CI info functions: VOLINFO, DEVINFO and SPOOLINFO, which are available for beta-test. A Communicator article on Jazz provides more details on these functions.
A list of all beta-test patches is also available on Jazz at: http://jazz.external.hp.com/src/patches
Please help us and other customers by requesting, installing, and using this new feature.
Beta-test patches, of course, are only available to sites on HP support.
We've discussed the challenging state of HP 3000 enhancement testing in prior posts here on the blog. None of this engineering will get the general release status it deserves without the community's experimentation. Some of this CI work has been done from the 3000 group's Bangalore labs, to give credit where it is due.
Why not download a patch and try it today?
February 06, 2006
Stop looking for iSeries; IBM's renamed it i5
IBM's integrated alternative to the HP 3000, called the iSeries for several years, has been through a name change once more. Don't call IBM or a reseller to ask about an eServer iSeries; ask for the System i5. IBM added the i5 designation in 2004 to the server that most of the market knows as the AS/400. Now it's dropped iSeries, just as the installed base of IBM customers most similar to the 3000 community was getting used to calling their system by the newer name.
There's been other changes at the IBM division that manages the iSeries, er, System i5. After having been through two general managers in as many years, the group saw an unscheduled resignation of its VP of marketing, Peter Bingaman. IBM brushed off the departure — this too was the second of as many marketing VPs as years in iSeries-land — by saying what better time for new VP Elaine Lennox to take over: Just as the new systems were rolling out.
Although there's been a lot of change in the i5 offering, much of it seems to have been for the good. The server had a tough fourth quarter with revenues down 18 percent, as customers waited for the new hardware to come available. The HP 3000's performance in 2000 might have weathered the same hit, while customers put the brakes on 9x9 purchases while they waited for the A-Class and N-Class systems. IBM delivered its more powerful systems on time, however, without a struggle to meet engineering goals.
The rugged Q4 numbers meant the i5/iSeries/AS/400 saw only a 1 percent increase on 2005 for its business, even though sales were up 25 percent for Q3. HP 3000 sites who consider the iSeries usually wonder how long the server can hold out in a Unix-Linux-Windows world. IBM's been insistent on upgrading the technology in the iSeries and making it play easily with those three more popular environments.
Unlike what HP had to do to roll out its N-Class systems, IBM is letting customers stay on their current OS releases while they use the new System i5s, which are expected to come available next week. An entry-level Model 520 has the new POWER5+ CPU at a 1.9GHz clock speed. But like the HP 3000 models, the 520s have a governor that throttles them back. The difference? You can pay IBM an extra $13,500 and get use of Accelerator for System i5 that removes the "governor." HP has been adamant that the hamstrung N and A server models of the 3000 will never see such an upgrade option.
The unaccelerated models run about $21,000, while the faster versions come in at just over $35,000. Since it's an integrated solution, these ship with the software you need to put them to work. Most people buy into the server line through an application provider, so the total quote will run higher.
The new chip at the heart of the i5 runs at reduced power, just like the forthcoming Montecito processor that will power the latest HP Integrity servers once Intel ships the chips to HP. That will happen sometime this year, kicking off a race to reduce the wattage your servers use. HP and Intel claim to have a superior entry coming in 2008 with Tukwila, a processor that can idle at miniscule power when it's not in use. Itanium's future in Integrity is wedded close to any success the project will realize. We'll have more on that in our podcast tomorrow, examining the extra $7 billion HP and its Itanium partners pledged to the chip — an essential element of your future if your company is making the shift to HP-UX.
February 03, 2006
Using the 3000 as a migration platform
While migration projects grind away in the 3000 community, IT still needs to respond to user needs from existing MPE applications. The balancing act is on display all over the world, as thousands of customers try to satisfy today's demands while building a better — well, at least different — tomorrow.
Among the banks and lenders served by CASE software, customers kept needing enhancements and features to the ABLE assets suite at the same time a migration to HP-UX was underway. Rick Gilligan, the Senior Software Specialist at CASE, said the company had to walk the line between needs and wishes for tomorrow during its five-year migration project.
“We could not postpone most of the enhancements our customers wanted or needed," Gilligan said. "Many were driven by government regulation and law changes, by mergers and by other changes in our client’s businesses. We had to accomplish all of the normal and necessary improvements to our application for MPE while also migrating and making the same changes to our application for HP-UX."
These demands are commonplace in the world of 3000 enterprise solutions, a place where mission-critical services must maintain their place alongside an IT project more complex than Y2K. CASE resolved their conflict by slowing its clock on leaving the platform. HP's 1990s re-engineering of MPE/iX helped, too.
“We solved this by delaying the actual move to HP-UX of the 'migrating' product," Gilligan reported. "We did so by using the Posix features of MPE that were nearly identical to the same features on HP-UX. We did our migration to PDF on MPE, though we never made the same changes to the actual product we shipped for MPE. We also migrated from batch jobs to CGI scripts using Apache on MPE.
When it came time to migrate the VPlus forms and do testing with Eloquence, Gilligan said CASE developed scripts on MPE that would migrate and build the ABLE application on HP-UX every night. The scripts let CASE test on HP-UX during the day at the same time they kept working on migration of MPE source code to the HP-UX target. "We did this for more than a year, focusing almost exclusively on replacing the MPE-specific features of our application with Posix-specific features, before making a final cut-over to migration directly on HP-UX."
The 3000 became the development bed for the HP-UX version of the application. The developers who were working on the MPE product, during this long overlap between the two projects, were able to use their familiar tools on MPE to make similar enhancements to the migrating product — while the other team was migrating it. "This was possible primarily because the source code was still on the MPE system," Gilligan said, "as were most of the development tools. This scheme reduced our training costs, especially since we were breaking new ground. We didn't see training others on tools you are just learning to use effectively as an effective approach."
He added, "We basically limited most of the research and initial learning to a small subset of the migration team whose task it was to do the build and test on HP-UX."
As for surprises on things that were easier to accomplish than expected, Gilligan said choosing the right tool vendors helped keep most of the ABLE business logic intact.
“I had a vision that the tools we chose would allow us to keep the majority of our business logic and presentation logic unchanged, while completely changing the foundation that was underneath our application," he reported. “This vision was developed before all of the tool vendors had completed the necessary tools, and before we had anything other than a firm belief that we would be successful. Our major tool vendors — Marxmeier, ScreenJet, and Acucorp — all did a great job at making this possible.”
February 02, 2006
Migration plan needs tweaking at times
No matter how diligent your research, you may have to change your migration plans in mid-stream. When Computer and Software Enterprises (CASE) set out on its migration project in the wake of HP's departure from the 3000 market, the company thought it had the right COBOL compiler chosen for its banking and lending customers. In matters of finance, however, the bottom line turned out to be something customers watched closely.
CASE planned to take its application suite to MicroFocus COBOL from HP's COBOL II on the 3000. "We believed they had the most comprehensive solution," said Senior Software Specialist Rick Gilligan. "We also liked their support for upcoming COBOL standards for Object-Oriented COBOL." But after working alongside MicroFocus to develop a pre-processor to handle the HP COBOL II macros, CASE selected HP-UX on PA-RISC as its target platform.
That's when the cost of their COBOL choice tossed up a roadblock.
In addition to evaluating AcuCOBOL and the MicroFocus'product, CASE had also looked at Fujitsu's COBOL. "We were going to completely re-write the data maintenance user interface screens in MicroFocus," Gilligan said, "and had prototyped some browser-based forms representing some of the more complex screens.
"When we finally had selected the platform, we did research on run-time licenses for MicroFocus COBOL. Their prices, in my opinion, were far too high, especially since we had been spoiled by free HP COBOL run-times on MPE.
"We were not looking forward to breaking the news to our customers about the fees they were likely to have to pay. That forced us to re-evaluate the three COBOL tool vendors."
Eighteen months had passed by now, and Gilligan reported that the time had delivered two important changes. "The first was that the AcuCOBOL tools had matured — many early defects had been fixed," he said. "The second was that ScreenJet had a VPlus to AcuCOBOL/AcuBench convertor. This makes migration much faster with less re-writing and re-testing than the other two COBOL vendors. For those reasons, we selected AcuCOBOL and abandoned our investment in MicroFocus tools."
Meanwhile, CASE's choice of a database had gotten even more useful to the project. Eloquence beat out IBM's DB2 database, Gilligan said. "DB2 would have required a complete re-write of the database access portion of our application to convert to SQL-based access, or some sort of TurboIMAGE emulation layer on top of it," he said. "It was not going to be a good use of resources to migrate to DB2."
"Performance is excellent with Eloquence on HP-UX on Itanium, and little time was spent re-writing application code. What changes were made were usually done to take advantage of Eloquence-specific features, such as read-only locks."
Gilligan said that CASE broke down the advantages it gained by moving from IMAGE/SQL to Eloquence:
• Eloquence is less expensive than IMAGE/SQL
• No need to repurchase Eloquence licenses when replacing an existing server with a new server vs. IMAGE/SQL, where a new license was always required (future replacement and growth savings vs. IMAGE/SQL)
• There is a minor server license transfer fee for Eloquence, but no additional charge for platform changes (such as from HP-UX to Linux or Windows)
• Eloquence licenses for additional servers are less expensive than IMAGE/SQL for additional test/development/disaster recovery systems (current and future savings vs. IMAGE/SQL)
• Eloquence will include database shadowing (replaces NetBase Shadowing) at no additional cost to purchase or support (current and future savings vs. IMAGE/SQL)
• Lower annual support costs for Eloquence vs. bundled IMAGE/SQL
The advantages illustrated a truism about selecting software: smaller providers tend to be more flexible than larger software vendors. While Gilligan had praise for both MicroFocus and Acucorp technical staffs, Acucorp could provide a better business environment for CASE's licensing needs.
"From a business relationship point of view, we were much more impressed with the team from Acucorp," he said. "Their run-time prices were better, they were easier to work with, they were willing to meet with us to understand licensing issues and develop new pricing models for hot disaster recovery systems."
Challenges, and meeting them
CASE ran up against challenges both technical and business-related. The first major challenge was moving away from a system where all reports and processing functions were selected from VPlus-based forms and run as MPE batch jobs — which produced HP PCL5-formatted reports solely for HP LaserJet printers. ABLE was using the MPE spooler and network printing features.
The new reporting and processing function selection in the ABLE application suite is completely Web browser-based. "It is much friendlier," Gilligan said. "For example, it has pull-downs which are populated with only the data a user is allowed to access."
All reports are now produced in PDF format, in addition to the HTML format ABLE has supported for a subset since 1997. They can now be saved, e-mailed and faxed, "and not just to LaserJet printers attached to the network."
Gilligan said the migration has been a very large undertaking. "One goal we had was that, where possible, if we had to re-engineer a major part of our application, we also would provide an improved user experience by improving the usability of our migrated application."
The other major challenge was that CASE had embarked on a five-year project which had to parallel major enhancements to our existing application on MPE. More on that aspect tomorrow.
February 01, 2006
Banking on diligent change in migration
Few HP 3000 customers could be more careful than banks and lenders. The nature of their business is probably more regulated than any industry. These institutions aren't known as risk takers, especially in their IT choices.
So that seems like a set-up for a calamity when a vendor of enterprise systems like the 3000 cuts off the future of the line. But reports from CASE Software show what the right tools and patient testing will deliver to the migrating customer. Like any migrating set of customers, these banking firms want a minimum of costly change.
Rick Gilligan is the Senior Software Specialist at Computer and Software Enterprises, which serves lending institutions with its Asset Based Lending Environment. The ABLE application suite ran for years, well, in an able manner, at places like Chase Business Credit, which participated in an HP 3000 success story about their installation. But HP has moved on from its 3000 futures, which means places like GE Finance, Wells Fargo and others have to shift to another platform. Gilligan and CASE have been making that happen ever since HP stepped back. The work began in earnest in 2003, when CASE announced its primary and secondary migration goals.
Windows, Linux and HP-UX were the platforms to consider in 2003. CASE also wanted to replace IMAGE databases with Eloquence, and swap out the VPlus screens with AcuBench screens. ScreenJet's tool set went to work to support the VPlus transfers. Every one of those tools needed some time to grow up into a superior solution.
The road to migration looked long in 2002, judging by Gilligan's review of the state of CASE's tool choices back then. "At that point (April 2002), the ScreenJet convertor was still in development," he said, "AcuCOBOL support for HP Macros had quite a few major defects, Eloquence didn't have audit trail and database replication features and many of our internal development tools were SPL/Splash based (having been developed during the 80's and 90's)."
But CASE was keeping in close touch with the vendors of all those solutions. "We have been working closely with Marxmeier and M.B. Foster regarding Eloquence since 2001 to clearly define the business need for enhancements to Eloquence, such as audit trail and database replication." Gilligan said. "It’s gone very smoothly with both organizations."
"We also worked closely with ScreenJet and successfully converted our on-line application maintenance forms from VPlus to ScreenJet with almost no application code changes (mostly those that could take better advantage of some AcuCOBOL Thin Client features like pop-up message boxes). That relationship has also gone smoothly."
'Toss in Acucorp and you’ve got our major partners (tool vendors) who made our migration relatively smooth. All four were responsive to our business needs (enhancements, beta testing and defect repair)."
HP was the ultimate winner in platform destination; the applications are being revised to run under HP-UX on Itanium-based servers. CASE has taken steps to make its next shift easier, though.
"We have, however, carefully chosen our tools so that we could port to Linux on IA-32/EM64T, or Linux on Itanium with a minimum of work," Gilligan reports. "We currently have no plans to support other platforms, but we did want to make our next migration easier, whenever it might be forced upon us."
Choices for database and COBOL compiler took two different journeys. One technology never varied thoughout the project, which goes "Gold" to the clients with a release this summer. The other choice had code developed and tested for it, until a licensing issue forced a change of plans. More on those topics tomorrow.