[Editor’s note: HP has been working on repairing IMAGE/SQL’s Large File DataSets (LFDS) since 2004. Using LFDS, the default in MPE/iX 7.5, can result in corrupt HP 3000 databases, as we have reported in the NewsWire. As of this month, HP was working with database tool vendors in testing a new fix for the problem; a 2005 attempt failed in alpha testing. Database tools vendors have documented corruption at some customer sites. Vendors recommend avoiding the use of Large File DataSets in IMAGE/SQL. Stan Sieler, executive vice president of Allegro Consultants and a 3000 veteran who led the design of major enhancements to IMAGE/3000 while working at HP, offered his insights on the problem.]
By Stan Sieler
Background: Before Large Files
In the beginning, or at least in 1994, IMAGE datasets were limited to 4 gigabytes (GB). This was because a single datatset was a single file, and MPE limited file sizes to 4 GB. Allegro Consultants, Inc., was contracted by HP to design and implement a solution to this limitation. At the time, along with other technical experts, we suggested that the solution lay in enhancing MPE/iX to support larger disk files — but we were told that that enhancement was uncertain, and certainly not likely to appear in the near future. So, we chose to implement “Jumbo datasets”, using a collection of 4 GB files to logically comprise a single dataset. Given the limitations of IMAGE filenaming, we chose to name the “chunks” with HFS names (e.g., set 23 of SALES would be SALES23, SALES23.001, SALES23.002, etc.)
Jumbo datasets arrived, and worked well — then and now.