Carriage wit tracks follow path to open MPE
And the Interex list winner is...

Disaster Recovery Optimization Techniques

By Gilles Schipper
GSA

While working recently with a customer on the design and implementation of disaster recovery (DR) plan for their large HP3000 system, it became apparent that the mechanics of its implementation had room for improvement.

In this specific example, the customer has a production N-class HP3000 in its primary location and a “backup” HP3000/969 system in a secondary location several hundred miles removed from the primary.

The process of implementing the DR was more manual-intensive than it needed to be.

As an aside, it was completed entirely from a remote location — thanks to the Internet, VPNs and the use of the HP Secure Web Console on the 969 – a subject I will expound upon in future.

One of the most labor-intensive aspects of the DR exercise was to rebuild the I/O configuration of the DR machine (the 969) from the full backup tape of the production N-class machine, which included an integrated system load tape (SLT) as part of the backup.

The ability to integrate the SLT on the same tape as the full backup is very convenient. It results in a simplified recovery procedure as well as the assurance that the SLT to be used will be as current as possible.

When rebuilding a system from scratch from a SLT/Backup, if the target system (in this case the 969) differs in architecture from the source system (N-4000) it is usually necessary to modify all the device paths and device configuration specifications with SYSGEN and then rebooting the system in order to even be able to utilize the tape drive of the target system to restore any files at all.

(This would be apart from the files restored during the INSTALL process - which does NOT require proper configuration of any I/O component at all).

Some would argue that this system re-configuration needs to be completed only once since any future system rebuilds would require only a “data refresh” rather than a complete system re-INSTALL.

I say that this would be true only in very stable system environments where I/O configurations – including network printer configurations – are static AND where TurboIMAGE transaction logging is not utilized.

Otherwise there could be unpleasant results and complications from using stale configurations in a real disaster recovery situation.

In any case, there really is no reason to take any chances, since the labor-intensive step of creating a proper DR target system configuration environment is achievable minus the labor-intensive part – or at least without repetition of the manual chore of re-configuring the target system each time the DR is exercised.

Unless both the production system and the DR system are architecturally similar (i.e. they belong to same HP3000 family) the configuration of the target system (the DR machine) cloned from the source system (the production machine) will be non-trivial.

At a minimum, before data restore can begin on the DR machine, the path hierarchy of the tape drive associated with the backup tape must be re-created.

Further, if the subsequent restore requires more than just the system disk, all the path components for all the disk drives must also be created.

In a real DR situation, this task can be daunting at best – particularly since ii may be difficult in that event to be able to access the appropriate documentation that describes the pertinent SYSGEN configuration requirements.

How much preferable would it be to be able to complete this configuration well in advance of the hope-to-never-happen event.

In actual fact, it is entirely possible to create an appropriate DR configuration environment that is (almost) completely integrated into one’s production environment.

SYSGEN I/O requirements

In order to provision a potential DR HP3000 system’s I/O configuration requirements into an existing production HP3000 SLT, it is only necessary to configure all of the DR path components into the existing production system’s I/O configuration.

The fact that these paths do not exist on the production (source) system is immaterial – as long as you can withstand the menacing – although perfectly innocuous – console error messages that accompany a reboot of a system so configured.

An example of these console error messages is shown later.

There is also the small matter of actual device numbers – and that is why I included the “almost” when mentioning “completely integrated” earlier.

Clearly, it is not possible to have duplicate device numbers when configuring both production and DR devices into the production SYSGEN I/O configuration.

So, in order to distinguish between the 2 systems (one the real production, the other virtual DR), I simply add 100 (you can choose any number) to the device numbers associated with the virtual machine.

Then when actually testing or invoking the DR process, it is a simple matter to change the device numbers in a batch job designed for that purpose.

Another batch job could be pre-built that would add the appropriate disk drives and volume sets to the system’s disk pool, using VOLUTIL.

These batch jobs would be included in the full backup tape and could be restored almost immediately following the INSTALL by referencing:

:file tape;dev=107 (to use my example of adding 100 to the corresponding virtual device)

:restore *tape;{fileset};directory;olddate;keep;create;show (where {fileset} corresponds to the fileset that would include the appropriate device number change and volutil batch jobs.

One could take this technique one step further in the case where the DR target machine is unknown.

In such a situation, you could create a SYSGEN I/O configuration that includes path constructs for any possible virtual machine that you could think of and include them in the host configuration – adding 100 for devices associated with virtual machine 1, 200 for virtual machine 2, and so on.

An example of a SYSGEN I/O configuration of, in this case, a model 969, integrated into the host N4000 follows below.

Note that the following example saves the changed configuration to the CONFIG.SYS group – which would be the one used upon a subsequent START NORECOVERY reboot.

You could easily keep the modified configuration to a different group – say CFGTEST.SYS - and test it with a START NORECOVERY GROUP=CFGTEST prior to subsequently saving as CONFIG.SYS once you are satisfied with the result.

:sysgen

SYSGEN version E.05.00 : catalog version E.05.00    WED, OCT 26, 2005,  9:15

AM

Copyright 1987 Hewlett-Packard Co. All Rights Reserved.

        **note** Retrieving NMMGR configuration data...

        ** First level command **

        io                log (lo)       misc (mi)        spu (sp)

        sysfile (sy)

        basegroup (ba)    keep(ke)       permyes (pe)     show (sh)

        tape (ta)

        clear (cl)(c)     exit (ex)(e)   help (he)(h)     oclose (oc)

        redo

sysgen> io

        ** IO configurator commands **

        aclass (ac)      adev (ad)       apath (ap)      avol (av)

        dclass (dc)      ddev (dd)       dpath (dp)      dvol (dv)

        lclass (lc)      ldev (ld)       lpath (lp)      lvol (lv)

        maddress(ma)     mclass (mc)     mdev (md)       mpath (mp)

        mvol (mv)        hautil (ha)

        clear (cl)(c)    exit (ex)(e)    help (he)(h)    hold (ho)

        oclose (oc)      redo

     io> lp 10

        **warning** path doesn't exist

     io> ap 10 id=a2372-60003

io> ap 10/4 id=a2372-60003

     io> ap 10/4/0 id=a2372-60003-console/lan

io> ap 10/4/4 id=hp28696a   

io> ap 10/4/4.0 id=pseudo

     io> ap 10/4/4.1 id=pseudo

     io> ap 10/4/4.2 id=pseudo

     io> ap 10/4/4.3 id=pseudo

     io> ap 10/4/4.4 id=pseudo

     io> ap 10/4/4.5 id=pseudo

     io> ap 10/4/4.6 id=pseudo

io> ap 10/4/12 id=hp28696a

     io> ap 10/4/12.0 id=pseudo

     io> ap 10/4/12.1 id=pseudo

     io> ap 10/4/12.2 id=pseudo

     io> ap 10/4/12.3 id=pseudo

     io> ap 10/4/12.4 id=pseudo

     io> ap 10/4/12.5 id=pseudo

     io> ap 10/4/12.6 id=pseudo

     io> ap 10/4/16 id=hp28642a-scsi

     io> ap 10/4/16.1 id=pseudo

     io> ap 10/4/16.2 id=pseudo

     io> ap 10/4/16.3 id=pseudo

     io> ap 10/4/16.4 id=pseudo

     io> ap 10/4/16.5 id=pseudo

     io> ap 10/4/16.6 id=pseudo

     io> ap 10/4/20 id=a2372-60003-scsi

     io> ap 10/4/20.0 id=pseudo

     io> ap 10/4/20.2 id=pseudo

     io> ap 10/4/20.3 id=pseudo

     io> ap 10/4/20.8 id=pseudo

     io> ap 10/4/20.9 id=pseudo

     io> ap 10/4/24 id=hp28696a

     io> ap 10/4/24.5 id=pseudo

     io> ap 10/4/24.6 id=pseudo

     io> ap 10/16 id=a2372-60003

io> ap 10/16/4 id=hp28696a

     io> ap 10/16/4.3 id=pseudo

     io> ap 10/16/4.4 id=pseudo

     io> ap 10/16/4.5 id=pseudo

io> ap 10/16/4.6 id=pseudo 

io> ap 10/16/12 id=hp28696a

     io> ap 10/16/12.3 id=pseudo

     io> ap 10/16/12.4 id=pseudo

     io> ap 10/16/12.5 id=pseudo

     io> ap 10/16/12.6 id=pseudo

io> ad 80 path=none id=hptcpjd

     io> ad 101 path=10/4/24.6.0 id=st34573wc

     io> ad 102 path=10/4/24.5.0 id=hvddisk

     io> ad 106 path=10/4/20.8.0 id=lp_pp_id

     io> ad 107 path=10/4/20.0.0 id=hpc1537a mode=autoreply class=tape,dds

     io> ad 108 path=10/4/20.3.0 id=hpc1557a mode=autoreply class=tape,dds,autodds

     io> ad 110 path=10/4/20.9.0 id=jobtape_id outdev=lp         

     io> ad 111 path=10/4/20.2.0 id=hpa1999a

     io> ad 120 path=10/4/0.0    id=a2372-60003-console-terminal

     io> ad 121 path=10/4/0.1    id=a2372-60003-console-terminal

     io> ad 122 path=10/4/0.3    id=hpa2994a

     io> ad 130 path=10/4/4.6.0  id=hvddisk    

     io> ad 131 path=10/4/4.5.0  id=hvddisk 

     io> ad 132 path=10/4/4.4.0  id=hvddisk

     io> ad 133 path=10/4/4.3.0  id=hvddisk                      

     io> ad 134 path=10/4/4.2.0  id=hvddisk

     io> ad 135 path=10/4/4.1.0  id=hvddisk

     io> ad 136 path=10/4/4.0.0  id=hvddisk

     io> ad 137 path=10/16/4.6.0 id=hvddisk

     io> ad 138 path=10/16/4.5.0 id=hvddisk

     io> ad 140 path=10/4/12.6.0 id=hvddisk

     io> ad 141 path=10/4/12.5.0 id=hvddisk

     io> ad 142 path=10/4/12.4.0 id=hvddisk

     io> ad 143 path=10/4/12.3.0 id=hvddisk

     io> ad 144 path=10/4/12.2.0 id=hvddisk

     io> ad 145 path=10/4/12.1.0 id=hvddisk

     io> ad 146 path=10/4/12.0.0 id=hvddisk

     io> ad 147 path=10/16/12.6.0 id=hvddisk

     io> ad 148 path=10/16/12.5.0 id=hvddisk

     io> ad 150 path=10/4/16.6.0  id=st15150n

     io> ad 151 path=10/4/16.5.0  id=st15150n

     io> ad 152 path=10/4/16.4.0  id=st15150n

     io> ad 153 path=10/4/16.3.0  id=st15150n

     io> ad 155 path=10/16/4.4.0  id=hvddsik

     io> ad 156 path=10/16/4.3.0  id=hvddisk

     io> ad 165 path=10/16/12.4.0 id=hvddisk

     io> ad 166 path=10/16/12.3.0 id=hvddisk

 

     io> hold

     io> exit

sysgen> keep

        keeping to group CONFIG.SYS

        Purge old configuration (yes/no)?y

        ** configuration files successfully saved **

sysgen> exit

:

Systems that are configured to include “virtual” devices in this fashion will be accompanied by console error messages during bootup that appear worrisome but are in fact completely safe to ignore.

Here is what they may look like:

LLIO Error - subsys: 213,  proc num: -116,  error_num: -77

for path 10/4/24.

ERROR  - LLIO device configuration error: -11

          status - subsys: #150  info: #-11

higher component configuration failed.10/4/24.5.0

higher component configuration failed.10/4/24.6.0

LLIO Error - subsys: 213,  proc num: -116,  error_num: -77

for path 10/4/4.

ERROR  - LLIO device configuration error: -11

          status - subsys: #150  info: #-11

higher component configuration failed.10/4/4.0.0

higher component configuration failed.10/4/4.1.0

higher component configuration failed.10/4/4.2.0

higher component configuration failed.10/4/4.3.0

higher component configuration failed.10/4/4.4.0

higher component configuration failed.10/4/4.5.0

higher component configuration failed.10/4/4.6.0

….

followed later, by:

Warning  - Volume is not available for mounting.  The ldev is 155

Warning  - Volume is not available for mounting.  The ldev is 138

Warning  - Volume is not available for mounting.  The ldev is 137

Warning  - Volume is not available for mounting.  The ldev is 146

Warning  - Volume is not available for mounting.  The ldev is 145

Warning  - Volume is not available for mounting.  The ldev is 144

Warning  - Volume is not available for mounting.  The ldev is 143

Warning  - Volume is not available for mounting.  The ldev is 142

Comments