Y2K
We have developed a method of maintaining Y2K compliant copies of old date fields on a record, using a trigger program. We first heard of this approach in the Summer of 1997, when a well known magazine, dedicated to IBM AS/400 and S/36 subjects, published an article describing it. We expanded on their idea by replacing the hard coded file layout, with a call to the AS/400 file field API. This allows our programs to work with any AS/400 file, without changes.

Y2K - The basic idea:
1. Leave your existing date fields as they are and add copies of them to the end of the record which include the extra digits needed to hold all four digits of the year.
2. Use a trigger program to maintain the new fields every time a record is added or updated.
This approach has some obvious advantages and disadvantages.

Advantages:
  • No need to change any display or printer files. You are probably tired of chopping an extra couple of characters off description fields to squeeze an extra piece of information onto displays and printouts. The prospect of expanding all date fields by two characters on already over-crowded layouts is not appealing.
  • Using a trigger program leaves the system in charge of when to call the routine and avoids adding code to existing programs.

  • Disadvantages:
  • Bad database design. Duplicating information is a flagrant breach of the "Normalisation" rules for good database design. Aside from wasting disk space, it opens up the possibility of inconsistent values in the fields. While using a trigger program helps to reduce the danger, this should probably be regarded as a temporary solution to cope with an emergency. In the future you can gradually drop the old fields and only use the new Y2K compliant fields.
  • Performance. Calling a new program for every update or add of every record with date fields in your system, will inevitably downgrade its performance. While testing interactive programs, that update only a few records at a time, we notice no performance impact. Long batch jobs, that update a lot of records with a lot of date fields are significantly slower. We will do some more testing and change the programs to improve performance, but, to use them you will probably need to buy more primary memory and maybe a faster processor.
  • Having to choose between writing one trigger program per file, or one for the whole system. When we first tried the trigger approach we decided to try to group the files into ones that would be most likely to be updated in the same job. This avoided the prospect of a single gigantic program, with a 10km long compile listing, that allocated space for every file layout in the system. It also avoided the maintenance headaches, and potential performance impact involved, in having a separate version of the program for every file.

  • Then we came up with a better idea:
    Using the File Field List API, a single trigger program can maintain Y2K date fields, for any file on your system.

    This approach, as long as you accept some basic assumptions, leads to a program that can be used on any file in any AS/400 system. If you think this sounds too good to be true, you are right. The problem is those assumptions. You will probably find some of them cannot be made with your system, and so you will need to make some little changes to some of the programs.

    The assumptions.
  • All legacy dates are in YYMMDD, YYDDD, YYMM or YY format. The latest version of the programs adds support for DDMMYY or MMDDYY. You have to remove some asterisks to activate the format you want and disable YYMMDD format. If you have other format it should be pretty easy to modify the little date conversion program to cope with any formats used by your system.
  • The new Y2K date fields are signed numeric fields in YYYYMMDD, YYYYDD, YYYYMM or YYYY format. The latest version of our programs allows *ISO date format fields. Using the AS/400's native date format has many advantages, such as ensuring that the system only accepts valid dates into the fields, makes them recognisable to other systems like PC's running ODBC, etc. If some of your date fields can contain zero or blank today, using native date format fields may cause problems, as they must contain a valid date or null. The next version of our programs will include support for Nulls, but today we use the system default date 0001-01-01 as the equivalent of zero or blank.
  • The new date fields are grouped together, near the end of the record. This should give a very slight improvement in performance. Defining them in the same order as the legacy date fields are defined in the file, should help too, but is not necessary. If you don't like this assumption it would be very easy to change the program that calls the API to continue searching right through the record for Y2K date fields.
  • The new date field names start with the characters "Y2". Depending on your field naming standards, (or lack thereof), this could lead to big problems.
  • If you have any other fields in your database starting with these characters, you should either rename them first, or use another method of making the fields recognisable for the API program.
  • Unless your field names are less than 4 characters long, you either have to:
  • 1. Convert any program using the file to RPG IV, which can cope with field names longer than six characters.
    2. Add I-specs to each program that uses the file to rename the field in the program. This means you have to change output files to input with add, and do a dummy read at some point in the code.
    3. Change the file to LVLCHK(*NO).
    4. Find another method of identifying the fields.

    The first customer to use our programs was initially enthusiastic when we recommended the first approach.
    We assured him that the IBM supplied CL command to convert from RPG III to RPG IV is very fast, easy to use and seemed to work every time.
    He tried it out on his own, and discovered that it did not work at all.
    By the time our consultant came back he had decided. We would use methods two and three. All his files were already LVLCHK(*NO), and, if it could speed up the project, then we should forget about the plan to change this.
    We tried explaining that the 2K error messages in his compilation listing were only caused by his RPG III copy members.
    We had already agreed that as part of the project, we would try to throw out all his copy members, replacing them with externally described files and sub-programs.
    We even suggested keeping some of his old S36 copy members.
    We said we could convert them to RPG IV and place them higher in the library list so the compiler could find them, but, it was too late. The customer is always right.

    Approach four is quite a reasonable alternative to approach one.
    If you look at the output from the File Field List API there are lots of pieces of information associated with every field.
    You could use any one of these to clearly identify a Y2K date field, and, unambiguously associate it with the equivalent legacy date field.
    I prefer using a field name, or perhaps the field text, especially if you are going for the native date format, as these are the things programmers and database managers notice first.
    A possible alternative approach would be to start all Y2K field texts with "Y2 version of XXXXXX xxxxxxxxxxxxxx". Where XXXXXX is the original field name and xxxxxx is the beginning of the original field text.


    The Programs Described by the consultant who developed them:

    The date conversion program
    * Input parameters:
    * Date
    * Date Type:
    *
    *
    *
    *
    *

    (Left adjusted)
    2 = YY
    4 = YYMM
    5 = YYDDD
    6 = YYMMDD
    D = DDMMYY
    M = MMDDYY


    Return:


    YYYY
    YYYYMM
    YYYYDDD
    YYYYMMDD
    YYYYMMDD
    YYYYMMDD


    * Last Record indicator.


    If = "1" the program ends.
    * Output parameters:
    * Date (Left adjusted) See above for format.
    * The program also returns a Status field:



    * 0 = Input date is blank or zeros
    * 1 = Valid date
    * 3 = Invalid date
    * 9 = Invalid date type


    The RPG III version of the date conversion program.
    This does not perform as good date validation as the RPG LE version, but is about three times as fast!
    We recommend against using it, but, if you have long batch jobs, that update files with lots of dates, you may have no choice.
    In April 1998, we ran a lot of performance tests.

      Click here to see the detailed figures.

    The tests were run on a CISC machine with V3R2 of OS/400. As soon as we run similar tests on a RISC machine we will publish the results here.

    The API program.
    Loading the file layout is done by calling a program that calls the AS/400 API for file field lists. I placed the code to call the API, and return the date field layout, in a separate sub-program.
    While this simplified programming and testing, it is probably not good for performance. Even a tiny degradation in performance, for a program that will be called for every add and update in a whole system, could lead to an unacceptable impact when running in production. I have considered the pros and cons of a number of alternatives. For example, moving all of the API program into the trigger program as a set of sub-routines. I will try turning the API and date conversion programs into ILE sub-procedures and binding them with the trigger program. When we have tested this we will publish the results here and include the best solution we can develop in the next version of the software.
    The main loop in this program starts by reading backwards from the last field name in the list, until it finds a field starting with the characters "Y2". It then starts building up a number of arrays with information about the field name, start position and type. It exits the loop when it finds the first field name starting with anything other than "Y2". You could easily change this, if you decide it is better to have your Y2 dates spread throughout the record. If no fields are found, an error message is sent to the calling program.
    I have now adopted the technique illustrated in the Y2K trigger program, of calling the Message Handling API, to send an escape message to the calling program, for serious errors, and, to write an information message to the job log for minor diagnostics.
    Maybe News/400 can show us how to send an inquiry message to ask whether to dump, continue, or power down the system. If IBM offers users the F and S options, why shouldn't our application programs also offer them a chance to share their frustrations when they are having a bad day.
    When all the Y2 dates have been found it goes through the field list again, to find the equivalent old field name. It stores the field length, type and start position for return to the calling program.

    The trigger program.
    The main parts of this program are the GetImage and UpdatImage sub-routines, that allow the field start positions and formats to be soft coded, and, the GetLayout and ThrowOutF sub-routines, to input a new file layout and to clear out memory, when we have opened too many files and need space for a new one.
    Our original externally described data structure for the file layout, was replaced with a general DS called RecImage. I then replaced the sub-routine that held the record format dependent field names, types and lengths, with one that maintained a multiple occurrence data structure, holding a number of twelve element arrays of start positions, types, etc. for up to fifteen files, i.e. one occurrence for each file. This assumes that no file will have more than twelve duplicate Y2K date fields, and that no job will regularly update more than fifteen files at around the same time.
    These limits, especially the first, will probably have to be increased, but, it seemed better to set them too low at first, and increase them later when necessary. If the layout has already been loaded, the sub-routine just returns the occurrence. If not, or, if all occurrences are full, it tries to load the layout into an occurrence that has been used less than others. The program counts the number of times each file has been used, to help it decide which file layout to replace first. When I first realised that it counts each add as one use, but each update as two, I thought I must fix this "bug". When I thought about it a little more, I wondered if I couldn't improve on this "feature". It may be reasonable to weight update files, which might be used by many programs in an interactive job, in favour of output only, files which might be temporarily opened to add a batch of records. Are you aware of other factors in your system that might make it sensible to weight certain files more than others?

    Adding the triggers.
    The triggers are added with the following two CL commands:

    ADDPFTRG &FILE *BEFORE *INSERT TRGY2KA +
    RPLTRG(*YES) ALWREPCHG(*YES)

    ADDPFTRG &FILE *BEFORE *UPDATE TRGY2KA +
    RPLTRG(*YES) ALWREPCHG(*YES) +
    TRGUPDCND(*CHANGE)

    The file name is a variable because I have coded them into a tiny CL program that is called from another CL program that looks like this:

    CALL Y2AADDTRG FILENAME1
    CALL Y2AADDTRG FILENAME2 etc.


    As each file gets the Y2 date fields added I add the file to this program. As each file is changed it can quickly be moved to production. You don't need to wait until you have gone through all programs that use the file, and checked to see if they use date fields in keys or in greater, or less than, comparisons. (Checking for equality is no problem. If the new dates are equal then so are the old ones and vice versa.) By gradually moving changed files and programs to production, you risk small bugs bothering the users over and extended period. But, you avoid time consuming, big system tests, and putting all the bugs you miss, and mistakes you make, into production at the same time.

    Other things to think about.
    When I first read the article describing this method of using a trigger program to tackle the millennium bomb, I thought it solved 90% of the problem. I was wrong. While eliminating the need to change all display and printer files with date fields does save a lot of work, there is still a huge job left:

  • Identify every date field in your system and where it is used. Some sort of documentation aid package could be very useful for this.
  • Go through all programs that use date fields to see if they compare dates for greater or less than conditions. These have to be changed to use the new Y2K date fields, or, to first perform a conversion, possibly by calling the same conversion program used by the trigger program.
  • Find and change all programs that perform date arithmetic.
  • Identify all logical or physical files with a date in the key, and change them to use the new Y2K fields.
  • Change all programs that use these files to use the new fields.
  • Find all CL programs that use the OPNQRYF, CPYF or if you still have S34/36 sorts, the FMTDTA CL commands, and see if any of them use date fields. Change any that do.
  • Find and change all queries that use dates.
  • Test it all.
  • Solve all the problems that we have not yet thought about. There will probably be more surprises.

  • Mistakes to watch for:
  • Files that are updated with add. When adding new records you must make sure that the new Y2 fields are initialised either with *zero, or the converted value of the original date. Otherwise, a wrong date may be left in the field from a previous update. The trigger program will not accept the record and your program will crash.
  • Copying code from one program to another and forgetting to change the field name or indicator. Consider a program that accepts a date from a display file, and uses it in a chain operation to a file. You might copy in the code from another program to move the date to a parameter field, initialise other parameters, and call the date conversion program, before chaining to the file with the converted Y2K date field. If the phone rings after the copy, but before you have finished replacing the old chain statement, you could end up with the copied "not found" indictor instead, of the original indicator in the program. The changed indicator may be used for something completely unrelated to dates in the program, and the unpredictable results may not be noticed while testing changes to do with dates.
  • Renaming two different Y2K date fields to the same six-character field name. If you decide not to convert to RPG IV, and use renamed fields to get the RPG III compiler to accept programs that use files with long Y2 field names, be careful defining and copying the I-specs. Renaming two external fields to the same name will be accepted by the compiler. By the time you discover it, you may have corrupted your original data, and may have hundreds of programs to go back and change.
  • Sub-file programs using files with dates as key fields. Saving key fields as hidden fields, or even as normal displayed fields, in sub-file records, is a very common programming practice. This is probably the one case where you should change the display file.


  • Y2K Support:

    Send questions, or details of problems, to kerstin@abrasive.se. We will do our best to help you. We will publish on our web site, details of any support questions that may be of interest to other users of the software.

      Back to the iSeries & AS/400 head page