Patch 37 is now available on our website or on CD through your dealer. The following changes have been made:

• Update: Social Security wage limits have been updated in Payroll and General Ledger.
  
  
• Update: Payroll tax tables have been updated for California, Hawaii, Louisiana, New Jersey, New York, North Dakota and Wisconsin.
  
  
• New Feature: In Accounts Receivable program 9 (Contract Pricing) you can now use the Copy option to copy a group of contract prices from one customer to a range of customers, rather than to just one.
  
  
• New Feature: Inventory program 6 (Allocations) has a new capability for viewing allocations for a specific warehouse.
  
  
• Update: Inventory program 6 (Allocations) has been updated for compatibility with the forthcoming Windows Point of Sale module.
  
  
• New Feature: The AR Customer Master 'State' field has been added to section 1 of all invoice formats for Point of Sale program 4.
  
  
• New Feature: A new security option was added, accessible in System Administrator program 4 (Security), that controls whether a login is allowed to convert layaways in Point of Sale program 4 (Full Invoicing).
  
  
• Fix: Minor screen clearing problem in Point of Sale program 51 (Estimate To Invoice).
  
  
• Fix: In Billing program 4 (Invoicing), if you edited the shipped or backordered quantity for an item on an invoice that originated as an order in Order Entry, the Order Entry order was not having its backorder quantity recalculated correctly if there were currently picked quantities outstanding for that item.
  
  
• Fix: System Administrator program 9 (Relink) was not doing as thorough a job as it could on one particular class of linked files.
  
  
• Update: added some missing fields from the Inventory Item Master to the Import Data module.

Click here to download the patch and review version changes.

How It Happens

First of all, what exactly do we mean by 'file blow-up'? Well, it occurs something like this. You're going about your business, probably doing a batch update of some sort, and notice that things seem to be taking an awfully long time to finish up. Quite possibly you see your hard drive flashing away at super-speed and your RTS-Advantage window, if not your whole machine, stops responding to you. You give it some more time but after 10 ... 20 .. or more minutes nothing seems to have changed. Finally you give up and either kill the process or turn the machine off completely.

On rebooting you find that one of your modules (the one that you were updating) keeps complaining that a file is locked. You check to see if someone else on the network is using it, but no one is. After trying various things to no avail, you go take a look at the files on your hard drive. One of the data files (the one RTS-Advantage says is locked) is huge, specifically it is right around 4.1 gigabytes.

Why that specific size? The Windows NTFS file system has a maximum file size that it can recognize, and it's no coincidence that your file is exactly that size. What has happened is that for some reason the file has been expanded to (or beyond) that size. In consequence Windows can no longer deal with the file. Anytime you try to work with it the operating system reports that it can't deal with it, and those reports are interpreted by most programs (including RTS-Advantage) as if the file were locked.

So how did it happen? It takes a combination of two things. The first requirement is that there be some trash in one of your data files. The specific bit of trash that causes this particular problem is junk in a record number field. Without getting overly technical here, many of the logical connections between our data files are determined by record numbers. Record number 'links' are used throughout our data files to associate one thing with another. For instance, each one of your Accounts Receivable customer records includes a set of record numbers that tell it where to find the customer's invoices. If you ever did any programming, the concept will be familiar to you -- they're called "linked lists".

So where does the giant data file come into the picture? For that to happen we need one more ingredient. Your system has all kinds of programs called 'drivers' running on it that take care of low-level tasks like managing the hard drive, the video display and so on. The second required ingredient is one of those drivers, one with a very odd and potentially destructive 'feature'. The one we know about was supplied only with Novell network systems, though it is possible there are others that have the same problem. This driver takes care of the low-level task of fetching and saving data to your hard drive. The oddity comes in when a program asks the driver to read some data from a file. If the request is reasonable all is well. But if the request asks for some data that is outside the bounds of the file (for instance, asking for information at position number 500,000 in a file that is only 10,000 positions in length) then this driver doesn't return an error -- instead it attempts to expand the file to a size big enough for it to read that position.

The result is obvious. If RTS-Advantage asks for some huge (junk) record number, far outside the bounds of the file, and this buggy driver is responsible for handling the request, then your file all of a sudden becomes huge, sometimes bigger than your hard drive can hold or recognize.

Fixing The File

If the expanded file is within the capacity that your computer recognizes then the solution is simple. You can run File Fixer and the file will be automatically resized to its proper setting. However, if the file has become so large that your system considers it 'locked' then File Fixer is powerless. There is no way for File Fixer to 'override' the system when it reports a file as locked.

If that is your situation, sometimes rebooting will make the system wake up and tweak the file back to being readable (though still gigantic). Often this is not the case. Sometimes running CHKDSK or the Windows File Defragmenter will tip the scales back in your favor. If any of these work, then File Fixer will be able to do the job and you'll be back in business (after you relink!). Unfortunately sometimes none of these methods works. Until our recent occurrence if none of these methods worked then we were left only with the option of restoring files from a backup.

The situation we had recently was with a user who had no backups for the last six months. Need we make the point yet again how important it is to make daily backups? In any case, going to a backup was a very distasteful prospect, so the dealer called in asking for help. After a number of false starts we came up with a method to deal with the file, and I don't mind saying that we're pretty proud of our ingenuity. We had the dealer zip up the file and send it to us. Although the file was gigantic, the zipped version was small because almost all the space was just blank. Knowing that we'd have the same problem as the user did with the file on a hard drive configured under the NTFS system, we unzipped the file onto a drive that was running FAT32. (You can determine the underlying formatting of any hard drive by going into Windows Explorer, right clicking on the drive letter and selecting Properties.) A FAT32 drive can deal with much larger file sizes, so the 4 gigabyte file was within its capacity. The process of unzipping the file produced several bogus errors telling us that the hard drive was full (it wasn't) but the file did unzip properly. Then it was simply a matter of running the RESIZE utility on that file (RESIZE is located in the UTILS directory of your RTS-Advantage distribution CD). Presumably File Fixer also would have worked, but that computer didn't have RTS-Advantage installed on it. We ran RESIZE successfully, just chopping off the bad file at a still gigantic size, then moved the files to another computer where we ran File Fixer to complete and 'fine tune' the repair. Then we ran Relink to take care of the original problem that started the whole ball rolling and sent the files back to the very relieved user.

How To Keep This From Happening In The First Place

If you never get any garbage in your files, of course you have nothing to worry about. Unfortunately this isn't a perfect world, and so it is important to run Reorganize, Relink and File Fixer regularly to make sure that your files stay squeaky clean. But even then the unexpected can still happen if you have a bad driver on your system. As we said earlier, the only buggy one we know of is an old Novell network driver. Maybe you aren't running Novell, or stopped using Novell years ago. That doesn't mean the driver isn't still lurking on your computer. If you have ever had one of these file 'blow ups' happen, chances are excellent that you do in fact have that buggy driver. The solution is to have an expert check your system thoroughly and remove these old legacy drivers.

Postscript -- Why Doesn't RTS-Advantage Keep This From Happening in the First Place?

If you've followed things so far and you think like a programmer, you are wondering why we don't proactively avoid this situation in the first place. It is absolutely true that in the software we could easily check the size of a file before we attempt each reading operation. If the read operation is to look at byte 150 and the size of the file is only 100, then just give an error before even attempting the read operation. Simple!

In fact we created a version of the software in-house that incorporated that seemingly minor tweak. It worked like a charm. Only one problem with the solution. The file size check is a system call, and not a very quick one. Incessantly checking the file size made a significant impact on the performance of the software. We found that batch processes, very read and write intensive, took an average of 60% more time than without this check. That was considered an unacceptable performance degradation considering the rarity of the problem. We feel that it is better to solve the problem at its source -- the faulty driver -- than to penalize the vast majority of users who will never have the problem.