Let’s go back to the intrinsic nature of the U.S. Customs Data itself.
U.S. Customs data is gathered electronically through the AMS (Automatic Manifest System), for sea, air and rail. However, only waterborne manifests are available publicly. Each daily tally contains detailed records of the tens of thousands of shipments that arrive at U.S. ports, many millions of shipments each year. Since we are a country of consumers and most imports arrive via ship, U.S. Customs Waterborne Import data represents MOST of our trade activity… to the tune of $1 trillion annually.
In spite of its inherent shortcomings, pause to appreciate the fact that detailed records of virtually every waterborne shipment, every foreign seller, every corresponding U.S. buyer, every product and component, every carrier, every port, for every day is made available publicly. The potential value contained therein is staggering. Most countries (perhaps wisely) don’t publish such information. In some countries releasing such information would be /is a capital offense.
First of all, U.S. Customs data comes as a “flat file”. It is not conveniently delimited for easy assimilation. For each and every Bill of Lading (BOL) the respective data fields have a reserved number of characters rigidly assigned; some fields are filled with interesting data, others remain completely empty. Analogous to a train hauling rail cars of varying lengths, each BOL must have its fields carefully unloaded and organized. One mishandled BOL field can wreak havoc with accurate assimilation and analysis of the data.
For instance, there is a very important single character field contained in the data string that signifies whether the proceeding data for the BOL is original and new or whether it represents a revision of an already processed BOL (from a previous day!). I have seen cases where there are several dozen revisions published to a single record occurring over a span of several months!
If not accounted for, you have multiple (and inaccurate) shipments counts. The difficulty is that the only way to adequately correct the problem is to go back into already processed and published data to completely erase and replace the previous record or retain the inaccuracy.
Another significant problem is that there are many times multiple containers for one BOL AND/OR multiple BOLs for one container (LCL –less than container loads). If not prudently accounted for there will be huge discrepancies when calculating TEUs (the standard measure for shipment volumes).
Implications? If you are evaluating whether or not to construct a new distribution center, expand port capacity, open up a freight forwarding office, evaluate economic development, perform competitive analyses on a particular U.S. buyer or foreign seller, or deconstruct and improve supply chain logistics, what you don’t know or what you think you know (but is really fictitious) can kill you.