George Gaebler, PG&E, Read Hayward and Eric Unger, DST Controls
[This article in pdf format]

Portable digital assistants are helping with natural gas pipeline data collection at Pacific Gas and Electric (PG&E, San Francisco, Calif.), one of the largest utility companies in the free world.

PG&E pipelines transmit natural gas from source wells in Canada to the San Francisco Bay area for retail consumption. Over such long distances pressure drops due to friction losses require compressor stations, approximately every hundred miles, organized around multiple 10,000 hp, gas-fired turbines and maintained by PG&E. Typical pipeline throughput can exceed 800 million standard cubic feet per day.

To keep things safe and running, prudent compressor station management can require over 500 instrument readings per day at each station. Many of the functions monitored are so critical that problems with same, if unattended to, could truly light-up the night sky, and make the utility’s current public-relations problems look like “good publicity.” Although not “nuke-grade-dangerous,” natural gas is methane a lethally noxious gas and a high-energy explosive.

Low cost, safety go hand in hand
PG&E opted out of fully automating its data acquisition system because it would have been cost prohibitive, and more importantly, not entirely safe.

Much of PG&E’s compressor station instrumentation is just too far-flung to be hard-wired cost effectively. Furthermore, many of the thousands of gauges and meters that must be read daily, throughout the utility’s archipelago of pipeline compressor stations, are old, mechanical, or otherwise too costly to match up with transducers or hang on a local area network.

As noted, some compressor station instrumentation monitors process parameters so critical, that despite advances in (and PG&E’s use of) remote SCADA technology, PG&E also employs hand-held data acquisition to ensure that a human will always “absolutely positively” be right there to verify proper functions of life-safety systems.

The utility recently chose to use data collection devices that might have been previously dismissed by traditional plant-floor thinking as “cheap commercial-grade,” but PG&E recognized that much of this equipment is now reliable enough for many industrial situations. This is largely due to advances in semiconductor manufacturing that create more robust IC components. Additionally, advances in industrial ruggedization (shock-proofing and liquid-proofing) with low-cost molded rubber coatings and impact resistant plastic packaging offer even more protection to the increasingly tougher electronics inside.

Need for data
Critical data that PG&E records and archives at least once daily include:

  Number of starts and total hours run by each compressor
  Compressor engine oil pressure/ temperature
  Oil filter differential pressure
  Lubricating oil level
  Air intake differential pressure
  Compressor-seal oil differential pressure
  Compressor turbine fuel flow rate
  Ambient temperature and relevant weather conditions
  Fuel gas pressure
  Turbine bearing vibration readings
  Cooling water level, pressure, and temperature.

A look at the problem
In the past, use of clipboards and proprietary data collection devices kept pipeline equipment working and safe. However, these tools were inefficient, costly, and not always accurate. Clipboards just produced reams of dirty, dog-eared paper, erratically garnished with grease spots and dull-penciled scribbling. These readings then had to be deciphered, sorted, archived, and eventually transcribed via keyboard to PG&E’s database. Omissions and transcription errors haunted the system.

Proprietary industrial hand-held devices were also tried. They didn’t eliminate transcription errors and suffered other drawbacks including limited functionality and high costs to buy and maintain. Single-line displays could not be customized for easy entry of data from gauges and meters with dissimilar data formats. Finally, there was no way to produce alarms for out-of-range entries from human error or meter malfunction. The handhelds tried accepted any data no matter how preposterous.

Application challenges
PG&E engineers in charge of the Delevan compressor station, just north of Sacramento, Calif., brought in DST Controls, a control system integrator from Benicia, Calif., to find a solution. DST’s experience with personal digital assistants (PDAs) in a previous project lead it to investigate their use for PG&E’s data acquisition problem. The basic idea was to use an “open” handheld computer, in this case a Palm OS device with integrated barcode scanner from Symbol Technologies (Holtsville, N.Y.).

Indeed, even with the integrated barcode scanner, a hybrid Palm device would still be a fraction of the size and cost of the WWII walkie-talkie-sized devices previously tried by PG&E. Additionally, Palm OS was, by comparison, an “open” system, allowing almost unlimited customization of the operator interface and data handling process.

Major design problems included:

  Resolution of the classic ergonomic conundrum of more data points on fewer screens vs. less data points on more screens, that is, easy screen navigation vs. easy screen reading.

  Development of an intuitive application to minimize training time.
Ensuring that data could not accidentally be resubmitted to the database if several users independently read the same devices on the same day.

  Enabling the application to reject mistakenly entered out-of-range data.

  Enabling the PDA to automatically recognize specific instruments being polled. (One less opportunity for human error.)

  Prevention of system conflicts that could occur if a PDA is removed from its “cradle” during upload. Because data are deleted from the device after upload to ready it for the next shift, problems could arise if the deletion process were interrupted, allowing old data to be resent to the database during the next upload.

The system also had to be flexible enough in operation to allow technicians to easily change their inspection routes, or to add or delete data points while on route. In addition, the operating software would have to be “open” enough to allow transfer of data into any of several database programs, including Microsoft Access, SQL Server, Oracle, any ODBC-compliant database, and proprietary HMI databases. The handheld also had to be Class I, Div. II compliant.

Filling the gap
Because data collection was not fully automated at the compressor stations, DST Controls did not have to spend time dealing with existing esoteric data formats. The system’s database was designed from scratch around Microsoft SQL Server. Database programming was accomplished using Microsoft ADO and OLE DB tools. One of the benefits of this approach was that the application ended up being small. However, size varies with how much data PG&E wants to keep on this hand-held computer. The current application enables saving 30-day data histories in the handheld before the need to delete or upload. Screen development and programming takes place in a Visual Basic-like environment.

Because the applications (ClearControls dBeHold and ClearControls dB) DST Controls developed do not conflict with standard Palm OS applications, PG&E’s handhelds retain all basic Palm OS features.

Because upload to the PDA is via an integrated barcode scanner, each piece of monitored equipment is labeled with a unique Code 3 of 9 (or Code 39) barcode. Code 39 is easy to print and is used extensively in industrial applications. It is also the easiest to use of alphanumeric barcodes and is designed for character self-checking.

How it works
When a technician, armed with the handheld barcode scanner, approaches an instrument to poll, the following takes place:

Using the stylus, the user first calls up the Data Entry screen by tapping the “New Reading” button on the PDA. The instrument’s identification number is then entered by either scanning the meter’s barcode or manually by using “Graffiti” (Palm’s calligraphic technology) or on-screen keypad.

Once the instrument ID is entered, its description automatically appears, as does the “Last-Read” value for that device. Once the new value is entered, tapping “Enter” saves the record. If the operator makes an entry the system deems “out-of-range,” DST’s proprietary Palm application, “dBehold,” will alarm and prompt for data reentry. This then prompts a visual inspection of the meter by the technician to determine if it was a transcription error or an equipment problem. The parameters of “in range” readings for each instrument are user-configurable.

If an equipment fault is discovered, the user can flag it for maintenance using the device’s “Maint/Req” box. Mis-entered data are easily corrected manually. All screens have a “Comments” field for additional textual notes on any abnormal conditions, again via Graffiti or keyboard. At the end of the shift, the device is returned to its cradle and the “HotSync” button is pushed. This action initiates upload of the data to the station’s database PC, where PG&E’s corporate SCADA system later retrieves it.

Gabriel Acosta-Lopez, senior director of platform development services for Palm Inc., is not surprised that Palm OS devices filled PG&E’s bill. According to Mr. Acosta-Lopez, “We are seeing increased appreciation in the plant environment for tough, inexpensive, and versatile handhelds. This trend should continue as more Palm OS partners “morph” hand-held products for industrial use with custom software and hardware add-ons.”

Software makes it happen
Software components of the PG&E hand-held system include standard bundled Palm OS Desktop software, including HotSync Manager (used to upload Palm devices to a PC) and DST Controls’ proprietary ClearControls dB and ClearControls dBeHold applications.

ClearControls dB is the Windows application that runs on a PC and transfers data between the hand-held device and PG&E’s GE Fanuc Cimplicity database. It also allows supervisors to add/modify/ delete user access and information. For example, the application can be password-protected tying database modifications to predefined user privilege.

ClearControls dBeHold is the software that runs on the hand-held device (Palm OS, or WinCE/Pocket PC) for the purpose of managing entries and sending them to ClearControls dB. Use of this software can also be password protected.

Larger view

The system is being integrated into each PG&E compressor station’s data acquisition system. It will make data files available to staff engineers over PG&E’s intranet and interact with PG&E’s condition-based maintenance and work management systems.

Commercial hand-held computing systems continue to evolve rapidly, driven by the shear volume of demand in that marketplace. Because of this, more and more unnecessarily “bullet-proof” industrial “smart” devices, that are short on functionality and long on price, will be challenged by “rubber-coated” invaders that are long on functionality and very short on price.