TableChopper (read ascii file)

From ObjectVision

Jump to: navigation, search

Configuration examples: TableChopper

Introduction

The TableChopper is a scriptblock using the Str StorageManager and a set of GeoDMS functions to read data from a delimited |ASCII file.

Delimited ASCII Files are often csv files. The gdal.vect StorageManager is advised to read these files, use the TableChopper for very large files or for files with other separators as comma or semicolon.

To write data to delimited ASCII Files, use the TableComposer.

To read data from non-delimited fixed length ASCII files, use the Xdb StorageManager.

Example

container TableChopper
{
   parameter<string> filename := '%projdir%/data/TableChopper.csv';
   unit<uint8>             domain : nrofrows = 5;
   parameter<string>  fieldseparator := ';';

   parameter<string>  filedata
   :  StorageType   = "str"
   ,  StorageName = "=filename";
  parameter<string> headerline := readLines(filedata, void, 0);
       
  unit<uint32> field:= Range(uint32, 0, strcount(headerline, fieldseparator) + 1)
   {
      attribute<string> name := ReadArray(headerline , field, string, 0);
   }
        
  attribute<string> bodylines (domain) := readLines(filedata, domain, headerline/ReadPos);
        
  container data := for_each_nedv(
          field/name
         ,'ReadElems(
             BodyLines
            ,string
            ,'+ MakeDefined(field/name[id(field)-1] + '/ReadPos','const(0, domain)')+'
         )'
         ,domain
         ,string
      );
}

Explanation

When the TableChopper is used as a template to read multiple files, these first three items are often used as case parameters.

  • The filedata string parameter refers to all the data from the ASCII file. It is read with the Str StorageManager.
  • The parameter headerline will read the first line from the filedata parameter.
  • The domain unit field is configured with as subitem name. This name attribute contains the names of the fields read from the header of the ASCII file.
  • The bodylines attribute will read the other (none header) lines from the filedata parameter.

The resulting data container will result in a subitem for each field. The bodylines attribute is split up in the separate values per field. The resulting items are all string attributes. Use conversion functions to convert the string values to desired values units/value types

Personal tools