About Auto Trimming

CyberAudit-Web automatically trims old audit events and Journal of Changes entries from it's database on a periodic basis. This keeps daily backups of CyberAudit-Web smaller and helps ensure that audit reports and journal logs can be displayed in a timely manner. Neither the trimmed audit data nor the journal data can be restored to CyberAudit-Web. They can be imported into Excel or a third party database for SQL queries and reporting.

Trimmed data is written by default to the /var/caw-backups folder.

The format of the file is CSV (comma separated value). The general CSV format is documented in RFC 4180. CyberAudit-Web CSV files conform to the following rules:

• The file is Unicode text encoded as UTF-8. If text fields appear incorrectly after import be sure to specify UTF-8 when importing.
• One record per line.
• Lines are terminated using the Windows conventional CRLF (\r\n)
• Fields in a record are delimited by comma: ,
• The first line is the table header. It lists the field names for each column. Each field name is suffixed with it's type.
• Text fields are double quoted.
• Text fields will never contain newlines.
• Double quotes within a text field will be escaped by a second double quote.
• All fields can be NULL (unknown or not-applicable)
• NULL fields are left empty (even text fields). (eg ...,"before null",,"after null",...)
• Field types are:
  
  • Integer (i) Example: 12345
  • Text (t) Example: "hello world"
  • Boolean (b) 0 for false, 1 for true
  • Real (r) Example: 3.14
  • Time (m) Example: 14:37:41
  • Date (d) Example: 2012-02-14
  • Timestamp (p) Example: 2012-02-14 14:37:41
• Real numbers will be in either standard or scientific notation. (eg 3.14 or 1.230e-09)
• Dates and times are output according to ISO 8601 (yyyy-mm-dd"T"hh:mm:ss). For example: 2012-02-28T11:09:03  The time zone designator suffix is ommitted.
• Dates and times are local to the time zone in which the event occurred unless the field name has the "_utc" suffix. UTC (aka GMT) times will need to be converted to your local time zone when displaying reports.
• The records are un-ordered.

Spreadsheet Import -

Most spreadsheet programs such as Microsoft Excel can open CSV files directly. However, if the CSV file is too large, it may tax the RAM of the computer loading it. Loading it into a database such as: Microsoft Access, OpenOffice Base or MySQL can help work around RAM limitations.

Microsoft Access -

Generic instructions for importing CSV files into Microsoft Access can be found here.

Filemaker Pro -

Follow these steps:

1. Add a new table in the FileMaker database
2. Add fields to the table corresponding to the CSV file you are importing.
   Note: FileMaker may not parse ISO 8601 dates properly. The workaround is to import them as Text fields.
3. Select File->Import Records->From File...
4. Verify that the fields are aligned properly
5. Select the UTF-8 character set.
6. Check the box to ignore the first record (header).
7. Import (will be slow for millions of records).

Fields in an Audit Trail Trim File
Fields in a Trim File for Journal of Changes Data

Manage System Context Help Articles