pipe

Description

A pipe allows plugins that use files to read data from a vql query.

NOTE: this is not the same as a Windows named pipe.

This is needed to be able to use the “pipe” accessor.

Example

In the following example we create a pipe from a query which reads a log file line by line. Each line is being transformed by a regex and potentially filtered (perhaps to fix up buggy CSV implementations that generated a bad CSV).

The pipe is then fed into the parse_csv() plugin to parse each line as a csv file.

LET MyPipe = pipe(query={
    SELECT regex_replace(
      re='''^(\d{4}-\d{2}-\d{2}) (\d{2}:)''',
      replace='''${1}T${2}''', source=Line) AS Line
    FROM parse_lines(filename=IISPath)
    WHERE NOT Line =~ "^#"
  }, sep="\n")

SELECT * FROM parse_csv(
   columns=Columns, separator=" ",
   filename="MyPipe", accessor="pipe")