Generic.Detection.Yara.Glob

This artifact returns a list of target files then runs Yara over the target list.

There are 2 kinds of Yara rules that can be deployed:

Url link to a yara rule.
or a Standard Yara rule attached as a parameter.

Only one method of Yara will be applied and search order is as above.

The artifact leverages Glob for search so relevant filters can be applied including Glob, Size and date. Date filters will target files with a timestamp before LatestTime and after EarliestTime. The artifact also has an option to upload any files with Yara hits.

Some examples of path glob may include:

Specific binary: /usr/bin/ls
Wildcards: /var/www/*.js
More wildcards: /var/www/**/*.js
Multiple extentions: /var/www/*\.{php,aspx,js,html}
Windows: C:/Users/**/*.{exe,dll,ps1,bat}
Windows: C:\Users\**\*.{exe,dll,ps1,bat}

NOTE: this artifact runs the glob plugin with the nosymlink switch turned on. This will NOT follow any symlinks and may cause unexpected results if unknowingly targeting a folder with symlinks. If upload is selected NumberOfHits is redundant and not advised as hits are grouped by path to ensure files only downloaded once.


name: Generic.Detection.Yara.Glob
author: Matt Green - @mgreen27
description: |
  This artifact returns a list of target files then runs Yara over the target
  list.

  There are 2 kinds of Yara rules that can be deployed:

  1. Url link to a yara rule.
  2. or a Standard Yara rule attached as a parameter.

  Only one method of Yara will be applied and search order is as above.

  The artifact leverages Glob for search so relevant filters can be applied
  including Glob, Size and date. Date filters will target files with a timestamp
  before LatestTime and after EarliestTime. The artifact also has an option to
  upload any files with Yara hits.

  Some examples of path glob may include:

  * Specific binary: `/usr/bin/ls`
  * Wildcards: `/var/www/*.js`
  * More wildcards: `/var/www/**/*.js`
  * Multiple extentions: `/var/www/*\.{php,aspx,js,html}`
  * Windows: `C:/Users/**/*.{exe,dll,ps1,bat}`
  * Windows: `C:\Users\**\*.{exe,dll,ps1,bat}`

  NOTE: this artifact runs the glob plugin with the nosymlink switch turned on.
  This will NOT follow any symlinks and may cause unexpected results if
  unknowingly targeting a folder with symlinks.
  If upload is selected NumberOfHits is redundant and not advised as hits are
  grouped by path to ensure files only downloaded once.

aliases:
  - Windows.Detection.Yara.Glob
  - Linux.Detection.Yara.Glob
  - MacOS.Detection.Yara.Glob

type: CLIENT
parameters:
  - name: PathGlob
    description: Only file names that match this glob will be scanned.
    default: /usr/bin/ls
  - name: SizeMax
    description: maximum size of target file.
    type: int64
  - name: SizeMin
    description: minimum size of target file.
    type: int64
  - name: UploadHits
    type: bool
  - name: DateAfter
    type: timestamp
    description: "search for events after this date. YYYY-MM-DDTmm:hh:ssZ"
  - name: DateBefore
    type: timestamp
    description: "search for events before this date. YYYY-MM-DDTmm:hh:ssZ"
  - name: YaraUrl
    description: If configured will attempt to download Yara rules form Url
    type: upload
  - name: YaraRule
    type: yara
    description: Final Yara option and the default if no other options provided.
    default: |
        rule IsELF:TestRule {
           meta:
              author = "the internet"
              date = "2021-05-03"
              description = "A simple ELF rule to test yara features"
          condition:
             uint32(0) == 0x464c457f
        }
  - name: NumberOfHits
    description: This artifact will stop by default at one hit. This setting allows additional hits
    default: 1
    type: int
  - name: ContextBytes
    description: Include this amount of bytes around hit as context.
    default: 0
    type: int

sources:
  - query: |
      -- check which Yara to use
      LET yara_rules <= YaraUrl || YaraRule

      -- time testing
      LET time_test(stamp) =
            if(condition= DateBefore AND DateAfter,
                then= stamp < DateBefore AND stamp > DateAfter,
                else=
            if(condition=DateBefore,
                then= stamp < DateBefore,
                else=
            if(condition= DateAfter,
                then= stamp > DateAfter,
                else= True
            )))

      -- first find all matching glob
      LET files = SELECT OSPath, Name, Size, Mtime, Atime, Ctime, Btime
        FROM glob(globs=PathGlob,nosymlink='True')
        WHERE
          NOT IsDir AND NOT IsLink
          AND if(condition=SizeMin,
            then= SizeMin < Size,
            else= True)
          AND if(condition=SizeMax,
            then=SizeMax > Size,
            else= True)
          AND
             ( time_test(stamp=Mtime)
            OR time_test(stamp=Atime)
            OR time_test(stamp=Ctime)
            OR time_test(stamp=Btime))

      -- scan files and prepare hit metadata
      LET hits = SELECT * FROM foreach(row=files,
            query={
                SELECT
                    OSPath,
                    File.Size as Size,
                    Mtime, Atime, Ctime, Btime,
                    Rule, Tags, Meta,
                    String.Name as YaraString,
                    String.Offset as HitOffset,
                    upload( accessor='scope',
                            file='String.Data',
                            name=format(format="%v-%v-%v",
                            args=[
                                OSPath,
                                if(condition= String.Offset - ContextBytes < 0,
                                    then= 0,
                                    else= String.Offset - ContextBytes),
                                if(condition= String.Offset + ContextBytes > Size,
                                    then= Size,
                                    else= String.Offset + ContextBytes) ]
                            )) as HitContext
                FROM yara(rules=yara_rules,files=OSPath,
                  context=ContextBytes,number=NumberOfHits)
            })

      -- upload files if selected
      LET upload_hits = SELECT *, upload(file=OSPath,name=OSPath) as Upload FROM hits

      -- return rows
      SELECT * FROM if(condition= UploadHits,
                        then= upload_hits,
                        else= hits )

column_types:
  - name: HitContext
    type: preview_upload