Windows.Collectors.Remapping

name: Windows.Collectors.Remapping
description: |
  Creates filesystem and registry remappings for offline analysis of
  Velociraptor collection containers.

  The remapping allows Velociraptor to treat the collection container as a dead
  disk image in a similar way to `Generic.Utils.DeadDiskRemapping`. This means
  that you can directly analyze the files contained in the collection zip file,
  without needing to extract or import it.

  There are 2 ways to use the remapping generated by this artifact:

  1. Non-interactive command line analysis, as per the example below. \
     This approach only requires a Velociraptor binary, i.e. it  does not
     require a server or client, and then uses either the CLI `artifacts
     collect` or `query` commands.

  2. Interactive "virtual" client managed via the Velociraptor GUI (see
     reference below). \
     This approach uses a client which connects to the server. The client
     accesses the collection container as a simulated filesystem using the
     remapping that this artifact generates.

  In both cases it uses the remapping impersonation feature to impersonate the
  original endpoint's name and platform (Windows). Because of this
  impersonation the virtual client can be run on Linux while still appearing as
  Windows to artifacts and queries.

  Collection containers are typically created by offline collectors using the
  `Windows.Triage.Targets` or `Windows.KapeFiles.Targets` artifacts and contain
  files collected from the endpoint, although any artifact that "uploads" files
  as part of the collection can be used.

  Collection containers can also be created by exporting collections from the
  GUI, using the "Download Results" facility. This allows you to port
  collections across servers - perhaps to allow an independent analyst to
  examine some files and run further collections without giving them access to
  your server. If they don't have their own Velociraptor server, they can do
  serverless collections/queries as in the following example.

  ### Example - command line use

  1. Collect files using a bulk file collection artifact - For example
     `Windows.Triage.Targets` with the `_BasicCollection` target is a good
     option.

  2. Generate the remapping file:
     ```
     velociraptor artifacts collect -v Windows.Collectors.Remapping \
       --args ZipPath=/path/to/collection.zip \
       --args WriteRemappingPath=/tmp/remapping.yaml
     ```

  3. Apply the remapping file when collecting further artifacts. These
     collections will target the files in the container:
     ```
     velociraptor --remap /tmp/remapping.yaml \
       artifacts collect -v Windows.Registry.Hunter \
       --args RemappingStrategy=None
     ```
     The CLI query command can also be used for running ad-hoc queries:
     ```
     velociraptor --remap /tmp/remapping.yaml query "SELECT * FROM ... "
     ```

  ### Notes

  - Direct analysis of collection containers protected by fixed passwords or
    X.509 certificates is supported. For password-protected containers you'll
    need to provide the password explicitly as a parameter. For X.509-protected
    containers they are transparently decrypted _if they are secured with your
    servers cert and you are running this artifact in the GUI_. If you are
    running this artifact on the command line then you'll need to supply the
    server config as an additional command line argument (using the `-c` flag).
    Alternatively you can remove the protection from the container first using
    the `decrypt` CLI command. Removing the protection first is also necessary
    if the container was secured using an X.509 (or PGP) cert other than your
    server's one.

  - If you want to run `Windows.Registry.Hunter` against the data, you'll need
    to disable its own remapping config (i.e. `RemappingStrategy: none`) so that
    it doesn't interfere with the remapping created by this artifact.

  - The remapping currently doesn't support Volume Shadow Copies contained in
    the collection, however you can still access these files via VQL by
    specifying their paths (e.g.
    `GLOBALROOT\DEVICE\HARDDISKVOLUMESHADOWCOPY1\Windows\...`)
    which are located outside of any remapped drive roots.

  - VFS browsing of the collection container currently doesn't work.

reference:
  - |
    [Dead disk analysis on a collection container]
    (http://docs.velociraptor.app/docs/deployment/offline_collections/collection_data/#dead-disk-analysis-on-a-collection-container)

type: SERVER

# Check if the collection container has file uploads
precondition: |
  SELECT * FROM glob(accessor=Accessor,
                     globs="/uploads/{auto,file,ntfs}",
                     root=pathspec(DelegatePath=ZipPath,DelegateAccessor="file"))

parameters:
  - name: ZipPath
    default: /tmp/collection.zip
    description: Path to the collection zip file to inspect

  - name: Accessor
    default: collector
    type: hidden

  - name: Hostname
    description: If NOT specified we try to read it from the collection container

  - name: ZIP_PASSWORDS
    friendly_name: ZipPassword
    description: Required if the zip container is protected by a fixed password

  - name: Upload
    type: bool
    default: "Y"
    description: |
      If specified, store the generated YAML remapping file in the collection's
      File Uploads

  - name: WriteRemappingPath
    description: If specified, write the YAML remapping file to this path

  - name: CommonRemapping
    description: Common clauses for all remapping in YAML format
    default: |
      remappings:
      - type: permissions
        permissions:
        - COLLECT_CLIENT
        - FILESYSTEM_READ
        - FILESYSTEM_WRITE
        - READ_RESULTS
        - MACHINE_STATE
        - SERVER_ADMIN
        - COLLECT_SERVER
        - EXECVE
      - type: impersonation
        os: windows
        hostname: {{ .Hostname | quote }}
        env:
        - key: SystemRoot
          value: C:\Windows
        - key: WinDir
          value: C:\Windows
        disabled_functions:
        - amsi
        - lookupSID
        - token
        disabled_plugins:
        - execve
        - http_client
        - users
        - certificates
        - handles
        - pslist
        - interfaces
        - modules
        - netstat
        - partitions
        - proc_dump
        - proc_yara
        - vad
        - winobj
        - wmi
      - type: shadow
        from:
          accessor: zip
        "on":
          accessor: zip
      - type: shadow
        from:
          accessor: zip_nocase
        "on":
          accessor: zip_nocase
      - type: shadow
        from:
          accessor: collector
        "on":
          accessor: collector
      - type: shadow
        from:
          accessor: raw_reg
        "on":
          accessor: raw_reg
      - type: shadow
        from:
          accessor: data
        "on":
          accessor: data

export: |
  LET CollectionHostinfo = parse_json(
      data=read_file(accessor=Accessor,
                     filename=pathspec(DelegatePath=ZipPath,
                                       DelegateAccessor="file",
                                       path_type="zip",
                                       Path="client_info.json")))

  -- Set the hostname either from the parameters or read it from the
  -- file itself. Validate the hostname to have only allowed hostname chars.
  LET DerivedHostname = parse_string_with_regex(
    string=Hostname || CollectionHostinfo.Hostname || CollectionHostinfo.hostname,
    regex="([a-z0-9-.]+)"
  ).g1

  LET Remappings = parse_yaml(
                    filename=template(template=CommonRemapping,
                                      expansion=dict(
                                        Hostname=DerivedHostname || "Unknown")),
                    accessor="data")

  -- Normalize the drive name from device to drive. NTFS accessor uses
  -- device notation. TODO - we currently do nothing with VSS devices.
  LET NormalizeDrive(Path) = upcase(string=regex_transform(
      source=Path,
      map=dict(`\\\\\\\\.\\\\`=""), key="A"))

  -- Get all the unique drive letters so we can combine them at the same place.
  LET AllDrives = SELECT enumerate(items=OSPath) AS Paths,
           NormalizeDrive(Path=OSPath.Basename) AS Drive,
           OSPath AS OSPath
      FROM glob(
        accessor=Accessor,
        globs="/uploads/*/*",
        root=pathspec(DelegatePath=ZipPath, DelegateAccessor="file"))
      GROUP BY Drive

  LET ScopeTemplate <= if(condition=ZIP_PASSWORDS,
                          then=template(template='''
  LET OVERLAY_ACCESSOR_DELEGATES <= dict(
       accessor={{ .Accessor | quote }},
       paths={{"{{ .Paths }}"}})

  LET ZIP_PASSWORDS <= {{ .ZipPassword | quote }}
  ''', expansion=dict(ZipPassword=ZIP_PASSWORDS, Accessor=Accessor)),

       else=template(template='''
  LET OVERLAY_ACCESSOR_DELEGATES <= dict(
       accessor={{ .Accessor | quote }},
       paths={{"{{ .Paths }}"}})
  ''', expansion=dict(Accessor=Accessor))
  )

  -- Overlay the files that were fetched with all accessors into the
  -- same directory.
  LET Overlays = SELECT * FROM foreach(row=["auto", "file", "ntfs"],
   query={
      SELECT
         dict(type="mount",
              scope=template(template=ScopeTemplate,
                             expansion=dict(Paths=serialize(item=Paths))),
              `from`=dict(accessor="overlay"),
              `on`=dict(accessor=_value,
                        prefix=Drive,
                        path_type="windows")) AS Clauses
      FROM AllDrives
    })

  -- Search for the hive with this glob and add the first match into
  -- the registry path specified.
  LET _FindHive(HiveGlob, RegPath) = SELECT
        dict(type="mount",
             description=format(format="Map %v to %v",
                          args=[OSPath.Path, RegPath]),
             `from`=dict(accessor="raw_reg",
                         path_type="registry",
                         prefix=serialize(format="json",
                            item=dict(DelegateAccessor=Accessor,
                                      DelegatePath=OSPath.PathSpec.String))),
             `on`=dict(accessor="registry",
                       path_type="registry",
                       prefix=RegPath)) AS Clauses
    FROM glob(globs=HiveGlob,
              accessor=Accessor,
              root=pathspec(Path="/uploads/",
                            DelegateAccessor="file",
                            DelegatePath=ZipPath))
      LIMIT 1

  -- Look for user hives and map them in HKEY_USERS
  LET _FindDirUserHives = SELECT
        dict(type="mount",
             description=format(format="Map %v to %v",
                          args=[OSPath.Path, RegPath]),
             `from`=dict(accessor="raw_reg",
                         path_type="registry",
                         prefix=serialize(format="json",
                            item=dict(DelegateAccessor=Accessor,
                                      DelegatePath=OSPath.PathSpec.String))),
             `on`=dict(accessor="registry",
                       path_type="registry",
                       prefix="HKEY_USERS/" + OSPath[4])) AS Clauses
    FROM glob(globs="*/*/Users/*/NTUser.dat",
              accessor=Accessor,
              root=pathspec(Path="/uploads/",
                            DelegateAccessor="file",
                            DelegatePath=ZipPath))

  LET RegistryMappings = SELECT * FROM chain(
    a={
      SELECT *
      FROM _FindHive(HiveGlob="*/*/Windows/System32/config/Software",
                     RegPath="HKEY_LOCAL_MACHINE/Software")
    }, b={
      SELECT *
      FROM _FindHive(HiveGlob="*/*/Windows/System32/config/Security",
                     RegPath="HKEY_LOCAL_MACHINE/Security")
    }, c={
      SELECT *
      FROM _FindHive(HiveGlob="*/*/Windows/System32/config/System",
                     RegPath="HKEY_LOCAL_MACHINE/System")
    }, d={
      SELECT *
      FROM _FindHive(HiveGlob="*/*/Windows/System32/config/SAM",
                     RegPath="SAM")
    }, e={
      SELECT *
      FROM _FindHive(HiveGlob="*/*/Windows/appcompat/Programs/Amcache.hve",
                     RegPath="Amcache")
    }, f={
      SELECT * FROM _FindDirUserHives
    })

sources:
- name: WriteRemapping
  query: |
    LET CalculateMappings <= Remappings.remappings +
      Overlays.Clauses +
      RegistryMappings.Clauses

    LET YamlText = serialize(format="yaml",
                              item=dict(remappings=CalculateMappings))

    LET _ <= log(message="Will use hostname %v for impersonation",
                 args=DerivedHostname, level="INFO")

    SELECT CalculateMappings,
        Upload && upload(accessor="data", file=YamlText, name="remapping.yaml") AS Upload,
        WriteRemappingPath && copy(
              dest=WriteRemappingPath,
              accessor='data',
              filename=YamlText) AS RemappingFile
    FROM scope()


- name: CollectionHostInfo
  query: SELECT CollectionHostinfo FROM scope()

- name: TestRegistryAccess
  query: |
    LET _ <= remap(config=YamlText, clear=TRUE)
    SELECT OSPath
    FROM glob(globs="HKEY_LOCAL_MACHINE/Software/*", accessor='registry')

- name: TestFileAccess
  query: |
    LET _ <= remap(config=YamlText, clear=TRUE)
    SELECT OSPath
    FROM glob(globs="*/**")
    LIMIT 5