Managing Artifacts

In this section we look at managing artifacts using the GUI, via VQL, and the CLI.

Finding, managing, creating and editing artifacts in the GUI

In the GUI, Artifacts are created, edited and managed on the View Artifacts screen. Here you can search and view all artifacts known to your server. The GUI also includes a feature-rich artifact editor, which you can access on this screen.

Searching Artifacts

The search bar on the right side of the Artifacts screen allows you to search by any text string, regular expression, and certain category expressions (see table below).

As explained here the name, aliases, and description fields from all artifacts are indexed and searchable.

When searching, you can also limit the results using predefined Category Filters, which are available as a drop-down list next to the search bar.

** For more information about artifacts without sources see here .

The above filter expressions can also be used in the search bar directly and combined with search strings. Searches are not case-sensitive.

For example:

• process type:client_event will show all client event artifacts that also (i.e. and is implied) have the word “process”.
• tool:sysmonbinary will show all artifacts that use the tool named “SysmonBinary”.

Searching artifacts by tag

Artifact Tags can be applied to selected artifacts to make finding and managing those artifacts easier in the GUI. These tags are stored as metadata in the server’s internal artifact repository, and are therefore not part of the artifacts’ YAML.

Tags applied to a specific artifact are displayed in the Artifact preview in the Artifacts screen.

All tags in the artifact repository are added to the end of the Search dropdown list on the Artifacts screen.

The tag term can also be used in filter expressions, for example tag:exchange defender will match all artifacts tagged with “exchange” and having the term “defender” in their searchable fields .

Importing Artifact Packs

There are many ways to add artifact definitions to Velociraptor. On the Artifacts screen you can import zip archives containing multiple artifacts.

When importing artifact packs you are given the option to add a custom prefix to the artifact name (if you want a . then you need to include it to your prefix!), and the option to only import artifacts that match a name filter.

The file structure inside the zip doesn’t matter. Velociraptor will search for all files in the zip with a .yaml or .yml file extension. The imported artifacts will be saved to the server’s datastore using the file and folder structure described here .

Importing Artifacts From Velociraptor’s Specialized Artifact Projects

Over time Velociraptor has spawned many sub-projects for curating and managing certain larger, more complex artifacts. As some artifacts became more complex and powerful, we moved them into separate projects so that they could be developed and managed independently of the main Velociraptor project. Splitting these off allows for independent release cycles, thus facilitating more rapid development and innovation.

Velociraptor includes several SERVER artifacts which can import additional artifacts from these external projects.

We also have a built-in server artifact which updates selected artifacts to their latest version, and another which imports all artifacts from a previous releases.

Server.Import.UpdatedBuiltin

The purpose of the Server.Import.UpdatedBuiltin artifact is to update either of the following 2 artifacts which may be updated between releases:

• Windows.KapeFiles.Targets
• Generic.Forensic.SQLiteHunter

By default, this artifact will add the prefix Updated. to the imported artifact.

Server.Import.PreviousReleases

The Server.Import.PreviousReleases artifact allows importing the artifacts from older releases. This is to cater for situations where clients cannot be easily upgraded, yet need functionality only found in the latest artifacts.

Creating and Editing Artifacts

In the Artifacts page you can create a new artifact by clicking the Add Artifact () button in the artifacts toolbar.

This brings up the artifact editor, populated with a very basic boilerplate artifact to help get you started.

When you select an existing built-in artifact in the GUI, and then attempt to edit it, you are actually creating a custom copy of that artifact.

• If the artifact is a compiled-in artifact then editing it will result in a copy being created with the Custom. prefix added to the name.

• If the artifact is a built-in artifact then editing it will result in a copy but you’ll need to ensure that you choose a new name for it.

• If the artifact is custom (that is, not built-in) then you can just edit it and save it, which will overwrite the previous version.

If you try to use the same name as an existing built-in artifact (or an existing alias) then you will receive an error message and the artifact will refuse to be saved.

You cannot delete built-in artifacts through the GUI either; the delete button is grayed out when they are selected.

Editor Preferences and VQL Reformatting

The editor provides syntax highlighting for YAML and VQL. If you don’t like the default colors, you can choose a different theme in the editor’s preferences pane. These preferences are user-specific, that is, they are saved as preferences for your Velociraptor user.

There are many things that you can change in the editor’s settings but the most common items to tweak are the theme and font size.

The editor preferences you set here will also be used in other places in the GUI, for example notebooks which use the same editor component.

VQL doesn’t care about formatting, but for us humans it’s nice to have VQL code that is neat and legible. The Reformat VQL button will reformat the code in all VQL blocks. In general it wraps and aligns the VQL so that it can be easily read.

Note that this action applies to all VQL blocks in the artifact. If you reformat the VQL and don’t like the result then you can use the Crtl+z keyboard shortcut to revert the changes. You can also copy a section of reformatted VQL that you do like, revert the changes, and then paste that section over the original.

Keyboard Shortcuts

The artifact editor is based on the Ace code editor and therefore supports it’s extensive list of keyboard shortcuts, shown here: https://ace.c9.io/demo/keyboard_shortcuts.html

Many of these keyboard shortcuts are the same as in other code editors that you may already be familiar with.

Autosuggest/Autocomplete

The artifact editor offers suggestions and completions, as you type, for VQL keywords, functions and plugins, as well as their arguments.

The suggestions also include previously defined strings in the artifact such as parameter names and variable names. This helps you to avoid mistakes when typing the names of previously defined parameters and variables in your VQL, since you can simply select them from the suggestions list.

1. When you type ?, or start typing a word, the editor presents a dropdown list of suggestions.

2. Use your <Up> and <Down> arrow keys to select an item from the suggestions list, and <Enter> to complete it. You can also use your mouse to hover over the suggestions list and select items, but using only your keyboard is much faster.

3. For each item in the suggestions list it also includes the type of each item being suggested (keyword, function, plugin, local string).

4. For VQL functions and plugins the suggestions list also provides a preview of the help documentation for the function/plugins as you step through them (or hover over them with your mouse).

Deleting Artifacts

All artifacts created or imported during runtime, are created in the server’s datastore and can therefore be deleted.

This can be done:

• from the toolbar in the Artifacts screen, or
• via VQL (in a notebook or artifact) using the artifact_delete function, or
• if you wish to perform bulk deletion you can use the built-in utility artifact Server.Import.DeleteArtifacts.

Artifacts loaded from these external sources are deemed “built-in” and cannot be deleted during runtime:

• embedded in the config’s autoexec.artifact_definitions section
• a directory specified by the Frontend.artifact_definitions_directory config setting
• additional directories specified by the defaults.artifact_definitions_directories config setting
• a directory specified by the --definitions CLI flag

If you need to delete such artifacts then you should manually remove the corresponding YAML documents from their source locations, and then restart the server.

For a more extensive discussion of built-in artifacts please see Built-in vs. Compiled-in vs. Custom Artifacts .

Working with the internal artifact repository in VQL

The Velociraptor server maintains an internal artifact repository that includes both built-in and custom artifacts.

On server startup, all compiled-in artifacts are loaded from the binary and additional artifacts are loaded from several possible locations . Artifacts loaded from these locations are deemed “built-in” and cannot be modified or deleted during runtime. Only custom artifacts loaded from the datastore can be modified or deleted.

Most often artifacts are viewed and managed in the GUI’s “View Artifact” screen , however artifacts can also be managed from VQL using the plugins and functions described in this section. Since the artifacts repository is maintained on the server, these VQL functions and plugins can only be used on the server - that is, in server artifacts or notebooks.

Accessing artifacts in other orgs

The artifact-related VQL queries described in this section apply only to the org that your user is currently working in.

As explained here , artifacts from the root org are propagated to non-root orgs, while custom artifacts created in non-root orgs are only visible within that org, and can potentially “mask” custom artifacts that are inherited from the root org.

If you wish to work with artifacts in a different org without switching to that org, then you can wrap your queries in the query plugin which allows you to target a different org using its org_id argument and optionally specify a different user using its runas argument.

For example:

LET OtherOrgId <= "OCDC0"
LET OtherUser <= "fred"
SELECT name
FROM query(query={ SELECT * FROM artifact_definitions()
                   WHERE built_in = FALSE },
           org_id=OtherOrgId,
           runas=OtherUser)

Viewing and Listing Artifacts

You can view all known artifacts in the current org using the artifact_definitions plugin. This plugin returns artifacts in their parsed form, which means you can easily filter the results using WHERE clauses against the many available artifact fields.

For example, with this query we can see all SERVER type artifacts and also see whether or not they are considered built-in :

SELECT name, description, built_in, compiled_in
FROM artifact_definitions() WHERE type =~ "server"

If an artifact is dependent on other artifacts then you can include those in your results by using the deps=TRUE arg.

For example:

SELECT * FROM artifact_definitions(names="Windows.Sysinternals.Autoruns", deps=TRUE)

will also show the Generic.Utils.FetchBinary artifact.

Artifacts are available in their original YAML form in the raw field, for example:

LET ColumnTypes<=dict(`raw`='nobreak')
SELECT name, raw FROM artifact_definitions(names="Windows.Sysinternals.Autoruns")

Creating and Modifying Custom Artifacts

The artifact_set function is used to create a custom artifact in the artifact repository.

While the artifact YAML can be read from a file using the read_file function, it is preferable to add artifacts from the filesystem using one of the supported loading mechanisms . These allow you to add artifacts in bulk from specified locations.

It is therefore more common to see the artifact_set function used with artifacts added from network locations, typically using the http_client plugin. For example:

SELECT artifact_set(prefix="Custom.", definition=Content) FROM http_client(url=...

It is also commonly used to add multiple artifacts from a zip file that has been downloaded to the server using http_client. The artifact Server.Import.ArtifactExchange provides a good example utilizing this approach.

When you add custom artifacts using artifact_set they are written to the server’s datastore. See How artifact names translate to file and folder structure on disk for more information.

If a custom artifact with the same name exists in the datastore then artifact_set will overwrite it without warning. This allows you to update custom artifacts via VQL.

Artifact Metadata

Artifacts in the repository have a metadata field, which currently supports three special attributes that provide additional artifact management features for server administrators: hidden, basic and tags.

The hidden attribute controls the artifact’s visibility in the GUI. This is explained in more detail here .

The basic attribute is used to designate artifacts that can be collected by users who have only the COLLECT_BASIC (Collect Basic Client) permission. Such low-privilege users can only collect these specially designated artifacts. This is explained in more detail here .

The tags attribute allows grouping artifacts into distinct groups, independent of their names. In previous versions we used an artifact name prefix to distinguish and group artifacts. This is useful for broad artifacts like Windows, Linux etc. but proved to be problematic for imported artifacts such as those from the community-contributed Artifact Exchange , particularly when other artifacts need to refer to an artifact by it’s original name.

The artifacts screen supports searching for tags .

The metadata attributes are set for new artifacts using the artifact_set function, or can be updated for existing artifacts using the artifact_set_metadata function.

For example:

SELECT name, metadata AS OldMetadata, artifact_set_metadata(name=name, hidden=TRUE)
FROM artifact_definitions() WHERE type =~ "internal" OR name =~ "internal"

will hide all “internal” type artifacts from the GUI.

The hidden artifacts are of course still there, and you can verify this by querying the metadata across all artifacts:

SELECT name, metadata FROM artifact_definitions() WHERE metadata.hidden

By default all artifacts, including those created during runtime, have the hidden and basic attributes set to false That is, they are not hidden and not “basic” by default. New artifacts have no tags set by default.

Artifact tags can be cleared by using and empty list, for example artifact_set_metadata(name=...,tags=[]).

Note that when applying a single tag to artifacts, you must ensure that you include a trailing comma so that VQL doesn’t interpret the list as a string, for example to apply the single tag “Triage” you would specify: artifact_set_metadata(name=...,tags=["Triage",]).

Deleting Artifacts

The artifact_delete function is used to delete artifacts. Only custom artifacts in the server’s datastore can be deleted.

Artifacts loaded during startup, from locations other than the artifact_definitions folder in the server’s datastore, are considered “built-in” and cannot be deleted. If you need to delete such artifacts then you should manually remove the corresponding YAML documents from their source locations, and then restart the server.

Managing and collecting artifacts from the CLI

Artifacts can be managed and used on the command line, using the artifacts CLI command group. There may be circumstances where you don’t have access to the GUI, or don’t want to or need to use it, such as when performing a local investigation on a machine.

When using the CLI, you can also make custom artifacts available by pointing the binary to a folder containing their definitions using the --definitions flag. This will load the custom artifacts and allow the artifacts commands to include them as well.

If you want these commands work with your server’s artifact repository (as defined in your server config), they will need access to the server’s datastore, and therefore need to read the server.config.yaml in order to find the datastore. This means that these commands will need to be run with the --config (or -c) flag.

If you do not specify --config then you will be working with the built-in artifact repository only - that is, the default artifacts compiled into the binary and possibly also custom artifacts if you are using a binary that has an embedded config (such as an offline collector).

[ artifacts list ]

artifacts list [<flags>] [<regex>]
    Print all artifacts

    -d, --[no-]details ...  Show more details (Use -d -dd for even more)

Args:
  [<regex>]  Regex of names to match.

You can use the artifacts list command to see the full list of artifacts that Velociraptor knows about. This is also a helpful command if you don’t know, or have maybe forgotten, the exact name of an artifact when using the artifacts collect command.

This command is very similar to running a query in a notebook using data from the artifact_definitions plugin.

The regex expression argument is case-sensitive. To make it case-insensitive prefix your expression with the (?i) modifier.

Examples

$ velociraptor artifacts list "(?i).+services"
Linux.Sys.Services
Windows.System.CriticalServices
Windows.System.Services

By default the command does not show artifact details. The -d flag will cause the artifact content to be displayed. This is very similar to using the artifacts show command, except that it supports regex.

$ velociraptor artifacts list Windows.System.Services -d
name: Windows.System.Services
description: |
  List Service details.

parameters:
  - name: servicesKeyGlob
    default: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\
    ...

The -dd flag will display the normal artifact content plus it’s compiled VQL form.

[ artifacts show ]

artifacts show <name>
    Show an artifact

Args:
  <name>  Name to show.

The purpose of the artifacts show command is to view the contents of a specific Velociraptor artifact. This allows you to inspect the definition of an artifact, including its VQL query and parameters, directly from the command line.

Wildcards and regex are not supported. The command expects an exact artifact name.

Example

$ velociraptor artifacts show Windows.System.Services
name: Windows.System.Services
description: |
  List Service details.

parameters:
  - name: servicesKeyGlob
    default: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\
    ...

[ artifacts collect ]

artifacts collect [<flags>] <artifact_name>...
    Collect all artifacts

    --output=""           When specified we create a zip file and store all output in it.
    --timeout=0           Time collection out after this many seconds.
    --progress_timeout=0  If specified we terminate the colleciton if no progress is made in this many seconds.
    --cpu_limit=0         A number between 0 to 100 representing maximum CPU utilization.
    --output_level=5      Compression level for zip output.
    --[no-]require_admin  Ensure the user is an admin
    --password=""         When specified we encrypt zip file with this password.
    --format=json         Output format to use (text,json,csv,jsonl).
    --args=ARGS ...       Artifact args (e.g. --args Foo=Bar).
    --hard_memory_limit=HARD_MEMORY_LIMIT
                          If we reach this memory limit in bytes we exit.

Args:
  <artifact_name>  The artifact name to collect.

The artifacts collect command allows you to run Velociraptor artifacts from the command line, each of which can contain one or more packaged VQL queries. This is often used for artifact testing, automated collections via scripts, or for “server-less” triage operations, potentially from a disk image.

This is essentially a wrapper around the VQL collect() plugin , with the CLI arguments being passed as arguments to this plugin.

SELECT *
FROM collect(artifacts=Artifacts,
             output=Output,
             level=Level,
             timeout=Timeout,
             progress_timeout=ProgressTimeout,
             cpu_limit=CpuLimit,
             password=Password,
             args=Args,
             format=Format)

Basic Syntax

The general syntax for the command is velociraptor artifacts collect ArtifactName.

You need to specify the exact name of the artifact you want to collect - wildcards are not supported.

Specifying Parameters/Arguments

Many artifacts accept parameters to customize their collection. You can pass these using the --args flag, followed by the parameter name and value in the format ParameterName=Value, for example:

velociraptor artifacts collect ArtifactName --args Parameter1=Value.

For artifacts requiring multiple parameters, you can repeat the --args flag:

velociraptor artifacts collect ArtifactName --args param=value --args param2=value2

Handling shell escaping for arguments can be tricky from the command line, especially with the Windows command prompt, although using PowerShell might make it easier. Arguments are always passed as strings.

Saving Output

By default, collecting an artifact using artifacts collect without specifying an output means the output from all sources within the artifact will be emitted together on stdout. It is possible to pipe the stdout to a file (using >), however it is often better to specify an output file using the --output flag. Using --output will create a zip file containing the collected data. For example, outputting to a zip:

velociraptor artifacts collect MacOS.System.QuarantineEvents --output /tmp/output.zip

The output zip file is a standardized Velociraptor collection container which can be imported to a server, or mounted using the fuse container command.

Collecting Multiple Artifacts

Generally, artifacts collect is used to collect one artifact at a time. Collecting multiple artifacts is possible but not recommended due to the complexity and potential pitfalls of specifying parameters for multiple artifacts with potentially conflicting names. The recommended way to collect multiple artifacts in a single operation is by building an offline collector.

Collecting Custom Artifacts

You can use artifacts collect to collect custom artifacts. If your custom artifact definition is in a YAML file that is not in the default artifact definition paths, then you can use the --definitions flag pointing to the directory containing your custom YAML file. When collecting, you should use the name declared inside the artifact’s YAML definition, not the filename itself.

For example: velociraptor --definitions=. artifacts collect Custom.MyTestQuery

Using with an Offline Collector binary

If you have a pre-built offline collector binary, it is essentially a Velociraptor binary with an embedded configuration. You can still use this binary to collect additional artifacts via the command line using artifacts collect, just as you would with a regular Velociraptor binary. If you provide command line arguments, the binary will execute the command you requested instead of running its default embedded configuration.

If your offline collector binary has embedded custom artifacts then these are also available to the artifacts collect command.

[ artifacts reformat ]

artifacts reformat <paths>...
    Reformat a set of artifacts

Args:
  <paths>  Paths to artifact yaml files

Reads one of more artifact definitions as YAML files, and reformats the VQL sections in them using Velociraptor’s internal VQL formatter.

This command is mostly intended for automated build environments and CI pipelines, where consistent formatting across many artifacts is required. The reformatting ensures that VQL queries are consistently formatted, making them easier to read, maintain, and debug.

The reformatting will generally validate the artifact’s YAML structure and VQL syntax, however it does not do extensive checks. For more thorough artifact checking please use the artifacts verify command described below.

This command requires a server config.

Example

velociraptor --config server.config.yaml artifacts reformat *.yaml -v

In the output you will see Reformatted <artifact_name>.yaml: OK for each matched artifact.

[ artifacts verify ]

artifacts verify [<flags>] <paths>...
    Verify a set of artifacts

    --[no-]builtin  Allow overriding of built in artifacts

Args:
  <paths>  Paths to artifact yaml files

Performs static analysis on artifacts. Its purpose is to flag issues within artifact definitions, such as invalid arguments. The command will not update or fix the target artifacts, but will report any issues found in them.

This command is mostly intended for automated build environments, for CI (Continuous Integration) testing and review pipelines. It allows for the programmatic checking or “linting” of artifact definitions, including validation of VQL syntax, on the command line. While the Graphical User Interface (GUI) editor also performs linting, the artifacts verify command provides a way to do this outside the GUI environment.

With failing artifacts the command sets errorlevel=1, which can be acted on by automation scripts.

The same static analysis can be done via VQL using the verify function.

Example

velociraptor artifacts verify ./**/*.yaml -v