Troubleshooting Guide

Sometimes things don’t work when you first try them. 😓

This section provides strategies and steps for troubleshooting the most common issues that people encounter when deploying and using Velociraptor.

Velociraptor development moves fast, but stability and bug fixes are a top priority for us! So before you begin troubleshooting, it’s always worth checking that you’re running the latest version, just in case your issue is one that’s already been fixed.

The current version is available from our Downloads page.

  • Deployment Issues

    • Troubleshooting problems with getting the server or clients running.

  • Operational Issues

    • Troubleshooting problems encountered during post-deployment operations.

  • VQL Issues

    • Advice for basic troubleshooting of VQL queries.

  • Debugging Velociraptor

    • Velociraptor has built-in diagnostic capabilities to help with troubleshooting a broad range of issues.

    • Internal

      • The internal profiles are built in profiles provided by the Golang ecosystem. They are mostly useful for developers but can be collected for Velociraptor as well.

      • Metrics

        • Metrics are counters in the program that are used to collect high level statistics about program execution.

        Metrics are also exported on the server using the Metrics Server. This is controlled in the configuration file Monitoring section:

        Monitoring:
 bind_address: 127.0.0.1
 bind_port: 8003
 metrics_url: http://localhost:8003/metrics

        On the server, you can collect monitoring data using curl:

      • Client

        • These profiles only exist on the client. You can see those in the debug server by adding the --debug flag (you can also add the --debug_port flag to set a different port). The client will by default serve the debug server over http://localhost:6060/.

        • Monitoring

          • The client monitoring profile shows current information on the client event monitoring subsystem.

          Clients receive a Client Event Table update from the server, instructing them on a set of CLIENT_EVENT artifacts to run. The results from these artifacts are streamed back to the server in near-realtime.

        • Global

          • Datastore

            • Profiles related to the server’s datastore.

            • Replication

              • Reports current replication connections between master and minion.

            • Replication

              • Reports current replication connections between master and minion.

          • Services

            • Velociraptor contains many service modules that help the process perform certain tasks. These usually contain specific profiles to show how they are performing.

            • ExportContainers

              • The Velociraptor GUI allows exporting collections from Flows or Hunts into a Zip file. If the collection is very large this can take some time. While the GUI shows some progress information:

            • Org

              • Profiles for services associated with each org.

              • Services

                • Broadcast

                  • Track generators installed via the generator() plugin.

                • QueueManager

                  • Report the current states of server artifact event queues.

                • VFS

                  • The VFS service post processes results from VFS operations.

                • Notifier

                  • Information about directly connected clients.