.. highlight:: xml
.. index:: Auditing DPBuddy Tasks

.. _auditing:

Auditing DPBuddy Tasks/Commands
===============================

DPBuddy automatically creates and maintans an audit/log file tracking execution of the DPBuddy tasks that make changes to DataPower configuration or to the files residing on DataPower filesystems. The audited tasks include ``import``, ``delConfig``, ``copy``, ``delete`` and ``resetDomain``. Note that this file (dpbuddy-audit.log) is separate from the  :ref:`plain text log file (dpbuddy.log) <logging>`. The plain text log file can also be used for audit purposes, especially if it is configured to capture verbose output.

By default, DPBuddy creates its audit file in JSON format. The file contains a list of events. Each event represents an execution of one of the DPBuddy's tasks listed above.

An event contains the following data elements:

* Name of the task.

* Name of the local user who executed the task.

* Version/release info optionally supplied by the build process using ``dp.release.info`` property. This value can be used to specify the version of the DataPower artifacts that are being deployed to a domain.

* Timestamp of the event.

* DataPower URL.

* DataPower domain.

* Device/environment name.

* DataPower user name used by the task.

* List of affected (updated or deleted) DataPower configuration objects.

* List of uploaded or deleted files. For uploaded files their source location/timestamp will be captured.

All timestamps are saved in UTC timezone to be compliant with the ISO 8601 format.

Here is an example of an event generated by the :ref:`import` task:

.. code-block:: json

  {
      "@timestamp": "2017-07-04T17:57:58.833Z",
      "action": "import",
      "archive_file": {
          "last_modified": "2015-12-09T00:39:09.000Z",
          "local_path": "/Users/aananiev/myarchdev/dpbuddy/dpbuddy/samples/cli-shell/dpconfigs/XMLFirewall.xml",
          "name": "XMLFirewall.xml"
      },
      "dp": {
          "class": "Matching",
          "domain": "e2e-test",
          "object": "testFirewall"
      },
      "environment": "dev",
      "event_type": "AUDIT_CHANGE",
      "host": "dp7-local",
      "local_user": "aananiev",
      "node": "devloc",
      "release_info": "release 3.4",
      "tech": "DataPower",
      "type": "dp.audit",
      "url": "https://dp7-local:5550/service/mgmt/current",
      "user": "dpbuddy"
  }

For each event (e.g., an import command) DPBuddy creates multiple JSON records, one for each DataPower object or file. The fields are similar to the ones created by :ref:`DPBuddy Logstash plugins <logstash_main>`.

By default, the the JSON audit file is created in "${user.home}/.dpbuddy/log" directory ("/home/username/.dpbuddy/log" on Linux or "c:\\Users\\username\\.dpbuddy\\log" on Windows). The name of the file is "dpbuddy-audit.log".

DPBuddy uses `logback <http://logback.qos.ch/>`_ logging framework to write to the audit file. Logback is configured using `logback.xml <http://logback.qos.ch/manual/configuration.html>`_ and it is separate from the Ant/console logging. "logback.xml" is located in the "$DPBUDDY_HOME/conf" directory. You can change the name and location of the audit file, as well as its history/rollover policy in "logback.xml". You can also override the location of all log files by defining ``DPBUDDY_LOG_HOME`` environment variable.

Optionally, DPBuddy can create another audit file in XML format. The XML audit file contains same elements with JSON audit file, for example:

.. code-block:: xml

   <event>
       <action>import</action>
       <localUserName>Developer1</localUserName>
       <releaseInfo>release-1.0.1</releaseInfo>
       <timestamp>2014-01-17 14:34:39</timestamp>
       <dpUrl>https://dp-test:5550/service/mgmt/current</dpUrl>
       <domain>test</domain>
       <dpUserName>dpbuddy</dpUserName>
       <objects>
           <object class="HTTPUserAgent" name="default" />
           <object class="XMLManager" name="default" />
           <object class="Matching" name="testFirewall" />
           <object class="StylePolicyAction"
               name="testFirewall_request_xform_0" />
           <object class="StylePolicyAction"
               name="testFirewall_request_results" />
           <object class="StylePolicyRule" name="testFirewall_request" />
           <object class="StylePolicy" name="testFirewall" />
           <object class="XMLFirewallService" name="testFirewall" />
       </objects>
       <files>
           <file localPath="D:\dobuddy\dpconfigs\XMLFirewall.xml"
               lastModified="2014-01-17 00:14:41" />
       </files>
   </event>

The XML audit file is condifured using Ant properties, not logback.

DPBuddy keeps the XML audit file in multiple places:

* The audit file is created and maintained on the machine where DPBuddy was executed. The location of the file is defined by ``dp.audit.local.file`` property.

* A copy of the audit file is also uploaded to the DataPower domain against which a DPBuddy task was executed. This copy of the file will contain only the events relevant to this domain and the device. If ``dp.audit.domain`` property was set to "default", the default domain will contain all audit entries for this device.

All audit files in all locations are updated independently. For every audit event DPBuddy will download the file from the target domain/device, update it with the new event and then upload the file back to the domain.

By default DPBuddy uploads the audit file to the root of the ``chkpoints:`` filesystem (as opposed to ``local:``). This is done so that the audit file does get exported when the entire ``local`` filesystem is exported and does not get deleted when ``local`` is cleaned. The location can be changed using ``dp.audit.dp.file`` property.

In the current release of DPBuddy, only the XML audit file is uploaded to DataPower; JSON audit file is only maintained locally.

.. _properties_config:

Properties for Configuring XML Audit File
-----------------------------------------

Similar to other :ref:`common DPBuddy properties <common_properties>`, every audit-related property has two aliases, one using dot as a word separator (following Ant's naming conventions) and the second one using lower camel case. The second alias should be used when running DPBuddy from Groovy or Gradle (or any other scripting language) since Ant naming conventions conflict with the commonly used dot notation for accessing object properties.

.. list-table::
   :widths: 20 80 8
   :header-rows: 1

   * - Property name
     - Description
     - Required
   * - | dp.audit
       | dpAudit
     - If set to ``true``, create audit entries for auditable DPBuddy tasks.

       Defaults to ``false``.
     - No
   * - | dp.audit.local.file
       | dpAuditLocalFile
     - File name (including path) of the audit file on the machine where DPBuddy runs.

       Defaults to ``${tmpdir}/DPBuddy-audit-log.xml``. "tmpdir" is defined by the ``java.io.tmpdir`` system property of the JDK used to run DPBuddy.
     - No
   * - | dp.audit.save.to.dp
       | dpAuditSaveToDp
     - If set to ``true``, update the audit file on the device.

       Defaults to ``true``.
     - No
   * - | dp.audit.dp.file
       | dpAuditDpFile
     - File name (including path) of the audit file on the DataPower domain/device.

       Defaults to ``chkpoints:/dpbuddy-audit-log.xml``.
     - No
   * - | dp.audit.domain
       | dpAuditDomain
     - DataPower domain the audit file is saved (if ``dp.audit.save.to.dp`` is set to ``true``).

       Defaults to the target domain of an auditable task, i.e., if ``import`` runs against "TestDomain", this is where the audit file will be stored. Note that this results in each domain having its own audit file. Alternatively, if this property is set (e.g., to "default"), there will be a single audit file in the specified domain containing audit entries for all the domains of the device.
     - No
   * - | dp.release.info
       | dpReleaseInfo
     - Text to save in the ``releaseInfo`` field of the audit file. You can use this property to specify the version number of the released application and/or DataPower configuration.
     - No
   * - | dp.local.user
       | dpLocalUser
     - User ID of the user who executed the task that resulted in an audit event. If you run DPBuddy using a build server such as Jenkins, this property can be used to capture the logged-in name of the user who started the build.

       Defaults to: currently logged in user (``user.name`` system property in Java).
     - No