Whats New?

New Feature in 1.11.7

With this release, we have enhanced WAS findings Settings in TA setup page:

We have added the following options:

• Checkbox for Enable the checkbox to parse result list tag. 

      For details, refer to WAS Findings Settings section.

• Also, some minor improvements in logging are made. 

New Features in 1.11.5

With this release, we have enhanced Knowledge Base Settings in TA setup page:

We have added following options:

• Checkbox for Enable multi-threading to download Knowledgebase data
• Number of threads field for multithreading
• Added Page Size Field
• Checkbox for Enable to preserve knowledgebase ids API response.
  
  For details, refer to KnowledgeBase Settings section.

New Features in 1.11.4

With this release, we have enhanced Endpoint Detection and Response Settings in TA
setup page.
We have added the following options:

• Checkbox for Enable multi-threading to download EDR events
• Number of threads field for multithreading to pull EDR events in lesser time
  • Loop for each date range to get the counts with the count API. If a date range contains events, then the start date of the data range is the checkpoint date.
  • Again divide the data range with newer checkpoint date as per provided thread for example, 18hr/10 = 1.8 hr. In this way 10 threads are running to pull the 1.8 hr data each.
• The EDR API are changed to /ioc/events/searchAfter.
  • First API call do not have any value for the searchAfter parameter.
  • For next API calls pass searchaftervalues from the response header of previous request to  searchAfterValues field of input parameter.
  • The searchAfter parameter values are from previous request's header.

For details, refer to Endpoint Detection and Response Settings section.

Checkpoint files are individually created for each thread under the edr_events_cp_folder, containing their respective date ranges. Additionally, a checkpoint file is generated for the input edr_event data. These files facilitate comparison between the checkpoint files of each thread and the main checkpoint file, enabling the selection of the latest date range to initiate the next data pull.

New Feature in 1.11.3

With the new release, we have added Host IDs field and Enable to preserve Host Asset API response checkbox under VM Detection - Advanced Settings in VM Detection Settings in TA setup page.

New Feature in 1.11.2

With the new release, we have added Host IDs field and Enable to preserve Host Asset API response checkbox under VM Detection - Advanced Settings in VM Detection Settings in TA setup page.

New Feature in 1.11.1

With the new release, we have added Host IDs field and Enable to preserve Host Asset API response checkbox under VM Detection - Advanced Settings in VM Detection Settings in TA setup page.

For more details, refer to What are VM Detection-Advanced Settings.

New Feature in 1.11.0

The Qualys Splunk TA 1.11.0 requires a minimum supported version of Splunk v8.x.

Integration of the Qualys Cyber Security Asset Management with Splunk TA

We have now integrated Qualys Cyber Security Asset Management (CSAM) with Splunk TA. You can now configure the TA app to fetch your CSAM data from your Qualys account.

CSAM fetches data continuously in the following manner:

1) Once you enable the data input, it pulls the count API to get total number of assets which is updated after asset last updated datetime.

2) Then it calls search API to pull all the assets as per asset last updated datetime and specified parameters.

We added a new Cyber Security Asset Management Settings section on the TA setup page where you can specify:

• Log User Accounts
• Log Open Ports
• Log File System Volume
• Log Network Interfaces
• Log Software
• Log Tags
• Log Hardware
• Log Operating System
• Log Business App List Data
• Exclude Unmanaged Assets
• Page size
• Extra filters for CSAM API
• CSAM Maximum API retry coun

For details, refer to Cyber Security Asset Management Settings.

We also added a new Qualys Metrics cyber_security_asset_management that you can use to create a cron job to pull your CSAM data.

Integration of the Qualys CertView with Splunk TA

We have integrated Qualys Certview with Splunk TA. You can configure the TA app to fetch your Certview data from your Qualys account.

CertView continuously fetches data. Once you enable data input, it pulls the certificates updated after the certificate updated datetime.

We added a new Certview Settings section on the TA setup page where you can specify:

• Page size
• Extra filters for CertView API
• CertView custom operation
• CertView Custom Fields
• Certview Maximum API retry count

For details, refer to Certview Settings section.

We also added a new Qualys Metrics certview_certificates that you can use to create a cron job to pull your Certview data.

Minimum supported Splunk version for Qualys Splunk TA 1.11.0 is Splunk v8.x

From this release we are not removing qualys_kb.csv file from the search/lookups automatically if user disables KB indexing option after enabling it first as per Splunk's recommendation. Users need to remove it manually. For more details, refer to What happens if you disable KB indexing after enabling it initially from FAQs.

New Feature in 1.10.15

With this release, we have added following enhancements:

Parsing Newly Added Fields for following APIs in Splunk TA

Host List API

• LAST_ACTIVITY
• LAST_BOOT
• SERIAL_NUMBER
• HARDWARE_UUID
• FIRST_FOUND_DATE
• AGENT_STATUS
• CLOUD_AGENT_RUNNING_ON

Host List Detection API

• UNIQUE_VULN_ID

  To get the UNIQUE_VULN_ID, add UNIQUE_VULN_ID in Detection fields to log under VM Detection Settings in TA setup page.

• Renaming ARS Field Names to TRURISK

  The fields ARS and ARS_FACTORS are changed to TRURISK_SCORE and TRURISK_SCORE_FACTORS respectively.

New Feature in 1.10.14

In TA setup page under the Policy Compliance Reporting Service Settings, we have removed Do you want to enable SSL Certificate Verification? checkbox. Now SSL certificate verification is enabled by default for PCRS. In a recent policy change, Splunk no longer allows disabling SSL certificates to ensure secure connections.

If there is an error with the SSL certificate for PCRS data input, ensure that your API connection is secured with SSL.

New Feature in 1.10.12

We fixed an issue where the user faced data missing in Splunk TA when making the API call. The issue of parsing blank spaces in the streaming posture data for PCRS data input has been resolved.

New Feature in 1.10.11

With the new release, we have extended our support to CIM-5.x for CS and SEM data input.

We have fixed the issue where TA encounters an error if an XML tag had a blank value for host detection input data.

New Feature in 1.10.10

We fixed the issue for the TruRisk factors parsing in the host_detection data input. It was not parsing in the previous version.

New Feature in 1.10.9

With the new release, we have added following enhancements:

Introduced Parallelism into PCRS Data Input:

- We introduced two separate multi-threads, ResolveHostThread and PostureStreamingThread, to call the Resolve Host Ids API and the Posture Streaming API.

ResolveHostThread can select a batch of policies, the batch size defined in the number of policy Ids to use for the Resolve Host Ids API field, and each thread ResolveHostThread-1, ResolveHostThread-2, ResolveHostThread-3, and so on, can make the Resolve Host Id API call.

The Resolve Host Ids can be added to the Host Id queue once the thread completes the pull. The PostureStreamingThread can immediately pick it. Each thread PostureStreamingThread-1, PostureStreamingThread-2, PostureStreamingThread-3, etc., can make the Resolve Posture Streaming API call.

As a result, the Resolve Host Id API and Posture Streaming API can pull data in parallel, and data pull can be faster compared to the previous method.

Parsing isIgnored Field for WAS Findings:

We are now parsing the isIgnored tag and ingesting it into WAS finding events.

- A few minor bug fixes are made.

New Feature in 1.10.8

With the new release, we have added below enhancements:

Parsing TruRisk Fields for Host Detection:

We are now parsing ARS, ACS and ARS_FACTORS and adding it into VM detection events. To get the ARS, ACS and ARS_FACTORS, check the ARS, ACS and ARS_FACTORS for Host Asset API checkbox provided under VM Detection Settings in TA setup page.

We are parsing QDS and QDS_FACTORS and adding it into VM detection events. To get the QDS, add 'QDS' in 'Detection fields to log' add 'show_qds=1' in the extra parameter under VM Detection. To get the QDS_FACTORS, add 'QDS_FACTORS' in 'Detection fields to log' and add 'show_qds_factors=1'.

Parsing HOSTNAME field for Host Detection:

We are now parsing HOSTNAME and adding it into VM Detection Events.

New Settings under Policy Compliance Reporting Service

We have introduced the following settings under Policy Compliance Reporting Service settings on the TA setup page:

•    Do you want to truncate the evidence?

•    Do you want to enable SSL Certificate Verification?

•    Number of threads to use for PCRS (max 10)

•    PCRS Maximum API retry count

•    PCRS Custom Policy Ids

•    PCRS custom policy operation (include/exclude)

For more details, refer to the Policy Compliance Reporting Service Settings section.

New Feature in 1.10.7

With the new release, we have added the support for Truncation Limit under Policy Compliance Reporting Service Settings in TA setup page.

For more details, refer to Policy Compliance Reporting Service Settings section.

New Feature in 1.10.6

With the new release, we have added Host Ids Batch size for Posture Info Streaming API field under Policy Compliance Reporting Service Settings in TA setup page.

For more details, refer to Policy Compliance Reporting Service Settings section.

We are parsing CLOUD_PROVIDER_TAGS and adding in the VM events in CLOUD_PROVIDER_TAGS_name1 = val1, CLOUD_PROVIDER_TAGS_name2 = val2... format.

To get the CLOUD_PROVIDER_TAGS, add 'CLOUD_PROVIDER_TAGS' in 'host fields to log' field and add 'show_cloud_tags=1' in the extra parameter under VM Detection

Settings in TA setup page.

Also, some minor improvements in logging are made.

New Feature in 1.10.5

We are now compatible with SPLUNK v9.0 and minor improvements added are as follows:

AUTHENTICATION

• Parsing ASSET_ID tag and adding in VM events:

  With the new release, we are parsing ASSET_ID tag and adding it to the VM events.

  To get the ASSET_ID, add 'show_asset_id=1' in the extra parameter under VM Detection Settings in TA setup page.

• Additional fields added to the KB CSV File are as follows:
  • DISCOVORY_REMOTE
  • LAST_SERVICE_MODIFICTION_DATETIME
  • SUPPORTED_MODULES
• Parsing IPV6 tag and adding in VM events:

  With the new release, TA parses the IPV6 tag (if its present in XML) and ingests into VM events.

  

  If the tag is not present, then IPV6 field is not present in the event ingested to Splunk. If tag is present but value is empty/null/None, then empty string is present in the event.

  

New Feature in 1.10.4

In TA v1.10.4, we included a new parameter in the VM detection API call that is 'detection_updated_since'. This parameter filtesr out the QID whose status does not change since the datetime mentioned with this parameter.

New Feature in 1.10.2

The new release comes with improvements in logging and minor enhancements in utility script.

New Feature in 1.10.1

Integration of the Qualys Policy Compliance Reporting Service with Splunk TA

We have now integrated Qualys Policy Compliance Reporting Service (PCRS) with Splunk TA. You can now configure the TA app to fetch your PCRS data from your Qualys account.

The PC APIs also pull the posture data, but due to hindrances such as CPU usage, memory consumption, and time consumption to pull the complete information, we introduced PCRS for TA apps.

PCRS improves the data fetching of the huge data on the Qualys Cloud. Fetching data in PCRS is quicker for the accounts with millions of assets and postures.

PCRS fetches data continuously in the following manner:

1. Once you enable the data input, it pulsl the number of Policy IDs to the subscription ID.
2. Divides the Policy IDs into threads and starts pulling the associated hosts.
3. Calls the posture data for all the hosts associated to the policy IDs.

We added a new Policy Compliance Reporting Service Settings section on the TA setup page where you can specify:

• Add additional field evidence
• Add the number of policy Ids that can be used in Resolve Host Id API.

For more details, refer to Policy Compliance Reporting Service Settings section.

We also added a new Qualys Metrics 'pcrs_posture_info' that you can use to create a corn job to pull your PCRS data.

New Feature in 1.9.0

Integration of the Qualys Secure Enterprise Mobility with Splunk TA

With this release, we integrated Qualys Secure Enterprise Mobility (SEM) with Splunk TA. You can now configure TA app to fetch your SEM data from your Qualys account.

On the TA setup page, we added a new Security Enterprise Mobility Settings section where you can specify:

• the SEM data that you want to fetch from your account.
• the number of records that you want to fetch per API request.
• extra params, if any. See Secure Enterprise Mobility Settings.

We also added a new Qualys Metrics 'sem_detection' that you can use to create a corn job to pull your SEM data. The start date for Qualys Metrics should be in 'YYYY-MM-DDThh:mm:ssZ' and cannot be less than the default date '2021-01-26T00:00:00Z'

View Diagnosis, Consequence, and Solution information in KB data in Splunk

We added a new check box in the KnowledgeBase Settings section on the TA setup page. When you select this check box, TA fetches the Diagnosis, Consequence, and Solution information in the Splunk along with the other KB data. When you search for the KB data in Splunk, the new Diagnosis, Consequence, and Solution columns show the information in the respective columns.

Improvements in 1.8.9

Indication of Compromise (IOC) App rebranded as Endpoint Detection and Response (EDR)

With this release, Indication of Compromise (IOC) App is known as Endpoint Detection and Response (EDR) in Qualys TA. Because of this change, we replaced all the instances of IOC on TA UI (labels, IOC data input), log messages with EDR.

If you are using IOC data input and choose to upgrade to TA 1.8.9, now, show you a warning message in the TA log for IOC data input. The warning message informs that IOC data input is deprecated and you need to manually configure the EDR data input from the Splunk UI.

If you are using IOC data input and if you enable the new EDR data input, we check if IOC data checkpoint is available or not. After the check, If we find IOC checkpoint file and do not find EDR check point file then TA renames the IOC checkpoint file to the EDR checkpoint file and consider IOC checkpoint as the start date to fetch the data for the EDR data input.

If both the IOC and EDR checkpoint is available then TA fetches the data from the EDR checkpoint file and ignore the IOC checkpoint file.

We removed the event types of IOC data input. The new event type name for EDR data input is 'qualys_edr_event'. For backward compatibility, that is, to make the older IOC data available in Splunk along with the EDR data, we have merged the IOC and EDR source types into an EDR event type. When you use the EDR event type, older IOC data is fetched for IOC data input, and the latest EDR data for EDR data input.

As IOC App is deprecated, you need to manually add EDR and remove the IOC data input.

Issues Fixed

• TA setup changes for Qualys API credentials 

  We had an issue where the users using multiple technology add ons of different organizations were unable to configure username and password from the TA setup page.

  We fixed this issue by setting TA-QualysCloudPlatform-Api as realm name for Qualys API credentials in the passwords.conf file.

  The realm name was not set in previous releases. Now you can update the username and password from the TA setup page only if the user with 'TA-QualysCloudPlatform-Api' realm name exists in the passwords.conf file.

   When upgrading to TA 1.8.9, manually re-entering your Qualys API credentials is necessary; otherwise, you can not access to the Qualys API server. Before entering the credentials, we recommend that you clear your browser's cache and perform a hard reload.

  We create a new entry for the username with the realm name in the passwords.conf file. This user name with realm name is used to fetch data from your Qualys account.

• Add milliseconds in the checkpoint file for FIM data inputs 

  We fixed an issue where TA was not able to fetch FIM data because checkpoint date or start date is till seconds, whereas FIM supports date in the milliseconds (YYYY-MM-DDThh:mm:ss.msZ) format. To fix this issue, we now check the checkpoint or start date and add milliseconds to it if the checkpoint date is till seconds.

• Fixed incomplete API response XML file issue for Policy Compliance.

  We fixed an issue where, for the Policy Compliance module, TA was unable to fetch data for PC data input and showed an error message if the PC API returned incomplete data or an XML file.

  To fix this issue, now when TA receives incomplete data or XML file, it saves this file as error file and makes the PC API call again to fetch the data from your Qualys account.

• Fixed 400 bad request issue for Container Security

  We fixed an issue where due to limitation of elastic search for Container Security data input if the page size is not equally divisible 10000 then the CS API was throwing 400 bad Request error. We have updated the logic so that elastic search limitation of 10000 is not violated when fetching CS data.

Improvements in 1.8.8

TA to support date format in milliseconds for FIM data input

As FIM now supports date in milliseconds format, TA accepts date format with milliseconds to fetch FIM events, ignored events, and incident data. Due to this change, on the Data Input page, the start date to pull FIM data should be in UTC in ISO 8601 format: 'YYYY-MM-DDThh:mm:ss.msZ'.

If the Start Date field is blank, then we set the default start date to 1999-01-01T00:00:00Z and pull the data from this date. But as FIM requires milliseconds in the date format, we now show an invalid date format message if you leave the Start Date empty for any of the FIM Qualys Metrics. For FIM Qualys Metrics, you need to manually enter the Start Date in UTC in ISO 8601 format: 'YYYY-MM-DDThh:mm:ss.msZ'.

We added this information on the Data Inputs screen (Settings > Data Inputs > Qualys Technology Add-On).

Note that if you are upgrading to TA 1.8.8 and you have already added FIM data inputs, then edit the data inputs as per the new date/time format and save it again to let the data input run successfully.

Improvements in 1.8.7

Updated CS containers and CS images API Version to 1.3

We updated CS Container and CS Image API version from 1.2 to 1.3 for CS container and CS image data inputs.

From this version onwards, use in the CS API request:

• SHA value of the image (imageSha) instead of image ID (imageId) to fetch the image details
• SHA value of the container (containerSha) instead of container ID (containerId) to fetch the container details.
• the pageNumber parameter instead of PageNo parameter to fetch the page with the specified number.

Reset the username and password from the TA setup page

We made an improvement where earlier if a user with two Qualys API accounts on a Qualys platform tried to switch between accounts by changing the Qualys API credentials from the TA setup page, then the password.conf file was required to be removed.

Now, as per the new flow, you do not have to remove the password.conf file while setting the Qualys API credentials from the TA setup page. When you enter the username on the TA setup page, we check if the username already exists in the password.conf file. If the username already exists then we only update the password.

If the username specified on the TA setup page does not exist in the password.conf file, then we fetch the old username from the password.conf file. If the old username is not blank in the file, then we delete the old credentials and add the new username and password specified on the TA setup page. In the case of a new user, we add the new username and password specified on the TA setup page.

Show Splunk restart message when saving settings on TA setup page first time

We now show a message to 'restart the Splunk to load all settings' after you save the settings on the TA setup page for the first time. Earlier, when the user was saving the TA setup form the first time and was not restarting the Splunk, then on the data input and event types pages, the TA set up form was shown instead of the respective forms.

Added DISA STIG SV values to PC Data Input

Policy Posture API response now has <REFERENCE> tag shown under <GLOSSARY>. We show the value of the <REFERENCE> tag in Splunk when you search Policy Posture data using the posture info event. The value for the tag is blank if the <REFERENCE> tag has no value.

Improvements in 1.8.6

Change in processing logic of PC data input

Prior to this release, PC data input was using the 'policy_ids' parameter to pull posture information. With this release, we use the 'policy_id' instead of the 'policy_ids' parameter to pull the posture information. As per the new logic, TA first fetches all the policy IDs using the Compliance Policy List API and then for each policy_id, it fetches the posture information using the Compliance Posture Information API.

As a result of this change, on the TA setup page, we removed the 'Number of POLICY IDs to use for PC Posture Information (max 10)' option and added the 'Number of posture info records per API request' option for PC posture API request. The value in this field is used for the 'truncation_limit' parameter of the PC posture API request and define how many posture info records are returned per request. If the requested list identifies more records than the truncation limit, then the XML output includes the <WARNING> element and the URL for making another request for the next batch of records.

The default value is 1000. If you want to fetch all the posture information in a single output then specify 0. Paginated output is recommended if the posture info data is large.

Change in XML input file parsing logic for performance improvement

We changed the parsing logic for the XML input files to improve the processing time of XML files. TA now does not load the full XML input file in the Splunk memory which was making the system slow and causing the XML processing to take longer time. To improve the performance, TA now parses the XML file line by line or tag by tag.

Improvements in 1.8.5

Added three new fields in the VM Detection Setting section on the TA set up page

We have added three new fields: 'Host fields to log', 'Detection fields to log', and 'Max characters allowed in RESULTS field' in the VM Detection Settings section on the TA Set up page.

1) 'Host fields to log' shows default output values for host assets. You can add additional comma-separated host XML tag names such as 'Asset_ID' returned in the Host List API response that you want to log into the event or remove any existing tag that you don't want to log.

2) 'Detection fields to log' shows default output fields for host detection. You can add additional comma-separated detection XML tag names such as 'AFFECT_EXPLOITABLE_CONFIG' and 'AFFECT_RUNNING_KERNEL' returned in the Host List Detection response that you want to log in the event or remove any existing tag that you don't want to log.

3) Max characters allowed in the RESULTS field lets you specify how many maximum characters appear in the Results field. This means if the number of characters exceeds the maximum allowed characters, then TA truncates the excess characters after parsing the RESULTS field and append the message '[TRUNCATED XXX Characters]' in the RESULTS field.

The 'RESULT_TRUNCATED' field now shows values based on whether the RESULT field is truncated by the TA or Splunk.

1)RESULT_TRUNCATED is '0' if neither TA nor Splunk truncates the RESULTS field/raw event.

2) RESULT_TRUNCATED value is '1' if the RESULTS field is truncated by Splunk. Note that if Splunk truncates the RESULTS field then the message '[TRUNCATED XXX Characters]' in the Results field is not shown.

3) RESULT_TRUNCATED value is '2' if the RESULTS field is truncated by TA. Note that if TA truncates the RESULTS field then the message '[TRUNCATED XXX Characters]' in the Results field is shown.

Improvements in 1.8.4

Added option to index the KB data in Splunk

With this release, we now support indexing of the KnowledgeBase (KB) data in Splunk so that the Splunk TA users on the distributed setup environment can get the updated KnowledgeBase data on the Search Head from the Heavy Forwarder and generate the KB CSV file.

On the TA setup page, we added a KnowledgeBase Settings section that has a check box 'Index the KnowledgeBox...'

The check box indicates whether to index the KnowledgeBase data in Splunk or to write the data into a CSV file. When you select the check box and click Save, TA fetches the KB data and index the KB data in Splunk. If the check box is not selected, TA does not index the KB data into Splunk and creates a CSV file.

The CSV file has KB data from 1999-01-01.

On the Settings > Data Inputs > Add Data page for Qualys technology add on, we added the information that for knowledge_base 'Start Date' field is applicable only if 'index the knowledge base' is enabled on the TA set up page.

After you enable the index KB data option, you need to generate KB CSV lookup on the Search Head. See KnowledgeBase Settings.

CS image label Information now available in CS events

You can now see the CS label information along with the CS image vulnerabilities in CS events for images in Splunk. TA uses a new API '/csapi/v1.2/images/<imageId>' to fetch the CS label and image vulnerability information. TA uses the label key to fetch the label information and the 'vulnerabilities' key to fetch the vulnerability information. The image vulnerabilities and label information is available in cs_vuln_info_event event type.

The new API does not provide image vulnerability summary information in the response. TA generates vulnerability summary information with the help of severity and patch availability fields of vuln summary information. All this vulnerability summary information is available in the cs_vuln_summary_event event type.

Improvements in 1.8.3

We have fixed these issues in 1.8.3.

Issues Fixed

• We fixed an issue where the check box selection values for 'log host summary events' and 'Log Individual Host Vulnerabilities' options in the TA set up > VM Detection settings section was read from the app configuration file instead of qualys.conf file.
• We fixed an issue where TA was logging 'VM host summary events for host detection' in Splunk even though the user had configured to exclude the VM host summary events on the TA setup page.
• We fixed an issue where WAS summary events weren't fetched for all the threads when the WAS data was fetched using multiple threads. Now when the WAS data is fetched in the multi-thread mode, TA logs events in Splunk from all the threads.
• We fixed an issue where TA throws an error and terminates the WAS API call when the WAS data input is fetched using multiple threads and the web application IDs are not distributed appropriately to each thread. To fix this error, we have changed the logic of distribution for web application IDs between the threads so that web application IDs are appropriately distributed.

Improvements in 1.8.2

Enhancements to VM Detection Event

With this release, we have moved the Result field in the VM Detection event to the end of the event. When the Result field is placed before the other event fields, Splunk, at the time of processing the VM Detection event data, truncates all the fields after the Results field if the size of the event exceeds the truncation limit. To avoid truncation of fields, we have added the Results field at the end of the event. Now only the values in the results field is truncated, if the event size exceeds the truncation limit.

We have added a RESULT_TRUNCATED field before the 'Results' field in the event to inform you that the event is truncated or not. RESULT_TRUNCATED = 1 means event is truncated and RESULT_TRUNCATED = 0 means event is not truncated. You can search for truncated and non truncated events using this field.

TA also remove the leading and trailing white spaces from the Results field after TA fetches VM detection data from your Qualys account using the Host List Detection API.

Splunk reads the truncate value from the props.conf file in the TA in 'global/local' directory. If this file is removed from the app 'global/local' directory, then TA reads the truncate value from the global props.conf file in Splunk. TA never truncates the event data while sending it to Splunk. Splunk automatically truncates the event if the size of the event exceeds the truncate limit set in the props.conf or global props.conf file.

The VM Detection event shows the Results field when show_results is set to 1 in the 'Extra Parameters' fields in VM Detection Settings on the TA setup page. If this parameter is not set, then none of these changes have any impact on the VM Detection Event data.

Improvements in 1.8.1

Cleanup Script to remove API output files for Activity Log

We added the 'Activity Log' data input in the cleanup script to remove the API output files from the /tmp directory.

Issue Fixed

We fixed the byte string issue for the host detection data pulled in Splunk for versions above 8.x.x which uses Python 3 interpreter.

Improvements in 1.8.0

Added a new data input - Activity Log

We added a new data input 'Activity Log' to TA to let you pull activity logs from your Qualys Account.To access data input page, go to Settings > Data > Data Inputs > Qualys Technology Add-On. Click Add and from Qualys Metrics drop-down, select activity_log.

Page size field added for data inputs

We added Page size field for these data inputs to let you specify the number of records to be fetched in single API call. The default value for page size is 1000 records, but you can change the value.

• Container Security Data Settings for Images
• Container Security Data Settings for Containers
• FIM settings for events
• FIM settings for ignored events
• FIM settings for incidents
• Indication of Compromise (IOC)

Redesigned TA setup form

We have redesigned the TA setup form to make TA 1.8.0 Splunk cloud compatible, as suggested by the SplunkAppInspect tool, and to improve the user experience.

Issues Fixed

We have fixed the proxy server validation issue in this release.

You can now update Qualys's password in the TA setup form without removing the password.conf file & restarting Splunk.

We now log the error in TA log if the CRON format of data input is invalid.

Improvements in 1.7.1

We made these improvements in 1.7.1

• TA is now compatible with both Python v2.7 and v3.7. See How to switch python interpreter for Python3?
• Container Security APIs now support the API gateway. Private cloud provider can use the gateway URL to connect to and fetch CS data from Qualys Cloud platform.

TA v1.7.1 no longer supports macro definition for indexes

Due to a known issue with Splunk, the user was getting a 255 error on the distributed Search Head setup. We have used macros for the ease of handling indexes and event types.

However, in the distributed setup, macros definitions were not expanded, and as a result, the user got errors on the dashboard or while searching with event types.

To resolve this issue, the Splunk team has suggested not to use macros till further notice from them. See How to assign a custom index to an event type? from KnowledgeBase FAQ

Improvements in 1.6.7

Policy Compliance data to show additional fields

You can now view REMEDIATION, RATIONALE, EVIDENCE and CAUSE_OF_FAILURE information in the compliance posture data for your policy.

To pull this data in Splunk, go to the TA setup page and in the 'Policy Compliance Settings' section, select the 'Add additional fields (REMEDIATION, RATIONALE, EVIDENCE, CAUSE_OF_FAILURE)' check box.

Issues Fixed

We fixed an issue where last evaluated date was not shown as the event date for the policy. Now if the policy has last evaluated date then we show this date as the event date.

Improvements in 1.6.6

TA to use 'updated' dateTime to download Container and Images data in Splunk

The new version of Container Security API uses a new parameter: 'updated' to address the issue with mismatch count between Qualys UI and Splunk.

In TA 1.6.6, we now use the new parameter 'updated' instead of 'created' to ensure that all the Container and Images that were updated in particular duration gets synced in Splunk.

Improvised Logging

We have now improvised logging to print exception messages and avoid logging empty messages.

Masked Passwords

Previously, the password was in plain text. But, we now mask passwords in proxy authentication.

Improved parsing for Host Detection RESULTS

We have improvised Host Detection RESULTS section to address the issue of parsing RESULTS in upper case.

Retry Interval

We have introduced a new configuration 'retry_interval_seconds' to retry same API request after configured interval, in case any error occurs while calling APIs.

Steps to configure 'retry_interval_seconds':

• edit qualys.conf file from below location:
  <Splunk_Home>/etc/apps/TA-QualysCloudPlatform/local/qualys.conf
• add below line to qualys.conf file
  retry_interval_seconds =<time_in_seconds>

Improvements in 1.6.5

TA to use 'processedTime' for downloading FIM Data in Splunk

The new version 2.0.2.0 of FIM API has a new parameter 'processedTime' to address the time lag issues with uploading the events on the Qualys portal by FIM agents.

In TA 1.6.5, we now use the new parameter 'processedTime' instead of 'dateTime' to ensure that all the FIM events that are generated in a particular duration are pulled in Splunk.

Due to this change, TA 1.6.5 works only with FIM API version 2.0.2.0 and later and not with versions earlier than 2.0.2.0.

Improvements in 1.6.4

KnowledgeBase data to show BUGTRAQ_ID field

In Splunk, we now show a new field 'BUGTRAQ_ID' in KnowledgeBase data that is pulled from Qualys. This information is shown for QIDs that has 'BUGTRAQ_ID' available.

FIM events to show event generated time in search results

When you search for FIM events in Splunk, the Time column in search results are now show you the time when the FIM event occurred as reported in your Qualys account. Earlier the time shown was the time when the event is pulled in Splunk.

Improvements in 1.6.3

Error on saving proxy server credentials

Fixed an issue where the TA user was getting an error when saving proxy server credentials required for authentication to the proxy server on the Qualys App set up page. Now the credential details are getting saved.

KnowledgeBase Data not populating in the solution section of the KB lookup file

We fixed an issue where the solution section in the KB lookup file (qualys_kb.csv) was not getting populated due to a failure in parsing of KnowledgeBase data. The parsing error occurred because the parameters 'Threat_INTEL_IDs' and 'Threat_INTEL_VALUES' were not found in the KB lookup file. We have added these two parameters in the KB lookup file.

Handle XML parsing error for WAS data

We fixed an issue where TA used to parse the WAS XML response file that had XML parsing errors. Now when TA receives WAS data that contains parsing errors, it does not parse the file and request Qualys API server to resend the response file. TA keeps on requesting the WAS data from API server till it receives the data contains no parsing errors.

Certificate authentication failure when connecting to Qualys API server

We fixed an issue where authentication to the Qulays API server was getting failed when the user tried to connect to the API server via the proxy server using the certificate.

New Enhancements in 1.6.2

We have made the following enhancements in 1.6.2 release. TA can now:

• Pull EC2 metadata in host detection events using the extra parameter. For example, {'host_metadata': 'ec2', 'host_metadata_fields':'region,accountId,instanceId'}.
• Pull 'cwe' information in Qualys WAS events.
• Retry the request that failed due to corrupted response XML.

New Features in 1.6.1

You can now configure Qualys App for Splunk Enterprise to pull IOCevents data in Splunk from your Qualys account. We added a new Qualys metric (data input feed) 'ioc_events' that you need to configure and enable for pulling the IOC events from your Qualys account. A new event type 'ioc_info_event' is added for searching pulled IOC events in Splunk.

You can now preserve API output files in Splunk using the 'Enable to preserve the XML/JSON files of API output' option. This option is available on the Qualys app setup page. By default, this check box is not be selected.

Added FIM Dashboard

We have also added a FIM dashboard to give you a graphical analysis of your FIM data pulled from your Qualys Account. You can view graphical data for total number changes, events by severity, file and directory changes by change action, and top changes by OS, user and process.

Multithreading not supported for FIM

We removed multithreading support for FIM as the new APIs (FIM API Version 2.0) do not support multithreading.

New Feature in 1.5.0

Qualys App for Splunk Enterprise can now pull FIM data for events, ignored events and incidents from your Qualys Account. On the TA set up page, you can now view 3 new sections: FIM Settings for Events, Ignored Events and Incidents. Specify configuration settings in these sections for collecting FIM data. Next, enable the FIM data feeds to pull the FIM data based on the configuration settings provided on the TA set up page.

New Features and Fixed Issues in 1.4.1

View Qualys Real-time Threat Indicators (RTIs) for vulnerabilities

We are now sending the Qualys Real-time Threat Indicators (RTIs) data in the data input for the Knowledge_base metric. Only, the user account with Threat Protection subscription can view this information for vulnerabilities found in the host based scans. You can set up your dashboard to monitor vulnerabilities for various threat level values.

The sample search shows vulnerabilities for which threat value is High_Data_Loss.

eventtype=qualys_vm_detection_event | dedup 1 HOST_ID, QID | lookup qualys_kb_lookup QID OUTPUT THREAT_INTEL_VALUES | search THREAT_INTEL_VALUES='*High_Data_Loss*' | table HOST_ID, LAST_SCAN_DATETIME, QID, THREAT_INTEL_VALUES

Support for arf_kernel filters parameter for VM host detection

We now support 'arf_kernel filters' parameter to identify vulnerabilities found on running or non-running Linux kernels.. You can update the optional parameter to include the arf_kernel parameter in VM Detection Settings on the TA setup page.

Set show_results=1 to view TCP/UDP port information

We have fixed an issue where the user was unable to view the open TCP/UDP ports information in the HOSTSYMMARY events. To view the information, update optional parameters in VM Detection Settings on the TA setup page to include 'show_results=1'.

Newline character removed from the port data in vulnerability data feed

We have fixed an issue where whitespace and newline characters in the port data in the Results tag in the vulnerability data feed fetched from the Qualys Server were introducing new events when imported in Splunk. Now, we have fixed this issue by removing these characters from the vulnerability data feed before importing it in Splunk.

Enable CVSS scoring in your account to view CVSS scores for vulnerabilities

We have fixed an issue where Splunk was showing an error for missing CVSS data when importing KnowledgeBase API response in Splunk TA. This issue was occurring for the user accounts that have CVSS Scoring not enabled for their subscriptions. As a result, the KnowledgeBase API response does not have CVSS data for vulnerabilities. To Enable CVSS Scoring in your Qualys account, go to 'Reports > Setup > CVSS > Enable CVSS' and click 'save'.

Now, Splunk does not show missing CVSS data error if you do not enable CVSS scoring for your subscription. In this case, Splunk now shows no CVSS metrics scores for vulnerabilities in the Splunk KnowledgeBase.

New Feature in 1.4.0

TA now supports ingesting Container Security data

Qualys App for Splunk Enterprise can now pull vulnerability information for docker image and container in Container Security from your Qualys account. TA pulls CS data based on the configuration information you have provided in the Container Security Settings for Images and Containers. CS data is in JSON format.

New Feature in 1.3.4

New information added in HOSTSUMMARY and HOSTVULN events

Added NETWORK_ID, LAST_VM_SCANNED_DATE and LAST_VM_SCANNED_DURATION information in HOSTSUMMARY.

Added LAST_FIXED_DATETIME, TIMES_FOUND, IS_IGNORED, IS_DISABLED information in HOSTVULN.

New Features in 1.3.3

New Basic option for fetching policy posture compliance data

You can now specify to Posture API to fetch only basic details of the policy posture compliance data for policy IDs. This option is for policy IDs with large posture compliance data. Keep the 'Log All details (when unchecked, logs 'Basic' details)' check box deselected in the Policy Compliance Settings for the API to get basic details.

Configure total number of policy IDs to be fetched

You can now configure in the Policy Compliance Settings the total number of policy IDs to be fetched by the Posture API. The valid number range is 1 to 10. Set this value low for policy IDs with large policy posture compliance data.

New Features in 1.3.1

Introducing new data input for Policy Compliance

TA is now able to pull and ingest Policy Compliance posture information! The TA Setup page includes new Policy Compliance configuration settings. The extra parameters option accepts API parameters for Posture Information API (/api/2.0/fo/compliance/posture/info/ with action=list). When pulling policies information, Posture API parameter policy_ids becomes the parameter ids for Policy detail API call.

Support for using client certificates to call API

Now you can specify a client certificate in TA so that TA uses it while making API calls. A new section has been added to the TA setup page for this.

New utility script to clean up left-over XML and PID files

This new script is useful for cleaning up orphan XML files in the TA-DIR/tmp directory. While running the utility, you can provide command line options to specify data inputs for the XML files to be cleaned up. The utility deletes all the XML files for the chosen data inputs, except those belonging to currently running TA processes.

Additional Improvements 1.3.1

Update to Host List Detection API

You can now see the parameter vm_processed_after in TA logs. With Qualys 8.9, we 1) changed the way we report host scan time so it’s based on when a scan finished, not when the scan started. 2) Introduced new parameters to filter the Host List VM Detection API by scan end dates and processed dates. The vm_processed_after parameter is used to filter the list to only show hosts with vulnerability scan results processed after a certain date and time.

Setup page save fails if there are any validation errors

TA tries to validate inputs given on the TA setup page. If validation fails, it does NOT save any details, but raise a ValueError. This results in a generic error message in the Splunk UI. You can see a more detailed error message given by TA in splunkd.log.

When installed on Search Head, do not run data inputs other than knowledge base

Checks were added to the code (with help from the Splunk team) to ensure that TA only run the knowledgebase data input when TA is installed on a Search Head, even when other data inputs have been added and enabled. In other words, TA does not run host detection, WAS findings and PC posture information data inputs when installed on Search Head.

Log error messages given by Qualys API

If the Qualys API responds back with an error (in response body), TA now logs the error message in the TA log for troubleshooting. This way you can know if there’s an API reason for not getting data (e.g. Rate Limit exceeded).

PID repeat issue resolved

TA writes PID in .pid file for every input run . This file is deleted at the end of the run. TA uses this pid file to check if any process with the PID is running. If it finds any such process, TA checks if the process is running qualys.py then only it terminates itself, else TA runs the qualys.py script for the scheduled input.

Configurable API Timeout period

The API timeout period is 300 seconds by default. If this value is not adequate, you can set a different timeout value on the TA setup page.

Display API parameters not allowed by TA

To avoid operational problems, API parameters that are not allowed by TA are now clearly listed for each Extra API parameter field on the TA setup page.

Log the index name being used in each run

To help with troubleshooting, TA now log the name of the index for each run's data. This is the same index name as selected by the user while adding/updating the data input.

Display data input name in each log entry

There are some common execution paths for all data inputs in TA, and they write some log entries. When multiple data inputs are running at the same time, it becomes hard to identify which log entry was written for which data input. To fix this, TA has to mention the data input it is running for in each log entry it writes. This way, one can grep all the log entries belonging to a particular data input. This would be useful if you are troubleshooting subsequent runs of the same data input.

Avoid unnecessary call to msp/about.php each time Splunk invokes modular input

Splunk invokes TA’s entry point script every 60 seconds. On each invocation, the code checks for the Qualys version by making a msp/about.php API call. This call was being made irrespective of whether the current time matched the configured cron/time interval. To avoid unnecessary calls, TA first check if now is the time for any input to run. If yes, the API call is made. If no, the API call is not made.