CSV Automation

CSV Automation enables automated processing and uploading of CSV files to a specified gateway URL. This topic provides comprehensive instructions for installing, configuring, and operating the automation system.

To begin with CSV Automation, you will need to download the required scripts from the Connectors app.

Within the script, you can specify the file path or directory where all the relevant CSV files are stored. This gives the script the location to look for and process the files. 

You can run the script periodically to check the specified file path and automatically upload any new or updated CSV files to the configured connector. The script iterates through the files in the directory and sends them one by one to the connector.

Instead of running the CSV automation script manually every time, you can create and run a scheduler script. This scheduler script can be set to run the CSV automation script on a periodic basis, such as hourly, daily, or weekly. This ensures the CSV files are uploaded to the connector on a regular schedule without manual intervention.

Now let's begin with creating your first CSV automation.

How to Download the CSV Automation Script?

1. Navigate to Generic CSV Connector.
2. Select any connector from your connector list.
   
3. From the Quick Actions Menu, select CSV Automation.
   

How To Run The CSV Automation Script?

Before running the CSV Automation script, you must ensure you have the following set up and running.

1. You can install the latest version of Python here.
2. Download the zip files from CSV Connector as described above.
3. Navigate to the Project Directory.
   cd connector-config-automation
4. Install Dependencies.
   pip install -r requirements.txt
5. Create Configuration file with name config.json in same folder as main.py
   {
       "header": 1,
       "baseUrl": "https://yourapi.example.com",
       "username": "your_username",
       "password": "your_password",
       "connectionUuid": "your_connection_uuid",
       "profileUuid": "your_profile_uuid",
       "envQualysUsernameProperty": "ENV_QUALYS_USERNAME",
       "envQualysPasswordProperty": "ENV_QUALYS_PASSWORD"
   }

Running the Script

Execute the following command to run the script.

cd connector-config-automation python3 main.py --header <header-line> --csvPath <csv-file-path> --baseUrl <base-url> --username <username> --password <password> --profileUuid <profile-uuid> --connectionUuid <connection-uuid>

Below, you can find the list of available commands and their descriptions.

Command	Description
--header	Line number of the header row (default: 1).
--csvPath	Path to the CSV file (required).
--baseUrl	Base apigateway Qualys URL for uploading chunks (required).
--username	Qualys Username for authentication (optional, if you provide value for envQualysUsernameProperty property we will read it from env).
--password	Qualys Password for authentication (optional, if you provide value for envQualysPasswordProperty property we will read it from env).
--connectionUuid	Connection UUID (required, this will be available on connector UI).
--profileUuid	Profile UUID (required, this will be available on connector UI).
--envQualysUsernameProperty	Environment variable name for username (optional).
--envQualysPasswordProperty	Environment variable name for password (optional).

To use this script as part of other scripts, run the following command.

   import start from connector-config-automation.main.py
   start(csv_path)

Your selected connector can now upload new CSV files automatically to run the connection and fetch findings.