Overview
The Glasswall Runner is a wrapper around the Glasswall engine using the Glasswall Python Wrapper. It features several modes which can be used to process or transform files or directories of files.
Command-line Interface
Options
-h, --help show this help message and exit
-m {Analysis,AnalysisProtect,Combine,CombinePerformanceTest,ExportUnpackCombine,Export,Import,Pack,Protect,Redact,Uncombine,Unpack,UncombinePackImportProtect,UnpackProtectPack}
Glasswall Runner functionality to call.
-i INPUT Absolute path to input file or directory containing files to process.
-o OUTPUT Absolute path to output file or directory to write processed files.
-l LIBRARIES_PATH Absolute path to directory containing Glasswall libraries.
-c CONFIG Optional absolute path to content management policy config XML file.
--quarantine QUARANTINE
Optional absolute path to directory to move files that Glasswall failed to process (unsupported or nonconforming files).
--raise-unsupported Terminate early when Glasswall cannot process a file.
--logging {CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET}
Set Glasswall Runner logging level.
--logging-glasswall {CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET}
Set Glasswall Python Wrapper logging level.
--debug Enable debug mode. In debug mode, quarantine will copy instead of move files.
--homoglyphs HOMOGLYPHS
Optional absolute path to homoglyphs JSON file (Redact mode).
--redact-list REDACT_LIST
Optional absolute path to a .txt file that contains search terms to redact. One search term per line (Redact mode).
--output-report OUTPUT_REPORT
Optional absolute path to output file or directory to write reports (ExportArchive/ImportArchive/Redact mode).
--archive-whitelist ARCHIVE_WHITELIST
Dangerous. Optional absolute path to a .txt file that contains whitelisted file formats that are never removed from archives, even if sanitisation fails. One file format per line (UnpackProtectPack mode).
--upp-report UPP_REPORT
Optional absolute path to output file to write JSON report (UnpackProtectPack mode).
--archive-file-type {7z,bz2,gz,tar,xz,zip}
Optional archive file format to use when packing with ArchiveManager, default: zip (Pack/UncombinePackImportProtect mode).
--unpack-include-file-type
Optional, include file type in directory name when unpacking archives, default: false (Unpack mode).
--performance-test PERFORMANCE_TEST
Optional absolute path to output file to write JSON performance test report (Analysis/AnalysisProtect/Export/Import/Protect mode).
--combine-performance-test-filter COMBINE_PERFORMANCE_TEST_FILTER
Only match files ending with the filter, default '.pt.json'. (CombinePerformanceTest mode).
Configuration
A file path to a content management policy config XML file can be specified using the optional -c command line argument. The Glasswall engine uses this config to decide which types of content within files to allow, sanitise, or disallow. If this argument is not set then the default "sanitise all" content management policy will be applied.
Example Editor default content management policy XML file:
<?xml version="1.0" encoding="utf-8"?>
<config>
<gifConfig>
<metadata>sanitise</metadata>
</gifConfig>
<pdfConfig>
<acroform>sanitise</acroform>
<actions_all>sanitise</actions_all>
<digital_signatures>sanitise</digital_signatures>
<embedded_files>sanitise</embedded_files>
<embedded_images>sanitise</embedded_images>
<external_hyperlinks>sanitise</external_hyperlinks>
<internal_hyperlinks>sanitise</internal_hyperlinks>
<javascript>sanitise</javascript>
<metadata>sanitise</metadata>
</pdfConfig>
<pptConfig>
<embedded_files>sanitise</embedded_files>
<embedded_images>sanitise</embedded_images>
<external_hyperlinks>sanitise</external_hyperlinks>
<internal_hyperlinks>sanitise</internal_hyperlinks>
<javascript>sanitise</javascript>
<macros>sanitise</macros>
<metadata>sanitise</metadata>
<review_comments>sanitise</review_comments>
</pptConfig>
<sysConfig>
<export_embedded_images>true</export_embedded_images>
<interchange_best_compression>false</interchange_best_compression>
<interchange_pretty>false</interchange_pretty>
<interchange_type>sisl</interchange_type>
<run_mode>enablerebuild</run_mode>
</sysConfig>
<tiffConfig>
<geotiff>sanitise</geotiff>
</tiffConfig>
<wordConfig>
<dynamic_data_exchange>sanitise</dynamic_data_exchange>
<embedded_files>sanitise</embedded_files>
<embedded_images>sanitise</embedded_images>
<external_hyperlinks>sanitise</external_hyperlinks>
<internal_hyperlinks>sanitise</internal_hyperlinks>
<macros>sanitise</macros>
<metadata>sanitise</metadata>
<review_comments>sanitise</review_comments>
</wordConfig>
<xlsConfig>
<dynamic_data_exchange>sanitise</dynamic_data_exchange>
<embedded_files>sanitise</embedded_files>
<embedded_images>sanitise</embedded_images>
<external_hyperlinks>sanitise</external_hyperlinks>
<internal_hyperlinks>sanitise</internal_hyperlinks>
<macros>sanitise</macros>
<metadata>sanitise</metadata>
<review_comments>sanitise</review_comments>
</xlsConfig>
</config>
Note: Some modes may require
sysConfigswitches to be set to specific nondefault values in order to function, such as Redact mode requiring theinterchange_typeswitch to be set toxml. When using a custom content management policy the Glasswall Runner will prioritise using the requiredsysConfigswitch value.
Modes
A mode must be specified when calling the Glasswall Runner using the -m command line argument. Different modes have different required and optional command line arguments. Some modes function without using the Glasswall engine and do not require a -l argument. Available modes are listed below and are case sensitive.
Protect
Required args: m, i, o, l
Protect a single file.
glasswall_runner.exe -m Protect -i "C:\input\DOCXimages.docx" -o "C:\output\protect\DOCXimages.docx" -l "C:\azure\current\2023-06-05"
Protect all files in a directory.
glasswall_runner.exe -m Protect -i "C:\input" -o "C:\output\protect_directory" -l "C:\azure\current\2023-06-05"
Protect all files in a directory using a custom content management policy, moving nonconforming, unsupported, and disallowed files to a quarantine directory.
glasswall_runner.exe -m Protect -i "C:\input\some_nonconforming" -o "C:\output\protect_directory2" -l "C:\azure\current\2023-06-05" -c "C:\input\config_docx_disallow_embedded_images.xml" --quarantine "C:\output\protect_directory2_quarantine"
Example config used: "config_docx_disallow_embedded_images.xml"
<?xml version="1.0" encoding="utf-8"?>
<config>
<wordConfig>
<embedded_images>disallow</embedded_images>
</wordConfig>
</config>
Protect all files in a directory, saving performance test metrics.
glasswall_runner.exe -m Protect -i "C:\input" -o "C:\output\protect_directory_performance_test" -l "C:\azure\current\2023-06-05" --performance-test "C:\output\protect_directory_performance_test\input.pt.json"
Example performance test JSON report:
[
{
"input_file": "C:/input/DOCXimages.docx",
"elapsed_milliseconds": 636,
"input_sha256": "0c33844c15589033ab7586c4ddc6378ff19a35186d72848c1562dedd2b9f9297",
"mode": "Protect",
"passed": true
},
{
"input_file": "C:/input/config.xml",
"elapsed_milliseconds": 21,
"input_sha256": "bd34129adc72ea3b8af818026f40229908abe795b849b9f0197d7630c862c2f3",
"mode": "Protect",
"passed": false
},
{
"input_file": "C:/input/config_docx_disallow_embedded_images.xml",
"elapsed_milliseconds": 9,
"input_sha256": "dc0f758eb72cc778d9b6693a18204db5594b092ccc1bf837ea1ae010bca694e9",
"mode": "Protect",
"passed": false
},
{
"input_file": "C:/input/config_redact.xml",
"elapsed_milliseconds": 8,
"input_sha256": "96693f0fc59ae305f2ab782cf2243f771d2a1ea68d7832be1af31b2fc9370c78",
"mode": "Protect",
"passed": false
},
{
"input_file": "C:/input/redact_list.txt",
"elapsed_milliseconds": 5,
"input_sha256": "a12927c25a413290032c8fe7aabe30a20468d6984012014790a631ca3e2c00c4",
"mode": "Protect",
"passed": false
},
{
"input_file": "C:/input/upp/archive_whitelist.txt",
"elapsed_milliseconds": 6,
"input_sha256": "e9aafe226374d2f75e719d4b9b5f4153b151841e17297410af78c928ff02e4b6",
"mode": "Protect",
"passed": false
},
{
"input_file": "C:/input/upp/nested2.zip",
"elapsed_milliseconds": 46,
"input_sha256": "dfa3d58ccb65e2128dfe9ff62a934b5fb4731f431c1f72f0820fbf0fa480bbaa",
"mode": "Protect",
"passed": false
}
]
Analysis
Required args: m, i, o, l
Analyse a single file.
glasswall_runner.exe -m Analysis -i "C:\input\DOCXimages.docx" -o "C:\output\analysis\DOCXimages.docx.xml" -l "C:\azure\current\2023-06-05"
Analyse all files in a directory.
glasswall_runner.exe -m Analysis -i "C:\input" -o "C:\output\analysis_directory" -l "C:\azure\current\2023-06-05"
AnalysisProtect
Required args: m, i, o, l
Analyse and Protect a single file.
glasswall_runner.exe -m AnalysisProtect -i "C:\input\DOCXimages.docx" -o "C:\output\analysis_protect\DOCXimages.docx" -l "C:\azure\current\2023-06-05"
Analyse and Protect all files in a directory.
glasswall_runner.exe -m AnalysisProtect -i "C:\input" -o "C:\output\analysis__protect_directory" -l "C:\azure\current\2023-06-05"
Export
Required args: m, i, o, l
Export a single file.
glasswall_runner.exe -m Export -i "C:\input\DOCXimages.docx" -o "C:\output\export\DOCXimages.docx.zip" -l "C:\azure\current\2023-06-05"
Export all files in a directory.
glasswall_runner.exe -m Export -i "C:\input" -o "C:\output\export_directory" -l "C:\azure\current\2023-06-05"
Import
Required args: m, i, o, l
Import a single file.
glasswall_runner.exe -m Import -i "C:\output\export\DOCXimages.docx.zip" -o "C:\output\import\DOCXimages.docx" -l "C:\azure\current\2023-06-05"
Import all files in a directory.
glasswall_runner.exe -m Import -i "C:\input" -o "C:\output\import_directory" -l "C:\azure\current\2023-06-05"
glasswall_runner.exe -m Import -i "C:\output\export_directory" -o "C:\output\import_directory" -l "C:\azure\current\2023-06-05"
Pack
Required args: m, i, o, l
Supported archive types include: "7z", "bz2", "gz", "tar", "xz", "zip".
Note: "rar" support is currently limited to unpacking only.
Pack a directory to ".zip".
glasswall_runner.exe -m Pack -i "C:\output\protect" -o "C:\output\pack" -l "C:\azure\current\2023-06-05"
The directory "C:\output\protect" is packed to "C:\output\pack\protect.zip"
Pack a directory to a different archive file format.
glasswall_runner.exe -m Pack -i "C:\output\protect" -o "C:\output\pack" -l "C:\azure\current\2023-06-05" --archive-file-type 7z
Unpack
Required args: m, i, o, l
Supported archive types include: "7z", "bz2", "gz", "rar", "tar", "xz", "zip".
Unpack a single archive.
glasswall_runner.exe -m Unpack -i "C:\output\pack\protect.zip" -o "C:\output\unpack" -l "C:\azure\current\2023-06-05"
The archive "C:\output\pack\protect.zip" is unpacked to "C:\output\unpack" maintaining the same directory structure. In this example the input archive "C:\output\pack\protect.zip" contained one file: "DOCXimages.docx" and is unpacked to the path: "C:\output\unpack\protect\DOCXimages.docx".
Unpack all archives in a directory.
glasswall_runner.exe -m Unpack -i "C:\output\pack" -o "C:\output\unpack_directory" -l "C:\azure\current\2023-06-05"
All archives in "C:\output\pack" are unpacked to "C:\output\unpack_directory" maintaining the same directory structure.
Note: In the above example the directory "C:\output\pack" contains two archives, "protect.7z" and "protect.zip". Each archive contains the same file: "DOCXimages.docx". Attempting to unpack both archives to the same output directory: "C:\output\unpack_directory" would result in one file overwriting the other if default settings are used. Files are processed alphabetically, "protect.7z" would be unpacked first resulting in the file path: "C:\output\unpack_directory\protect\DOCXimages.docx". "protect.zip" would be unpacked second and would overwrite the file at this path. This behaviour can be avoided using the "--unpack-include-file-type" command line argument. This will include the archive file type in the directory name when unpacking the archive and will result in two files present in the output directory: "C:\output\unpack_directory\protect.7z\DOCXimages.docx" and "C:\output\unpack_directory\protect.zip\DOCXimages.docx".
Combine
Required args: m, i, o
Combine all XML files in a directory into a single XML file.
glasswall_runner.exe -m Combine -i "C:\output\unpack_export\DOCXimages.docx" -o "C:\output\combine\DOCXimages.docx.xml"
Note: The input above is a directory containing XML files. It is the result of using Export mode followed by Unpack. See further example: Transforming a file to XML.
Uncombine
Required args: m, i, o
Uncombine a single XML file to a directory.
glasswall_runner.exe -m Uncombine -i "C:\output\combine\DOCXimages.docx.xml" -o "C:\output\uncombine"
The directory "C:\output\uncombine\DOCXimages.docx" now contains all of the XML files that were combined in "C:\output\combine\DOCXimages.docx.xml".
Redact
Text content can be redacted from files using Redact mode with the --redact-list argument:
glasswall_runner.exe -m Redact -i "C:\input\redact\Test1234.pptx" -o "C:\output\redact\Test1234.pptx" -l "C:\azure\current\2023-06-05" --redact-list "C:\input\redact_list.txt"
Example redact_list.txt used: "C:\input\redact_list.txt"
please
lorem
microsoft
The default behaviour of Redact mode is to replace matched text content with asterisks: *. This can be customised by using a content management policy configuration file instead of the --redact-list argument:
glasswall_runner.exe -m Redact -i "C:\input\redact\Test1234.pptx" -o "C:\output\redact_custom\Test1234.pptx" -l "C:\azure\current\2023-06-05" -c "C:\input\config_redact.xml"
Example config_redact.xml used: "C:\input\config_redact.xml"
<?xml version="1.0" encoding="utf-8"?>
<config>
<sysConfig>
<interchange_type>xml</interchange_type>
</sysConfig>
<textSearchConfig libVersion="core2">
<textList>
<textItem>
<text>please</text>
<textSetting replacementChar="*">redact</textSetting>
</textItem>
<textItem>
<text>lorem</text>
<textSetting replacementChar="*">redact</textSetting>
</textItem>
<textItem>
<text>microsoft</text>
<textSetting replacementChar="*">redact</textSetting>
</textItem>
</textList>
</textSearchConfig>
</config>
ExportUnpackCombine
Required args: m, i, o, l
Transform a file to XML
glasswall_runner.exe -m ExportUnpackCombine -i "C:\input\DOCXimages.docx" -o "C:\output\export_unpack_combine\DOCXimages.docx.xml" -l "C:\azure\current\2023-06-05"
The output XML file "C:\output\export_unpack_combine\DOCXimages.docx.xml" contains multiple file streams stored in CDATA.
If access to the intermediate files is required in between the use of Export, Unpack, Combine, see: Transforming a file to XML.
UncombinePackImportProtect
Required args: m, i, o, l
Transforming XML to a file
glasswall_runner.exe -m UncombinePackImportProtect -i "C:\output\export_unpack_combine\DOCXimages.docx.xml" -o "C:\output\uncombine_pack_import_protect\DOCXimages.docx" -l "C:\azure\current\2023-06-05"
If access to the intermediate files is required in between the use of Uncombine, Pack, Import, Protect, see: Transforming XML to a file.
UnpackProtectPack
Required args: m, i, o, l
Unpack an archive, Protect the contents, Pack to archive
glasswall_runner.exe -m UnpackProtectPack -i "C:\output\pack\protect.zip" -o "C:\output\unpack_protect_pack\protect.zip" -l "C:\azure\current\2023-06-05"
Files will be protected with the default sanitise all content management policy. A path to a custom content management policy configuration can be specified using the -c command line argument. Any files in the input archive that are nonconforming, unsupported, or that cannot be processed will be removed in the output archive. The --archive-whitelist argument can be used to specify a path to a .txt file containing a list of files that should never be removed.
Example archive whitelist file:
.cfg
.gitattributes
.gitignore
.json
.log
.md
.sisl
.txt
.xml
.yaml
.yml
Specialised reporting is available for this mode using the --upp-report command line argument
glasswall_runner.exe -m UnpackProtectPack -i "C:\input\upp\nested2.zip" -o "C:\output\unpack_protect_pack2\nested2.zip" -l "C:\azure\current\2023-06-05" --archive-whitelist "C:\input\upp\archive_whitelist.txt" --upp-report "C:\output\unpack_protect_pack2\nested2.zip.upp.json"
Example upp report:
{
"summary": {
"input_file": "C:\\input\\upp\\nested2.zip",
"library": "archive_manager",
"output_file": "C:\\output\\unpack_protect_pack2\\nested2.zip",
"nested_archives": 3,
"total_files": 7,
"protected_files": 3,
"whitelisted_files": 3,
"quarantined_files": 1,
"protected_archives": 1,
"status": "Success"
},
"detail": [
{
"input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip",
"library": "archive_manager",
"status": "Success",
"output_directory": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2"
},
{
"input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\tar.tar",
"library": "archive_manager",
"status": "Success",
"output_directory": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2"
},
{
"input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip\\b.zip",
"library": "archive_manager",
"status": "Success",
"output_directory": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip"
},
{
"input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip\\b.pptx",
"library": "editor",
"output_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\irahr9yj\\nested2\\a.zip\\b.pptx"
},
{
"input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip\\b.txt",
"library": "editor",
"whitelisted": true,
"output_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\irahr9yj\\nested2\\a.zip\\b.txt"
},
{
"input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip\\b.zip\\c\\d.pptx",
"library": "editor",
"output_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\irahr9yj\\nested2\\a.zip\\b.zip\\c\\d.pptx"
},
{
"input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip\\b.zip\\c\\d.txt",
"library": "editor",
"whitelisted": true,
"output_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\irahr9yj\\nested2\\a.zip\\b.zip\\c\\d.txt"
},
{
"input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\conforming\\launchCalculator.docx",
"library": "editor",
"output_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\irahr9yj\\nested2\\conforming\\launchCalculator.docx"
},
{
"input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\nonconforming\\NC-0000001.bmp",
"library": "editor",
"quarantined": true,
"status": "FileRejected"
},
{
"input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\tar.tar\\tar.txt",
"library": "editor",
"whitelisted": true,
"output_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\irahr9yj\\nested2\\tar.tar\\tar.txt"
}
]
}
The archive file format can be modified by changing the path specified in the -o command line argument. The example below would Unpack zip, Protect, and then Pack as 7z. For supported archive types see: Pack.
glasswall_runner.exe -m UnpackProtectPack -i "C:\output\pack\protect.zip" -o "C:\output\unpack_protect_pack\protect.7z" -l "C:\azure\current\2023-06-05"
CombinePerformanceTest
Required args: m, i, o
Combine multiple performance test JSON reports into a single report
glasswall_runner.exe -m CombinePerformanceTest -i "C:\output" -o "C:\output\combined_performance_test\output.pt.json"
By default this mode recursively searches the input directory for files ending .pt.json. This search string can be changed using the --combine-performance-test-filter command line argument:
glasswall_runner.exe -m CombinePerformanceTest -i "C:\output" -o "C:\output\combined_performance_test\output.pt.json" --combine-performance-test-filter ".json"
Further Examples
Transforming a file to XML
Transforming a file to XML using Export, Unpack, and Combine can be performed easily using the mode: ExportUnpackCombine. Alternatively if access to the intermediate files is required in between these modes, examples of using Export, Unpack, and Combine are below:
Starting with the file: "C:\input\DOCXimages.docx" we can first use Export mode to transform this file into multiple XML files within a zip archive:
glasswall_runner.exe -m Export -i "C:\input\DOCXimages.docx" -o "C:\output\export\DOCXimages.docx.zip" -l "C:\azure\current\2023-06-05"
The file has now been transformed to XML within a zip archive. If necessary, we can Unpack the zip archive:
glasswall_runner.exe -m Unpack -i "C:\output\export\DOCXimages.docx.zip" -o "C:\output\unpack_export" -l "C:\azure\current\2023-06-05"
The XML files are now present in the directory: "C:\output\unpack_export\DOCXimages.docx". If necessary, we can Combine these XML files into a single XML file:
glasswall_runner.exe -m Combine -i "C:\output\unpack_export\DOCXimages.docx" -o "C:\output\combine\DOCXimages.docx.xml"
The original file has now been transformed into a single XML file. The file size is expected to be larger than the original. For reference, the file used in this example was 6.56MB before transformation and 16.1MB after.
Transforming XML to a file
We can reverse the transformation using the mode: UncombinePackImportProtect. Alternatively if access to the intermediate files is required in between these modes, examples of using Uncombine, Pack, Import, Protect are below:
Starting with the XML file from the above example: "C:\output\combine\DOCXimages.docx.xml" we can first use Uncombine mode to transform this XML file into multiple XML files within a directory:
glasswall_runner.exe -m Uncombine -i "C:\output\combine\DOCXimages.docx.xml" -o "C:\output\uncombine"
The directory "C:\output\uncombine\DOCXimages.docx" now contains multiple XML files. We can pack them into a zip archive ready for Import to consume:
glasswall_runner.exe -m Pack -i "C:\output\uncombine\DOCXimages.docx" -o "C:\output\uncombine_packed" -l "C:\azure\current\2023-06-05"
The directory has been packed to "C:\output\uncombine_packed\DOCXimages.docx.zip". We can now use Import mode to transform the zip archive into a file.
glasswall_runner.exe -m Import -i "C:\output\uncombine_packed\DOCXimages.docx.zip" -o "C:\output\uncombine_imported\DOCXimages.docx" -l "C:\azure\current\2023-06-05"
The file has now been transformed from XML: "C:\output\uncombine_imported\DOCXimages.docx"