The performance of the Glasswall CDR Platform depends on several factors, including:
- File type, size, and complexity
- The number of Glasswall engines available for processing
- Current traffic/load on the system
Every file has a unique Glasswall engine rebuild time, which directly impacts total processing time. More engines enable more files to be processed concurrently.
Example:
1 deployed engine = 1 concurrent rebuild
5 deployed engines = 5 concurrent rebuilds
Note: The number of engines does not equal the number of concurrent API requests the platform can handle. For best performance, keep concurrent API requests at or below the number of available engines.
1. File Type
- Complex files (e.g. PDFs, archives) take longer than simpler formats (e.g. images).
- Archives process slower as each file inside is treated separately.
2. File Size
- Larger files require more scanning and rebuilding.
- Example: A 2MB PDF takes significantly longer than a 20KB PDF.
3. Current Load
- When concurrent requests exceed the number of engines, files are queued.
- Queued files are delayed by the processing time of earlier files in the queue.
- Wait times vary based on file complexity and processing time of prior batches.
Test Summary
- Files Tested: 2000 business files
- Size Range: 0.05 MB – 17 MB
- Mean Size: 3.74 MB
- Types: PowerPoint, Video, Excel, Word, Image, PDF, Audio
- Halo Version: v2.6.1
Kubernetes (AKS)
Pod Resource Configuration:
| Service | Requested CPU | Requested Memory | CPU Limit | Memory Limit |
|---|
| Engine | 200m | 500Mi (524MB) | 1 core | 1Gi (1.07GB) |
| API | 1500m | 4Gi (4.29GB) | 1500m | 4Gi (4.29GB) |
Small (1 Node)
- Node Type: 1 x DS4_v2
- Resources: 8 vCores, 28GB RAM, 5 Engines
| Metric | Value |
|---|
| Files per Hour | 21,600 |
| Data per Hour | 96 GB |
| Files per Day | 518,400 |
| Data per Day | 2.3 TB |
| Mean Processing Time | 4769.94 ms |
| API Success Rate | 99.999% |
Medium (3 Nodes)
- Node Type: 3 x DS4_v2
- Resources: 24 vCores total, 84GB RAM total, 15 Engines
| Metric | Value |
|---|
| Files per Hour | 64,800 |
| Data per Hour | 200 GB |
| Files per Day | 1,555,200 |
| Data per Day | 4.8 TB |
| Mean Processing Time | 1788.83 ms |
| API Success Rate | 99.999% |
Large (5 Nodes)
- Node Type: 5 x DS4_v2
- Resources: 40 vCores total, 140GB RAM total, 25 Engines
| Metric | Value |
|---|
| Files per Hour | 108,000 |
| Data per Hour | 343 GB |
| Files per Day | 2,592,000 |
| Data per Day | 8.23 TB |
| Mean Processing Time | 17 ms |
| API Success Rate | 99.999% |
File set
The requests made to induce load used a set of 2000 files gathered from https://www.gov.uk . Below is a summary of the test set broken down by file type:
| Filetype | Proportion (%) | Average Size (KiB) | Std Dev (KiB) | Min Size (KiB) | Max Size (KiB) | Supported |
|---|
| pdf | 89.05 | 566.90 | 1636.00 | 0.04 | 34122.18 | Yes |
| docx | 2.40 | 135.08 | 236.61 | 15.97 | 1361.26 | Yes |
| xls | 1.95 | 497.44 | 1111.57 | 40.00 | 6429.50 | Yes |
| xlsx | 1.80 | 952.31 | 3504.89 | 18.43 | 20207.05 | Yes |
| ods | 1.65 | 230.50 | 352.80 | 4.59 | 1948.65 | No |
| odt | 1.20 | 210.83 | 437.70 | 7.91 | 1798.76 | No |
| doc | 1.05 | 595.93 | 670.87 | 24.00 | 2062.00 | Yes |
| csv | 0.60 | 901.05 | 3002.84 | 0.48 | 10434.15 | No |
| ppt | 0.15 | 4168.55 | 5460.57 | 933.00 | 10473.14 | Yes |
| xlsm | 0.10 | 2239.00 | 3048.01 | 83.73 | 4394.27 | Yes |
| jpg | 0.05 | 15.23 | - | 15.23 | 15.23 | Yes |
OVA (Standalone VM)
Pod Resource Configuration:
| Service | Requested CPU | Requested Memory | CPU Limit | Memory Limit |
|---|
| Engine | 1300m | 3Gi (3.22GB) | 1300m | 3Gi (3.22GB) |
| API | 1 core | 3Gi (3.22GB) | 1 core | 3Gi (3.22GB) |
Small (Single VM)
- VM Type: F16s_v2
- Resources: 16 vCores, 32 GB RAM, 3 Engines
| Metric | Value |
|---|
| Files per Hour | 9,150 |
| Data per Hour | 33 GB |
| Files per Day | 220,000 |
| Data per Day | 800 GB |
| Mean Processing Time | 392 ms |
| API Success Rate | 99.999% |
Large (Single VM)
- VM Type: F32s_v2
- Resources: 32 vCores, 64 GB RAM, 13 Engines
| Metric | Value |
|---|
| Files per Hour | 13,800 |
| Data per Hour | 50 GB |
| Files per Day | 331,400 |
| Data per Day | 1.2 TB |
| Mean Processing Time | 260 ms |
| API Success Rate | 99.999% |
OVA vs Kubernetes: Key Differences
- The OVA uses RKE; Kubernetes control plane resources are shared with workloads.
- Kubernetes multi-node clusters require shared storage, introducing I/O overhead.
- OVA delivers faster inter-service network speeds as all services run on a single machine.
- Kubernetes offers better scalability but has slightly more orchestration and storage overhead.
