Frequently Asked Questions
When submitting a request to the CDR Platform, what steps does a file go through?
- File is sent to the REST API
- The file is stored on a persisted volume within the cluster
- A message is sent to the engine service to process the file
- The engine service receives the message to process a file
- That file is read from the persisted volume and is then processed by the Glasswall engine
- The clean file and analysis report are then stored in the persisted volume
- A message is sent to the report extractor so the engine report can be generated
- A message is then sent back to the API from the engine service
- The API receives the message from the engine and reads either the clean file, report or both from the persisted volume and generates the correct response to the client.
- A message is then sent to the report extractor from the API
- The report extractor receives this message and generates a metrics from the analysis file and the API
- Metric data is then generated
- Finally the original file, clean file and analysis report are deleted from the persisted volume
What types of files can Glasswall Halo process?
Glasswall Halo supports a wide range of file types including PDFs, Office documents (Word, Excel, PowerPoint), images, and more. It ensures that all processed files are safe and retain their original functionality.
How are files stored
Original files, clean files and analysis reports are stored in a persisted volume backed by either Azure File Share (Azure) or Amazon Elastic Block Store (AWS).
Reports can be stored in Azure Blob Storage (Azure), S3 (AWS) or a persisted volume
How is the system configured and deployed?
Glasswall Halo can be deployed in a Kubernetes environment, leveraging Helm charts for easy setup and configuration. Detailed deployment instructions are provided in the documentation. NOTE: Mention the OVA
How does Glasswall Halo handle file types that it cannot process?
_NOTE: Answer generic
How long does the file persist in shared storage?
Original files, clean files and analysis reports are stored until the report generation is complete
What are the max nested levels of archives
5 levels of nested archives are supported by the CDR platform
What are the archive types
The following archive types are supported: Zip, Tar, Zip, 7Zip, Rar
What are the Content Management Flags
The default value of all flags is allow - 0, other supported actions are sanitise- 1 or disallow - 2.
What does the code mean in the error response
Within the error response, we return a "code" this indicates a particular scenario. Error codes will start with the HTTP response code and then be appended with a number to indicate the specific error. For more information, please refer to our API documentation
Can I amend the replica count of the rebuild pods to increase performance?
This is possible. The current configuration includes headroom for workloads that are more challenging than the average file. Whilst it may well be possible to increase the replicas count and to observe improved performance, the consequence might be increased errors if concurrent load depleted available memory.
Can I use a larger node size to improve performance?
This is possible and would therefore allow the replicas count per node to be amended without necessarily experiencing any scarcity of compute resources. An obvious downside of assigning more workloads to a single node is the size of the blast radius should a node fail. Increasing the node count may be achieve the same objective with less risk.
Why are you allocating a maximum memory allocation that is larger than 1 GB if that’s the maximum file size?
The CDR process needs to create an intermediate representation of the original file. This may be much larger than the original file. We have assigned reserve values to ensure that multiple files
Do you have Helm Charts for bring your own cluster or other cloud environments other than AKS?
This is currently on our road map for Q1 2023
How does licence management work?
Glasswall provides the customer with the entitlement to process a quanity of files or an amount of data each day. Today Glasswall does not impose a technical limitation on over consumption of the licence entitlement. However, over-consumption would represent a contractual violation of the licencing agreement. Glasswall will be introducing a licence management layer into the CDR Platform which imposes limitation of use according to licence entitlements.
Does Glasswall limit the number of Engine calls?
Today there is no limitation on the number of Engine call that may be made in a given time frame, beyond the base configuration that is provided in the Helm Charts. In the future a licence management mechanism may throttle throughput to pro-rate hourly capacity with the daily peak capacity that has been purchased.
What is the maximum file size that you support?
1 GB. There may be examples of files that are impossible to CDR without allocating very large amounts of memory. CDR in general requires that an intermediate representation of a file is created. The memory required for the model can be a few times larger than the original file. Depending on the structure of the underlying file, memory requirements may vary.
Why does Glasswall use some 3rd party services?
Glasswall is the world's premier CDR solution provider. Glasswall is very cautious about the use of 3rd party software dependencies general. We employ a mixture of static analysis, software composition analysis and infrastructure as code automated scanning to ensure that the 3rd party components that we use, do not introduce security flaws. Most software development today, incorporates 3rd party libraries into software applications and Glasswall is no different in this regard. We are however explicit about what these dependencies are and have zero tolerance for insecure software.
Does Glasswall provide a software bill of materials (SBOM) for the deployment?
Glasswall generates SBOMs for the services that comprise the CDR Platform deployment. These can be made available on request.
Do you only support v3 of Helm?
That is correct.
What information is sent back to Glasswall from my deployment?
The CDR Platform is designed to be run in an air-gapped environment. No data is sent to Glasswall. Glasswall reserves the right to request summary log information to verify licence conformance, but data does not leave the environment without intervention by the system owner.
Why do some of your services use Alpine as the base OS?
In the future, most of the services will use a hardened Alpine base image. Where possible we implement CIS (Centre for Internet Security) guidelines for hardening. Alpine provides a small Linux distribution with a minimised attack surface which is attractive from both a performance and security perspective.
What does OOM mean?
When an application uses more memory than the memory that is allocated, it will see Out of memory (OOM) error.
What happens when an engine pod errors with OOM?
Every pod in Kubernetes is allocated with a minimum and maximum memory and when the engine pod uses more than the maximum memory allocation while processing a file, it will get OOM error and pod gets killed. When this happens, the kubelet will automatically restart the pod with an exponential back-off delay (10s, 20s, 40s, …), that is capped at five minutes. Once a container has executed for 10 minutes without any problems, the kubelet resets the restart backoff timer for that container.
How do I spot an OOM?
When a pod has OOM errors, the status of the pod will be OOMKilled or CrashLoopBackOff and the logs of the pod might contain seg fault error
What do I do if there are too many messages on the queue and there are no engine pods?
When the CDR Platform is overloaded with huge number of API requests the queue will be built up with the messages and 429 status code will be seen. The messages will be eventually consumed by the system and it will come back to normal state. If it takes long time to process the messages in the queue and if you wish to bring the system back to normal state and also ok with loosing messages in the RabbitMQ queue, purge the queue to delete all messages in the queue.
kubectl exec -ncdrplatform -it rabbitmq-server-0 -c rabbitmq -- rabbitmqctl purge_queue engine-request-queue
If I seen OOM errors on a regular basis what do I do?
When OOM errors are seen on a regular basis it means resources allocated to the egine pods are too less to process the files. To reduce the OOM errors, increase the memory request and limits of the engine service in the kubernetes.
How is the solution monitored and maintained?
Monitoring tools integrated with Kubernetes provide insights into the system’s performance, resource usage, and health. Regular maintenance and updates are managed through Kubernetes, ensuring the system remains secure and efficient.
What are the security measures in place to protect the data processed by Glasswall Halo?
Glasswall Halo employs encryption, secure communication protocols, and strict access controls to protect data. Additionally, the CDR process itself ensures that no malicious content passes through.
What support and resources are available for implementing and managing the solution?
Glasswall provides comprehensive documentation, support services, and training materials to help with the implementation and management of the solution. Support channels include online resources, customer support teams, and community forums.
What is the expected latency for file processing?
The latency for file processing depends on the file size, complexity and type. However, Glasswall Halo is designed to process files quickly to ensure minimal delay in data transmission.
How is the performance and scalability managed?
Kubernetes allows Glasswall Halo to automatically scale based on the load. Using KEDA it can add or remove instances of the CDR service to handle varying amounts of file traffic efficiently. For me information on KEDA please see here
QUESTION ABOUT DASHBOARDS
ALAN TO ADD QUESTIONS AND ANSWERS HERE