Deployment-Performance-Comparison
Overview
The provided performance data includes metrics for two different versions:
- New CDR Platform - VM Deployment (New OVA)
- New CDR Platform - Managed Cloud Deployment (Cloud)
The data presented contains results for two user scenarios: 5 users and 10 users for all environments, all utilising 5 Glasswall Embedded engines for a range of file types, all were run for a duration of 5 mins each.
Average Iteration Duration:
For 5 Users:
The comparison of average iteration durations for the New OVA and Cloud deployment under the 5-user scenario reveals significant performance differences across various file types. The New OVA consistently demonstrates lower average iteration durations compared to the Cloud deployment, indicating its overall faster processing capabilities for 5 users.
One noteworthy observation is that the New OVA outperforms the Cloud deployment, especially for smaller file sizes, where it shows significantly lower average iteration durations. However, when processing large files, the performance differences between the New OVA and the Cloud deployment might be less noticeable.
The variation in average iteration durations across different file types suggests that the performance of each version is influenced by the nature and size of the files being processed for 5 users. While the New OVA generally excels in efficiency, especially for smaller file sizes, the presence of large files could impact the overall results and moderate the performance differences observed in the comparison for 5 users.
It's important to consider specific file types and their sizes when evaluating the performance of different versions to make informed decisions about the optimal choice for specific deployment needs. The New OVA's consistently lower iteration durations make it a promising option for faster processing, while the Cloud deployment could present competitive results, particularly for large file processing under the 5-user scenario.
For 10 Users:
The comparison of average iteration durations for the New OVA and Cloud deployment under the 10-user scenario reveals distinct differences in their performance across various file types. The New OVA consistently demonstrates lower average iteration durations compared to the Cloud deployment, indicating its overall faster processing capabilities for 10 users.
Similar to the 5-user scenario, the Cloud deployment shows moderate performance across most file types. However, the impact of large files becomes more pronounced when analysing the 10-user data. Large files skew the average iteration duration results for 10 users, requiring more processing time and resources.
While the New OVA remains the most efficient option overall for 10 users, especially for smaller file sizes, the presence of large files in the dataset could moderate the performance differences observed in the comparison. In certain scenarios, the Cloud deployment may show relatively better performance compared to the New OVA when processing large files for 10 users.
In conclusion, evaluating performance based on specific file types and user scenarios is crucial to making informed decisions about the optimal version for different deployment needs. The New OVA consistently demonstrates faster performance, especially for smaller file sizes, under the 10-user scenario. However, the impact of large files on the overall results should be considered, and the Cloud deployment may present more competitive results for certain file types when handling larger user loads.
Failures:
For 5 Users:
The New OVA records failures in some instances, particularly with small audio files. The Cloud deployment reports no recorded failures for any of the file types, indicating its stability under the 5-user scenario.
For 10 Users:
The New OVA's failure pattern remains consistent with the 5-user scenario, with transient issues with certain file types. The Cloud deployment maintains its stability, with no recorded failures across all file types under the 10-user scenario.
Comparison
5 Users - Most Performant Results:
Average Iteration Duration:
-
New OVA:
- File Type: Small audio (<100kb)
- Average Duration: 47 milliseconds
- Iterations: 32167
- Performance: The New OVA performs exceptionally well for small audio files, with an average iteration duration of just 47 milliseconds. This makes it the most efficient option for this file type under the 5-user scenario.
-
Cloud:
- File Type: Large audio (<10Mb)
- Average Duration: 1129 milliseconds
- Iterations: 1331
- Performance: Among the two versions, the Cloud deployment performs best for large audio files, with an average iteration duration of 1129 milliseconds. It is the most efficient choice for this specific file type under the 5-user scenario.
Failures:
-
New OVA:
- File Type: Small Audio (<100kb)
- Failures: 3
- Performance: Despite its efficiency in terms of iteration duration, The New OVA experiences failures with small audio file types, making it a trade-off between speed and reliability for this scenario.
-
Cloud:
- No recorded failures for any file type.
- Performance: The Cloud deployment exhibits stability under the 5-user scenario, with no recorded failures for any file type.
10 Users - Most Performant Results:
Average Iteration Duration:
-
New OVA:
- File Type: Small Audio
- Average Duration: 47 milliseconds
- Iterations: 39,281
- Performance: The New OVA maintains its exceptional performance for small audio files under the 10-user scenario, with an average iteration duration of 47 milliseconds.
-
Cloud:
- File Type: All file types
- Performance: The Cloud deployment performs consistently well across all file types, showing no recorded failures under the 10-user scenario. At 10 Users, there is no instance where the Cloud deployment outperforms the New OVA.
Summary:
Across both 5 and 10 users, the New OVA consistently demonstrates better average iteration durations compared to the Cloud deployment for most file types. The New OVA's performance is remarkable in terms of efficiency and speed, but it also shows failures in some cases, particularly under the 5-user scenario. The Cloud deployment proves to be a stable version with no recorded failures across all file types in both user scenarios.
Under the 5-user scenario, the New OVA demonstrates the best average iteration duration for small audio files, while the Cloud deployment performs best for large audio files. Among the two versions, the New OVA remains the most performant for small audio files even under the 10-user scenario. The Cloud deployment maintains its stability with no recorded failures for any file type at this user load.
Depending on the specific use case, organisations may prioritise the New OVA for the fastest performance in certain scenarios or choose the Cloud deployment for its reliability and consistent results across all file types.
Conclusion:
The extended comparison of performance data for the New OVA and Cloud deployment across 5 and 10 users reaffirms that the New OVA generally outperforms the Cloud deployment in terms of average iteration durations. However, the New OVA also exhibits failures in some cases, which could impact its overall reliability. The Cloud deployment, on the other hand, maintains its stability and shows consistent performance across both user scenarios. The decision on the best version depends on specific use cases, considering factors such as iteration durations, stability, and failure rates under different user loads and deployment capabilities.