Tips & Tricks

Benchmarking the Performance of the Hyperon Runtime Engine

We created benchmarks measuring the number of calculations the Hyperon Runtime Engine can evaluate per second. Below you will see that the performance of the Hyperon Runtime Engine depends on used configuration.

Benchmark description

In all benchmarks, the same rule was processed. It was the complex rule which calculates insurance OC tariff. 

Hyperon Runtime Engine measurement configurations

  1. Raw Hyperon Runtime Library
  2. Hyperon Runtime Rest module (which contains Hyperon Runtime Library
  • docker container with runtime rest module - no security
  • Runtime Rest module - no security
  • Runtime Rest module - bcrypt security mechanism
  • Runtime Rest module - pbkdf2 security mechanism

For the raw Hyperon Runtime Library test was done with JMH (Java Microbenchamrk Harnes) tool. Other configurations based on REST were done with JMeter.

Benchmark environment

Benchmarks were done on the server with specification:

  • Processor: 2 cores 2.4GHz
  • RAM: 4 GB
  • Hard Disk: SSD
  • Operating system: Debian10

All configurations for the Hyperon Runtime Engine works with the H2 database.

Benchmarks in JMeter were processed with 100 threads(users) and 10000 repetitions. This configuration summary gives 1.000.000 iterations.

Benchmark Results

The best performance was obtained with the raw Hyperon Runtime Library, and the result was 6822,782 operations/second. 

All of the results are presented in the chart below.

Analyzing the data, we came to two conclusions.

Firstly, JSON serialization and deserialization is the primary performance factor. We can see that the fastest Hyperon Runtime REST module is slower, almost five times, than the raw runtime library.

The second conclusion is the security impact in Hyperon Runtime REST modules. We achieved the highest operations per second count with no security. But, if we want to have security, the best option is to use the JWT mechanism.

Bcrypt mechanism is configurable, and we can change the complexity of it. Default complexity is 5, and if the higher complexity number is used, automatically performance is reduced. In the above chart for complexity 10, we can see a performance decrease from 341,4 to 16,1 operations/second.