Ultimate
performance

In June 2019 we've conducted series of test to determine Higson efficiency and compare it to leading business rules management system (BRMS) for Java applications. Higson proved to be hyper efficient:

100,000 complex calculations in around 90 seconds
Less than 1 millisecond per calculation
Higson was between 62 to 119 times faster than other BRMS in all tests

Please read detailed case study below for more info. You can also request full case study.

Methodology

Laptop specification

  • HP Zbook 15 G3
  • Windows 10 Pro
  • Intel i7-6700HQ @ 2.60 GHz
  • 16 GB RAM

Test definitions

1) Average Time for 100,000 calculations

  • 2 forks
  • 1 thread
  • Ten 1-second warmups
  • Twenty 1-second iterations

2) Average Time for 1,000 calculations

  • 2 forks
  • 1 thread
  • Ten 1-second warmups
  • Twenty 1-second iterations

3) Throughput

  • 2 forks
  • 1 thread
  • Ten 1-second warmups
  • Twenty 1-second iterations

Algorithm

General algorithm consists of 3 steps:

  • Get Qx factor from Qx decision table (1 output value based on 2 input conditions).
  • Get Rate adjustment factor from decision table (1 output value based on 4 input conditions).
  • Multiply Qx by Rate adjustment.

However, there are 2 test cases that differs subtly:

  1. All test inputs are in decision tables. Qx decision table has 200 rows, Rate adjustment decision table has 18,000 rows.
  2. Not all test inputs are in decision tables, therefore there are „if nothing else match” rows for such cases. Qx decision table has 200 rows, Rate adjustment decision table has 8,400 rows.

Results

Average Time, 100,000 calculations, test case 1

Label: Hyperon vs other BRMS

  • Total time: 1m 27s vs 14m 52s
  • Score (sec/op): 0.213 vs 13.160
  • Error: 0.001 vs 0.337
  • Minimum: 0.210 vs 12.240
  • Average: 0.213 vs 13.160
  • Maximum: 0.220 vs 15.017
  • stdev: 0.002 vs 0.6

Higson, fork 1

Hyperon, heap usage, 100,000 calculations, fork 1
Hyperon, classes loaded, 100,000 calculations, fork 1
Hyperon, CPU usage, 100,000 calculations, fork 1

Higson, fork 2

Screens available on request

Other BRMS, fork 1

Other BRMS, heap usage, 100,000 calculations, fork 1
Other BRMS, classes loaded, 100,000 calculations, fork 1
Other BRMS, CPU usage, 100,000 calculations, fork 1

Other BRMS, fork 2

Screens available on request

Results comparison

  • Memory/
    Largest Heap Usage: Higson 225 MB vs other BRMS: 3 GB.
    Higson uses 93% less memory.
    Higson has got also far more Garbage Collector invocations that helps keeping low memory consumption.
  • Classes/
    Higson 4,000 vs other BRMS 92,400.
    Hyperon loads 96% fewer classes.
  • CPU/
    Startup: Higson 40% vs other BRMS 100%.
    During calculations: Hyperon 15% vs other BRMS 25-30%.
    Other BRMS graph is also far more volatile, constantly jumping between 40% and 15% CPU load comparing to smooth Higson graph that stays around 15%.

Test shows, that Hyperon is around 62 times faster on average.

Average Time, 100,000 calculations, test case 2

Label: Higson vs other BRMS

  • Total time: 1m 29s vs 13m 26s
  • Score (sec/op): 0.199 vs 12.254
  • Error: 0.002 vs 0.350
  • Minimum: 0.195 vs 11.319
  • Average: 0.199 vs 12.254
  • Maximum: 0.222 vs 14.153
  • stdev: 0.004 vs 0.622

Screens available on request

Results comparison

  • Memory/
    Largest Heap Usage: Higson 190 MB vs other BRMS: 3.25 GB.
    Higson uses 94% less memory.
    Higson has got also far more Garbage Collector invocations that helps keeping low memory consumption.
  • Classes/
    Hyperon 4,000 vs other BRMS 48,500.
    Hyperon loads 92% fewer classes.
    The number of Higson loaded classes is constant. It doesn’t depend on number of decision tables, rows etc.
  • CPU/
    Startup: Higson 35% vs other BRMS 100%.
    During calculations: Higson 15% vs other BRMS 25-30%.
    Other BRMS graph is also far more volatile, constantly jumping between 40% and 15% CPU load comparing to smooth Higson graph that stays around 15%.

Test case 2 shows similar test results to the first test case, Higson is around 62 times faster on average.

Average Time, 1,000 calculations, test case 1

Label: Higson vs other BRMS

  • Total time: 1m 25s vs 2m 10s
  • Score (sec/op): 0.002 vs 0.239
  • Error: 0.001 vs 0.019
  • Minimum: 0.002 vs 0.196
  • Average: 0.002 vs 0.239
  • Maximum: 0.003 vs 0.338
  • stdev: 0.001 vs 0.033

Higson, fork 1

Hyperon, heap usage, 1,000 calculations, fork 1
Hyperon, classes loaded, 1,000 calculations, fork 1
Hyperon, CPU usage, 1,000 calculations, fork 1

Higson, fork 2

Screens available on request

Other BRMS, fork 1

Other BRMS, heap usage, 1,000 calculations, fork 1
Other BRMS, classes loaded, 1,000 calculations, fork 1
Other BRMS, CPU usage, 1,000 calculations, fork 1

Other BRMS, fork 2

Screens available on request

Results comparison

  • Memory/
    Largest Heap Usage: Higson 180 MB vs other BRMS: 1.25 GB.
    Higson uses 86% less memory.
    Other BRMS memory usage looks better than in 100,000 calculation tests, but it is still far more than Higson’s results.
  • Classes/
    Higson 4,000 vs other BRMS 92,400.
    Higson loads 96% fewer classes.
  • CPU/
    Startup: Higson 40% vs other BRMS 100%.
    During calculations: Higson 15% vs other BRMS 25-30%.
    Other BRMS graph looks much better than in previous tests (not so many load peaks) but it is still worse than Higson results.

Tests on smaller calculation number shows even bigger gap, Higson is around 119.5 times faster on average than other BRMS.

Average Time, 1,000 calculations, test case 2

Label: Higson vs other BRMS

  • Total time: 1m 21s vs 1m 38s
  • Score (sec/op): 0.002 vs 0.190
  • Error: 0.001 vs 0.009
  • Minimum: 0.002 vs 0.164
  • Average: 0.002 vs 0.190
  • Maximum: 0.002 vs 0.224
  • stdev: 0.001 vs 0.016

Screens available on request

Results comparison

  • Memory/
    Largest Heap Usage: Higson 180 MB vs other BRMS: 800 MB.
    Hyperon uses 78% less memory.
  • Classes/
    Higson 4,000 vs other BRMS 48,500.
    Higson loads 92% fewer classes.
  • CPU/
    Startup: Higson 35% vs other BRMS 80%.
    During calculations: Higson 15% vs other BRMS 20%.

Test case 2 shows smaller gap, but Higson is still around 95 times faster on average than other BRMS.

Throughput, test case 1

Label: Higson vs other BRMS

  • Total time: 2m 25s vs 2m 58s
  • Score (sec/op): 453,242 vs 5,243
  • Error: 3,793 vs 298
  • Minimum: 430,310vs 3,385
  • Average: 453,242 vs 5,243
  • Maximum: 470,824 vs 6,920
  • stdev: 9,928 vs 780

Higson, fork 1

Hyperon, heap usage, throughput, fork 1
Hyperon, classes loaded, throughput, fork 1
Hyperon, CPU usage, throughput, fork 1

Higson, fork 2

Screens available on request

Other BRMS, fork 1

Other BRMS, heap usage, throughput, fork 1
Other BRMS, classes loaded, throughput, fork 1
Other BRMS, CPU usage, throughput, fork 1

Other BRMS, fork 2

Screens available on request

Results comparison

  • Memory/
    Largest Heap Usage: Higson 200 MB vs other BRMS: 1.5 GB.
    Higson uses 87% less memory.
  • Classes/
    Higson 4,000 vs other BRMS 92,300.
    Higson loads 96% fewer classes.
  • CPU/
    Startup: Higson 50% vs other BRMS 95%.
    During calculations: Higson 12% vs other BRMS 15-30%.

Tests one more time show how much Higson is faster than other BRMS, in the first test case around 86 times faster on average. It is worth noting that with each iteration other BRMS performance was increasing, score from last iteration is over 100% bigger than score from the first one. JIT and all its class optimization is probably the source of performance boost.

Throughput, test case 2

Label: Higson vs other BRMS

  • Total time: 2m 21s vs 2m 30s
  • Score (sec/op): 471,624 vs 5,706
  • Error: 4,553 vs 402
  • Minimum: 451,285 vs 4,073
  • Average: 471,624 vs 5,706
  • Maximum: 489,950 vs 8,525
  • stdev: 11,916 vs 1,053

Screens available on request

Results comparison

  • Memory/
    Largest Heap Usage: Higson 180 MB vs other BRMS: 800 MB.
    Hyperon uses 78% less memory.
  • Classes/
    Higson 4,000 vs other BRMS 48,400.
    Hyperon loads 92% fewer classes.
  • CPU/
    Startup: Higson 35% vs other BRMS 95%.
    During calculations: Higson 12% vs other BRMS 15-20%.

There is no big differences comparing results from both test cases. In test case 2, Higson is around 90 times faster on average. It is worth noting that with each iteration other BRMS performance was increasing, score from last iteration is over 100% bigger than score from the first one. JIT and all its class optimization is probably the source of performance boost.

Our customers use Higson to develop innovative products and generate new revenue streams.

These are their success stories