[Embench] Use of assert

Jon Taylor Jon.Taylor at arm.com
Thu Aug 29 10:16:22 CEST 2019

The nettle-aes and nettle-sha256 tests both have assert statements in the code.

Having in the other thread suggested that it's important to include libraries, I think assert is a different case as it isn't really a core part of the benchmark functionality. In the benchmark situation, the stimulus will be constant, and the asserts won't be triggered. However if the code is built with real libraries (as opposed to dummies), the library code for assert is a significant size (~20k for RISC-V, ~12k for Arm vs a test size of ~3k). I propose they are removed - their use is primarily debug/error handling and I'm not sure that has value within Embench.

Related to the library and size thread, assert also creates a challenge in calculating overhead, since assert uses some code in common with printf. This means that if printf is used in other common code (eg UART output), the size differential between the nettle tests and the empty test does not give an accurate size calculation. If library size is included in the overall benchmark measurement, then we need to try and make the common library code as consistent as possible. Multiple dummy/empty tests might be another approach, but that adds a maintenance overhead.

I'm happy to be challenged on this if I've missed a good reason for leaving them in place.

Kind regards,

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.librecores.org/pipermail/embench/attachments/20190829/c9a30f39/attachment.htm>

More information about the Embench mailing list