[Embench] Use of assert

Jeremy Bennett jeremy.bennett at embecosm.com
Thu Aug 29 19:17:41 CEST 2019


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 29/08/19 17:34, Jeremy Bennett wrote:
> On 29/08/19 09:16, Jon Taylor wrote:
>> The nettle-aes and nettle-sha256 tests both have assert
>> statements in the code.

Pull request to fix this here:

https://github.com/embench/embench-iot/pull/11

This only affects the two nettle crypto benchmarks. Their code size is
slightly reduced as a consequence. The references to assert in
sglib-combined are dummies.

I'll leave this a day or two for comments, then if there are no
objections, I'll merge it.

Best wishes,


Jeremy

> 
>> Having in the other thread suggested that it’s important to 
>> include libraries, I think assert is a different case as it
>> isn’t really a core part of the benchmark functionality. In the
>> benchmark situation, the stimulus will be constant, and the
>> asserts won’t be triggered. However if the code is built with
>> real libraries (as opposed to dummies), the library code for
>> assert is a significant size (~20k for RISC-V, ~12k for Arm vs a
>> test size of ~3k). I propose they are removed – their use is
>> primarily debug/error handling and I’m not sure that has value
>> within Embench.
> 
> Hi Jon,
> 
> I agree with you on this. I propose doing the same as we have with
> a number of other problematic functions (e.g. rand), and providing
> an Embench version in the support library. All that matters is that
> the benchmark should fail (i.e. exit with a non-zero return code)
> if assert is triggered, so this can be much simpler than the full
> library assert.
>> Related to the library and size thread, assert also creates a 
>> challenge in calculating overhead, since assert uses some code
>> in common with printf. This means that if printf is used in
>> other common code (eg UART output), the size differential between
>> the nettle tests and the empty test does not give an accurate
>> size calculation. If library size is included in the overall
>> benchmark measurement, then we need to try and make the common
>> library code as consistent as possible. Multiple dummy/empty
>> tests might be another approach, but that adds a maintenance
>> overhead.
> 
> Yes - I've run into this before. My proposed fix will deal with
> this. Once I have the patch, I'll post it here.
>> I’m happy to be challenged on this if I’ve missed a good reason 
>> for leaving them in place.
> 
> Thanks for spotting the issue.
> 
> Best wishes,
> 
> 
> Jeremy
> 
> 

- -- 
Tel: +44 (1590) 610184
Cell: +44 (7970) 676050
SkypeID: jeremybennett
Twitter: @jeremypbennett
Email: jeremy.bennett at embecosm.com
Web: www.embecosm.com
PGP key: 1024D/BEF58172FB4754E1 2009-03-20
-----BEGIN PGP SIGNATURE-----

iEYEARECAAYFAl1oCLMACgkQvvWBcvtHVOE5OgCeNDm5/F2aiazIBFOulA1Khxfv
JCQAn3kjbOYkcK/9v2t4wVZ3/CmsxfVk
=FIRx
-----END PGP SIGNATURE-----



More information about the Embench mailing list