[Embench] Floating point accuracy

Jeremy Bennett jeremy.bennett at embecosm.com
Fri Oct 4 16:17:40 CEST 2019


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/10/19 11:16, Jon Taylor wrote:
> Relatedly, I’ve also discovered the nbody test fails when using
> Arm’s fast math library.
> 
> In the verification function, this performs explicit != tests
> against the expected values, rather than testing whether the fabs
> difference is within a certain range.
> 
> If I check the fabs error, it’s always <1e-19.
> 
> It would be nice if we could find a value of acceptable FP error
> for all the FP tests and have a consistent checking mechanism for
> the results. The use of the fast-math library is part of the
> compiler flags, so will be recorded as part of the run.

Hi Jon,

Thanks for pick this up. The code error was also picked up by another
use in https://github.com/embench/embench-iot/pull/12. I've asked if
he'd like to offer a generalized fix for all the floating point in the
light of this discussion. I agree fabs is the solution. I'm not sure
what the correct value is to use - you might hope there is a function
to give you that value.

Floating point usage is supposed to be minimal in this version of
Embench, but you can't completely exclude it from real programs. But
we should make it consistent and we shouldn't make it too harsh in its
accuracy specifications. That can be for a floating point version of
Embench in the future.

Best wishes,


Jeremy
> 
> 
> 
> Regards,
> 
> 
> 
> Jon
> 
> 
> 
> *From:*Embench <embench-bounces at lists.librecores.org> *On Behalf Of
> *Jon Taylor *Sent:* 02 October 2019 11:21 *To:*
> embench at lists.librecores.org *Cc:* nd <nd at arm.com> *Subject:*
> [Embench] Floating point accuracy
> 
> 
> 
> I’m still testing the performance of Embench on various Arm
> devices.
> 
> 
> 
> During testing, I encountered the ST (stats) test failing. This
> performs various double precision floating point operations.
> 
> 
> 
> When using Arm’s fast math library on a device with no hardware
> floating point support, the test fails.
> 
> (see 
> https://developer.arm.com/docs/dui0774/i/compiler-command-line-options/-ffast-math-fno-fast-math)
>
> 
> 
> 
> On debugging this further, this is due to a comparison mismatch in
> the verify_benchmark function.
> 
> If I modify the code with some debug printf statements:
> 
> printf ("first check: %8.8e\n", fabs (expSumA - SumA));
> 
> printf ("second check: %8.8e\n", fabs (expSumB - SumB));
> 
> printf ("third check: %8.8e\n", fabs (expCoef - Coef));
> 
> printf ("Coef %8.8e, expCoef %8.8e\n", Coef, expCoef);
> 
> 
> 
> return (fabs (expSumA - SumA) < 1.0e13)
> 
> && (fabs (expSumB - SumB) < 1.0e-13) && (fabs (expCoef - Coef) < 
> 1.0e-17);
> 
> 
> 
> I see the following output:
> 
> first check: 0.00000000e+00
> 
> second check: 0.00000000e+00
> 
> third check: 2.22044605e-16
> 
> Coef 9.99900055e-01, expCoef 9.99900055e-01
> 
> 
> 
> (It also looks like there’s a code error here, since the first 
> comparison is <1e13, the second is <1e-13. Shouldn’t the first also
> be 1e-13?)
> 
> 
> 
> The third check result about causes the test to fail since the
> third comparison looks for an error of less than 1e-17. Is there a
> particular reason for a tighter threshold on the third check? In
> fact the absolute difference is less than 1e-17, it’s just that the
> comparison is done quickly and introduces some error. Can it be
> relaxed to 1e-13 to match the other comparisons? I don’t know where
> those numbers have been derived from.
> 
> 
> 
> Regards,
> 
> 
> 
> Jon
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 


- -- 
Tel: +44 (1590) 610184
Cell: +44 (7970) 676050
SkypeID: jeremybennett
Twitter: @jeremypbennett
Email: jeremy.bennett at embecosm.com
Web: www.embecosm.com
PGP key: 1024D/BEF58172FB4754E1 2009-03-20
-----BEGIN PGP SIGNATURE-----

iEYEARECAAYFAl2XVH4ACgkQvvWBcvtHVOFpTQCgkTDU4qMRNbTaFqm+QeUhjjUj
7oAAniINMTK8IBmiRTad8dvlKD4suzaq
=zXSN
-----END PGP SIGNATURE-----



More information about the Embench mailing list