[Embench] New metrics for Embench results - Base/Peak and Speed/Rate

Ofer Shinaar Ofer.Shinaar at wdc.com
Thu Mar 4 10:17:51 CET 2021


Hi Dave,
Its is more an enhancement and less a “problem-solving.”

For base/peak, we can also allow vendors to publish their best result with compiler tweaks, along with the current “base” typical results.
After all, riscv and arm compiler are not the same; for example, we do provide –msave-restore for RV, but we do not give that ARM because it is just a basic flag that we “must use”.
Peak will allow you to drive more flags and expose where we can get with each compiler per target.

For Rate/Speed:
We only publish Speed results, and some targets want to see IPC/throughput as well. Give the “Rate” option will solve that.

Thanks,
Ofer



From: David PATTERSON <pattrsn at cs.berkeley.edu>
Sent: Thursday, 4 March 2021 03:44
To: Ofer Shinaar <Ofer.Shinaar at wdc.com>
Cc: embench at lists.librecores.org
Subject: Re: [Embench] New metrics for Embench results - Base/Peak and Speed/Rate

What is the problem you're trying to solve with the new metrics?

Dave

On Wed, Mar 3, 2021 at 3:47 AM Ofer Shinaar <Ofer.Shinaar at wdc.com<mailto:Ofer.Shinaar at wdc.com>> wrote:
Hello all,
I would like to bring to the forum a suggestion on including more results metrics as we have on SPEC


1.       Include metric fo Base and Peak:

-          SPEC url or explanation: https://www.spec.org/cpu2017/Docs/overview.html#Q16

-          Currently, we can say we only have “Base”, on IoT results.

-          Having “Peak” will give us a possibility to expose more compiler flags that can boost performance, and we can publish results + links to tools that can “find the best set of compiler flags”

Example for work done by Craig Blackmore (from Embecosm):

o   https://github.com/craigblackmore/opentuner

o   https://www.groundai.com/project/automatically-tuning-the-gcc-compiler-to-optimize-the-performance-of-applications-running-on-embedded-systems/2

2.        Include metric for Speed and Rate

-          SPEC url for explanation: https://www.spec.org/cpu2017/Docs/overview.html#Q15

-          On the embedded space, sometimes throughput is more important than “time”. Those days we can start seeing IoT cores /MCUs that have a multithreaded design.
Also, we see emerging multithreading designs for RISCV. On those designs rate for throughput will be more important than Time.

Thoughts/comments?

Thanks,
Ofer




Ofer Shinaar
Senior Manager, R&D Engineering – Firmware & Toolchain, CTO Group

Western Digital®
Israel, Migdal Tefen 24959, P.O Box 3
Email: Ofer.shinaar at wdc.com<mailto:Ofer.shinaar at wdc.com>
Office: +972-4-9078783
Mobile: +972-52-2836160


--
Embench mailing list
Embench at lists.librecores.org<mailto:Embench at lists.librecores.org>
https://lists.librecores.org/listinfo/embench
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.librecores.org/pipermail/embench/attachments/20210304/cd6572bf/attachment.htm>


More information about the Embench mailing list