[Embench] New metrics for Embench results - Base/Peak and Speed/Rate

Ray Simar ray.simar at rice.edu
Fri Mar 5 02:20:53 CET 2021


Hi Ofer,

Thanks for the ideas.  I was wondering if we might be able to calculate these measures, at least in part, from the raw measures.  I do like the idea of a single top line number as the team is doing now.  But maybe if we offered the right set of raw measurements people could calculate these additional measures as they needed.

Thoughts?

All the best,
Ray

> On Mar 4, 2021, at 3:17 AM, Ofer Shinaar <Ofer.Shinaar at wdc.com> wrote:
> 
> Hi Dave,
> Its is more an enhancement and less a “problem-solving.”
>  
> For base/peak, we can also allow vendors to publish their best result with compiler tweaks, along with the current “base” typical results. 
> After all, riscv and arm compiler are not the same; for example, we do provide –msave-restore for RV, but we do not give that ARM because it is just a basic flag that we “must use”.
> Peak will allow you to drive more flags and expose where we can get with each compiler per target.
>  
> For Rate/Speed:
> We only publish Speed results, and some targets want to see IPC/throughput as well. Give the “Rate” option will solve that.
>  
> Thanks,
> Ofer
>  
>  
>  
> From: David PATTERSON <pattrsn at cs.berkeley.edu> 
> Sent: Thursday, 4 March 2021 03:44
> To: Ofer Shinaar <Ofer.Shinaar at wdc.com>
> Cc: embench at lists.librecores.org
> Subject: Re: [Embench] New metrics for Embench results - Base/Peak and Speed/Rate
>  
> What is the problem you're trying to solve with the new metrics?
> 
> Dave
>  
> On Wed, Mar 3, 2021 at 3:47 AM Ofer Shinaar <Ofer.Shinaar at wdc.com <mailto:Ofer.Shinaar at wdc.com>> wrote:
> Hello all,
> I would like to bring to the forum a suggestion on including more results metrics as we have on SPEC
>  
> 1.       Include metric fo Base and Peak:
> 
> -          SPEC url or explanation: https://www.spec.org/cpu2017/Docs/overview.html#Q16 <https://www.spec.org/cpu2017/Docs/overview.html#Q16>
> -          Currently, we can say we only have “Base”, on IoT results.
> 
> -          Having “Peak” will give us a possibility to expose more compiler flags that can boost performance, and we can publish results + links to tools that can “find the best set of compiler flags”
> 
> Example for work done by Craig Blackmore (from Embecosm): 
> 
> o   https://github.com/craigblackmore/opentuner <https://github.com/craigblackmore/opentuner>
> o   https://www.groundai.com/project/automatically-tuning-the-gcc-compiler-to-optimize-the-performance-of-applications-running-on-embedded-systems/2 <https://www.groundai.com/project/automatically-tuning-the-gcc-compiler-to-optimize-the-performance-of-applications-running-on-embedded-systems/2>
> 2.        Include metric for Speed and Rate
> 
> -          SPEC url for explanation: https://www.spec.org/cpu2017/Docs/overview.html#Q15 <https://www.spec.org/cpu2017/Docs/overview.html#Q15>
> -          On the embedded space, sometimes throughput is more important than “time”. Those days we can start seeing IoT cores /MCUs that have a multithreaded design.
> Also, we see emerging multithreading designs for RISCV. On those designs rate for throughput will be more important than Time.
> 
>  
> Thoughts/comments?
>  
> Thanks,
> Ofer
>  
>  
>  
>  
> Ofer Shinaar
> Senior Manager, R&D Engineering – Firmware & Toolchain, CTO Group
>  
> Western Digital®
> Israel, Migdal Tefen 24959, P.O Box 3
> Email: Ofer.shinaar at wdc.com <mailto:Ofer.shinaar at wdc.com>
> Office: +972-4-9078783
> Mobile: +972-52-2836160
>  
>  
> -- 
> Embench mailing list
> Embench at lists.librecores.org <mailto:Embench at lists.librecores.org>
> https://lists.librecores.org/listinfo/embench <https://lists.librecores.org/listinfo/embench>-- 
> Embench mailing list
> Embench at lists.librecores.org
> https://lists.librecores.org/listinfo/embench

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.librecores.org/pipermail/embench/attachments/20210304/35911db1/attachment.htm>


More information about the Embench mailing list