[Embench] Notes from Embench call 15 June 2020

paolo paolo.savini at embecosm.com
Thu Jul 16 13:29:25 CEST 2020


Hi all,

here is the slide I presented in the June meeting about the comparison
of the speed/size results obtained with embench for GCC 9.2.0 vs GCC
10.1.0 on RISCV (ri5cy) and ARM (cortex-m4).

sorry for the delay on this.

The action about the search of suitable FP benchmarks from BEEBS is in
progress.


Best,

Paolo

On 15/06/2020 17:03, Jeremy Bennett wrote:
> Hi all,
>
> These are the notes from last week's meeting, summarizing actions.
>
> 0. Welcomes
> -----------
>
> Welcome new participants Erin Olson, Nikos Fragoulis, Ray Simar and
> Paul Butcher.
>
> 1. Actions from previous meetings
> ---------------------------------
>
> ACTION: Jeremy Bennett. Update London Meetup slides and make available
> to group.
> - Complete, but needs updating as discussed in agenda item 5 below.
>
> ACTION: Jeremy Bennett to complete work on the embench-iot-results
> respository.
> - Complete, see pull request 3.
>
> ACTION: Paolo Savini to distribute his slide.
> - In progress.
>
> ACTION (continued): Jeremy Bennett to update the documentation with
> more details to make the board easier to find in future.
> - Complete, see pull request 67.
>
> ACTION: David Patterson/Jeremy Bennett. Chase up what is happening for
> Arm TechCon.
> - Complete. Proposal submitted.
>
> ACTION: Paolo Savini to explore suitable BEEBS floating point benchmarks.
> - In progress.
>
> ACTION: Jeremy Bennett. Set up a shared Google doc for people to
> record suggestions.
> - Complete, see
> https://docs.google.com/document/d/1tC0LXZRYeyoSRTM9aXtapzx3sA00vfFcL9QjOxU3vzQ
>
> ACTION: All. Reach out to friends in embedded computing for
> suggestions, including DSP suggestions.
> - In progress.
>
> 2. GitHub pull requests
> -----------------------
>
> From embench-iot:
>
> https://github.com/embench/embench-iot/pull/63
> - Add --baselinedir and --baseline-output options
> - Discussion felt this would be useful, will commit
> - Thanks to Ed  Jones for providing this.
>
> ACTION: Jeremy Bennett to commit the pull request.
>
> https://github.com/embench/embench-iot/pull/64
> - Always log compiler and linker output, even if successful
> - Merged
> - Issue 66 raised to add --quiet and --verbose flags
> - Thanks to Ed Jones for providing this.
>
> https://github.com/embench/embench-iot/pull/67
> - Document how to obtain the baseline platform
> - Merged
> - Thanks to Jeremy Bennett for providing this.
>
> From embench-iot-results:
>
> https://github.com/embench/embench-iot-results/pull/3
> - The rework to just hold Embench 0.5 results
> - Merged as a matter of urgency
> - Tagged embench-0.5
> - Thanks to Jeremy Bennett for providing this.
>
> Thanks to all those who provided patches.
>
> 3. New GitHub issues
> --------------------
>
> From embench-iot:
>
> https://github.com/embench/embench-iot/issues/62
> - Difficulty recreating results
> - Given some guidance, encourage to join call and mailing list
> - Thanks to joeski65 for raising this.
>
> ACTION: Jeremy to add tutorial example to reproduce baseline results
> in documentation.
>
> https://github.com/embench/embench-iot/issues/65
> - Issue of cubic using long double
> - Explained rationale (we have discussed this before)
> - Thanks to Matteo Perotti for raising this
>
> https://github.com/embench/embench-iot/issues/66
> - Add --verbose and --quiet flags
> - Enhancement request (see also pull request 64 above)
> - Thanks to Jeremy Bennett for raising this
>
> From embench-iot-results:
>
> https://github.com/embench/embench-iot-results/issues/1
> - Not clear what libraries are used for results
> - Latest commit clarifies this.
> - Thanks to Matteo Perotti for raising this.
>
> https://github.com/embench/embench-iot-results/issues/2
> - Reference data is now wrong.
> - Latest commit resolves this.
> - Thanks to Matteo Perotti for raising this.
>
> Thanks to all those who raised issues and provided responses.
>
> 4. Update on benchmarks
> -----------------------
>
> The embench-iot-results repository now captures the Embench 0.5
> results for Arm Cortex M4 and RISC-V RI5CY.  Pull requests for other
> architectures very welcome.
>
> Roger Shepherd has raised the issue of widening results to increase
> credibility and has volunteered to provide data for ST dev
> boards, Macs and Rasberry Pi’s.
>
> ACTION: Roger Shepherd to provide Embench results for ST dev boards,
> Apple Macs and Raspberry Pi.
>
> ACTION: Jeremy Bennett/Paolo Savini to attempt to get results for
> Atmel AVR using Arduino.
>
> Volunteers for more platforms are welcome
>
> 5. Publicity
> ------------
>
> David Patterson has made a submission to Arm TechCon for this year.
>
> We now have a Wikipedia page: https://en.wikipedia.org/wiki/Embench.
> Please edit to improve.
>
> ACTION: Jeremy Bennett to add link to results page.
>
> Jeremy Bennett's slides from the London meetup in January are now
> online for others to use. NOTE they predate the release of 0.5 and the
> Arm Cortex M4 baseline, so the data needs updating. See
> https://docs.google.com/presentation/d/1YW5sXZODsCitqGw6mWBzSMtAKQQtYRK61JHj3FkymZY.
> Google presentations struggle with error bars on graphs, so the graphs
> are all images copied from LibreOffice calc.
>
> Bill McSpadden noted that Google Docs/G-drive are not supported in China.
>
> ACTION: Jeremy Bennett to explore alternatives - possibly restructured
> text or hieroglyph, which could be held in a repository.
>
> 6. Embench 0.6
> --------------
>
> At our last meeting we all took an action to consider new benchmarks
> that could be included in a floating-point Embench suite.  There is a
> shared document to add these:
> https://docs.google.com/document/d/1tC0LXZRYeyoSRTM9aXtapzx3sA00vfFcL9QjOxU3vzQ
>
> Discussion of future benchmarks
> - Nikos Fragoulis has C code benchmarks in Irida Labs. Have managed to
> fit deep learning network on Cortex M0, process images at 5
> frames/second. Any benchmark would be open, but have to use
> non-proprietary or random data.
> - Ray Simar suggested ideas from the TinyML book by Pete Worden and
> Daniel Situnayake. He could give his fall students a project to come
> up with a new
> - BDTI kernel benchmarks - set of 12 kernels, FIR, IIR, FFT, vector
> operations etc. Offer to make some of their benchmarks available. Good
> low level DSP benchmarks.
>
> ACTION: Nikos Fragoulis to propose benchmark and suitable data.
>
> ACTION: Ray Simar to set class project to come up with new benchmarks
> based on the TinyML book.
>
> ACTION: David Patterson to convene a meeting with Ray Simar and Jeff
> Bier to discuss use of BDTI benchmarks, including how they would be
> licensed (anything GPL compatible is fine).
>
> A reminder that we need volunteers to lead particular aspects of the
> effort.
>
> 7. Future meetings
> ------------------
>
> Future meetings will be 1 hour long.
>
> We currently meet regularly on the third Monday of each month 8-10am
> California time. Our future meetings will be. Note that 2 dates have
> been changed to accommodate upcoming conferences/holidays.
> - Mon 20 Jul 2020
> - Mon 10 Aug 2020 (changed)
> - Mon 21 Sep 2020
> - Mon 19 Oct 2020
> - Mon 16 Nov 2020
> - Mon 14 Dec 2020 (changed)
>
> I've created a calendar entry for our regular attendees, to provide
> automated reminders. Please ask if you wish to be added to this.
>
> 8. AOB
> ------
>
> Paul Butcher introduced his work on fuzzing for security using Embench.
>
> Best wishes,
>
>
> Jeremy

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Embench_GCC_9-2_10-1.pdf
Type: application/pdf
Size: 57715 bytes
Desc: not available
URL: <http://lists.librecores.org/pipermail/embench/attachments/20200716/6bfd98bd/attachment.pdf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.librecores.org/pipermail/embench/attachments/20200716/6bfd98bd/attachment.sig>


More information about the Embench mailing list