[Embench] Notes from Embench call 15 June 2020

Jeremy Bennett jeremy.bennett at embecosm.com
Mon Jun 15 18:03:45 CEST 2020

Hash: SHA1

Hi all,

These are the notes from last week's meeting, summarizing actions.

0. Welcomes
- -----------

Welcome new participants Erin Olson, Nikos Fragoulis, Ray Simar and
Paul Butcher.

1. Actions from previous meetings
- ---------------------------------

ACTION: Jeremy Bennett. Update London Meetup slides and make available
to group.
- - Complete, but needs updating as discussed in agenda item 5 below.

ACTION: Jeremy Bennett to complete work on the embench-iot-results
- - Complete, see pull request 3.

ACTION: Paolo Savini to distribute his slide.
- - In progress.

ACTION (continued): Jeremy Bennett to update the documentation with
more details to make the board easier to find in future.
- - Complete, see pull request 67.

ACTION: David Patterson/Jeremy Bennett. Chase up what is happening for
Arm TechCon.
- - Complete. Proposal submitted.

ACTION: Paolo Savini to explore suitable BEEBS floating point benchmarks.
- - In progress.

ACTION: Jeremy Bennett. Set up a shared Google doc for people to
record suggestions.
- - Complete, see

ACTION: All. Reach out to friends in embedded computing for
suggestions, including DSP suggestions.
- - In progress.

2. GitHub pull requests
- -----------------------

- From embench-iot:

- - Add --baselinedir and --baseline-output options
- - Discussion felt this would be useful, will commit
- - Thanks to Ed  Jones for providing this.

ACTION: Jeremy Bennett to commit the pull request.

- - Always log compiler and linker output, even if successful
- - Merged
- - Issue 66 raised to add --quiet and --verbose flags
- - Thanks to Ed Jones for providing this.

- - Document how to obtain the baseline platform
- - Merged
- - Thanks to Jeremy Bennett for providing this.

- From embench-iot-results:

- - The rework to just hold Embench 0.5 results
- - Merged as a matter of urgency
- - Tagged embench-0.5
- - Thanks to Jeremy Bennett for providing this.

Thanks to all those who provided patches.

3. New GitHub issues
- --------------------

- From embench-iot:

- - Difficulty recreating results
- - Given some guidance, encourage to join call and mailing list
- - Thanks to joeski65 for raising this.

ACTION: Jeremy to add tutorial example to reproduce baseline results
in documentation.

- - Issue of cubic using long double
- - Explained rationale (we have discussed this before)
- - Thanks to Matteo Perotti for raising this

- - Add --verbose and --quiet flags
- - Enhancement request (see also pull request 64 above)
- - Thanks to Jeremy Bennett for raising this

- From embench-iot-results:

- - Not clear what libraries are used for results
- - Latest commit clarifies this.
- - Thanks to Matteo Perotti for raising this.

- - Reference data is now wrong.
- - Latest commit resolves this.
- - Thanks to Matteo Perotti for raising this.

Thanks to all those who raised issues and provided responses.

4. Update on benchmarks
- -----------------------

The embench-iot-results repository now captures the Embench 0.5
results for Arm Cortex M4 and RISC-V RI5CY.  Pull requests for other
architectures very welcome.

Roger Shepherd has raised the issue of widening results to increase
credibility and has volunteered to provide data for ST dev
boards, Macs and Rasberry Pi’s.

ACTION: Roger Shepherd to provide Embench results for ST dev boards,
Apple Macs and Raspberry Pi.

ACTION: Jeremy Bennett/Paolo Savini to attempt to get results for
Atmel AVR using Arduino.

Volunteers for more platforms are welcome

5. Publicity
- ------------

David Patterson has made a submission to Arm TechCon for this year.

We now have a Wikipedia page: https://en.wikipedia.org/wiki/Embench.
Please edit to improve.

ACTION: Jeremy Bennett to add link to results page.

Jeremy Bennett's slides from the London meetup in January are now
online for others to use. NOTE they predate the release of 0.5 and the
Arm Cortex M4 baseline, so the data needs updating. See
Google presentations struggle with error bars on graphs, so the graphs
are all images copied from LibreOffice calc.

Bill McSpadden noted that Google Docs/G-drive are not supported in China.

ACTION: Jeremy Bennett to explore alternatives - possibly restructured
text or hieroglyph, which could be held in a repository.

6. Embench 0.6
- --------------

At our last meeting we all took an action to consider new benchmarks
that could be included in a floating-point Embench suite.  There is a
shared document to add these:

Discussion of future benchmarks
- - Nikos Fragoulis has C code benchmarks in Irida Labs. Have managed to
fit deep learning network on Cortex M0, process images at 5
frames/second. Any benchmark would be open, but have to use
non-proprietary or random data.
- - Ray Simar suggested ideas from the TinyML book by Pete Worden and
Daniel Situnayake. He could give his fall students a project to come
up with a new
- - BDTI kernel benchmarks - set of 12 kernels, FIR, IIR, FFT, vector
operations etc. Offer to make some of their benchmarks available. Good
low level DSP benchmarks.

ACTION: Nikos Fragoulis to propose benchmark and suitable data.

ACTION: Ray Simar to set class project to come up with new benchmarks
based on the TinyML book.

ACTION: David Patterson to convene a meeting with Ray Simar and Jeff
Bier to discuss use of BDTI benchmarks, including how they would be
licensed (anything GPL compatible is fine).

A reminder that we need volunteers to lead particular aspects of the

7. Future meetings
- ------------------

Future meetings will be 1 hour long.

We currently meet regularly on the third Monday of each month 8-10am
California time. Our future meetings will be. Note that 2 dates have
been changed to accommodate upcoming conferences/holidays.
- - Mon 20 Jul 2020
- - Mon 10 Aug 2020 (changed)
- - Mon 21 Sep 2020
- - Mon 19 Oct 2020
- - Mon 16 Nov 2020
- - Mon 14 Dec 2020 (changed)

I've created a calendar entry for our regular attendees, to provide
automated reminders. Please ask if you wish to be added to this.

8. AOB
- ------

Paul Butcher introduced his work on fuzzing for security using Embench.

Best wishes,

- -- 
Cell: +44 (7970) 676050
SkypeID: jeremybennett
Twitter: @jeremypbennett
Email: jeremy.bennett at embecosm.com
Web: www.embecosm.com
PGP key: 1024D/BEF58172FB4754E1 2009-03-20


More information about the Embench mailing list