[Embench] Embench meeting/call 19 July 2020

Jeremy Bennett jeremy.bennett at embecosm.com
Sun Jul 19 18:31:30 CEST 2020

Hash: SHA1

Hi all,

Our next regular meeting of the Embench group will be tomorrow Monday,
19 July, 8-10am California time.

We will meet using the same Zoom details as before. Please email me if
you do not have them - I am reluctant to post details on an open
mailing list.


1. Actions from previous meeting
- --------------------------------

ACTION: Paolo Savini to distribute his slide.
- - complete

ACTION: Paolo Savini to explore suitable BEEBS floating point benchmarks.
- - see agenda item 6
- - in progress

ACTION: Jeremy Bennett to commit the pull request 63
- - complete

ACTION: Jeremy to add tutorial example to reproduce baseline results
in documentation.
- - captured as GitHub issue 68
- - in progress

ACTION: Roger Shepherd to provide Embench results for ST dev boards,
Apple Macs and Raspberry Pi.
- - see agenda item 4
- - Roger to report

ACTION: Jeremy Bennett/Paolo Savini to attempt to get results for
Atmel AVR using Arduino.
- - see agenda item 4
- - Paolo to report

ACTION: Jeremy Bennett to add link to results page to Wikipedia.
- - added text and link, but link removed - help from Wikipedia expert
- - see agenda item 5
- - complete

ACTION: Jeremy Bennett to explore alternatives to Google Docs - possibly
restructured text or hieroglyph, which could be held in a repository.
- - see agenda item 5
- - in progress

ACTION: Nikos Fragoulis to propose benchmark and suitable data from
Irida Labs
- - see agenda item 6
- - Nikos to report

ACTION: Ray Simar to set class project to come up with new benchmarks
based on the TinyML book.
- - see agenda item 6
- - Ray to report

ACTION: David Patterson to convene a meeting with Ray Simar and Jeff
Bier to discuss use of BDTI benchmarks, including how they would be
licensed (anything GPL compatible is fine).
- - see agenda item 6
- - David to report

2. GitHub pull requests
- -----------------------

- - fixes issue 69
- - thanks to Brian Grayson of SiFive for the fix

- - fixes issue 70
- - thanks to Jeremy Bennett for the fix

3. New GitHub issues
- --------------------

- From embench-iot:

- - captures action to document creation of baseline results
- - assigned to Jeremy Bennett

- - repeated builds fail noisily
- - bug in build_all.py fixed by pull-request 74
- - thanks to Brian Grayson of SiFive for the issue report and fix

- - Update doc to note pyelftools is a prerequisite
- - fixed by pull-request 75
- - thanks to Ed Jones of Embecosm for the issue report

- - results directory is not created by run_all.py
- - bug assigned to Jeremy Bennett
- - thanks to Ed Jones of Embecosm for the issue report

- - wikisort has a GCC extension in the code
- - assigned to Paolo Savini to fix and check all benchmarks
- - we need to agree what C standard we require (see agenda item 4)
- - thanks to Kjetil Østerås of Silicon Labs for the issue report

- - other benchmarks use GCC extensions
- - generalization of issue 72, also assigned to Paolo Savini
- - thanks to Kjetil Østerås of Silicon Labs for the issue report

- From embench-iot-results:

- - what is the performance of baseline board at full-speed
- - flat out would be affected by Flash memory access times.
- - assigned to Paolo Savini to investigate.
- - Thanks to Charles Papon (Dolu1990) for raising this.

4. Update on benchmarks
- -----------------------

Issues 72 and 73 have highlighted that we have no documented what C
standard the benchmarks follow. Historically benchmarks have always
gone for ANSI C89 (ISO C90) as the lowest common denominator, but
should we really base a new benchmark set on a coding standard set
more than 3 decades ago.

I should like to propose we mandate C11 as the minimum for all C
benchmarks and C++11 for all C++ benchmarks (which we will presumably
have in the future).

Roger Shepherd and Paolo Savini have an action to try to obtain more
data points for other platforms. They are invited to report on progress.

5. Publicity
- ------------

The Wikipedia page: https://en.wikipedia.org/wiki/Embench, now refers
to the results repository, but the link was removed (no external links
in the text).  We'd appreciate a Wikipedia expert to advise on the
correct way to add such links.

Jeremy Bennett had an action to explore alternatives to Google for the
standard presentation. There is work in progress on a version in
ReStructuredText using Sphinx/Hieroglyph version here:


Some of the tables are stretching the limits of ReStructuredText. Any
suggestions welcome.

Report on other conferences upcoming.

6. Embench 0.6
- --------------

There is an ongoing shared document to add potential programs for a
new floating point benchmark suite:


Following on from last week's meeting, Nikos Fragoulis, Ray Simar  and
David Patterson are invited to report back on their actions.

Paolo Savini has completed his action to look at candidate floating
point programs in the BEEBS benchmark set. He suggests matmult-float
(as already suggested), ludcmp, qurt, sqrt, fasta, stb_perlin and frac
as being plausible candidates. Paolo may be unable to join the
meeting, but has provided detailed notes to the vice-chair and chair.

An opportunity to discuss.

A reminder that we need volunteers to lead particular aspects of the

7. Next meeting
- ---------------

We currently meet regularly on the third Monday of each month 8-10am
California time. Our future meetings will be. Note that 2 dates have
been changed to accommodate upcoming conferences/holidays.
- - Mon 10 Aug 2020 (changed)
- - Mon 21 Sep 2020
- - Mon 19 Oct 2020
- - Mon 16 Nov 2020
- - Mon 14 Dec 2020 (changed)

I've created a calendar entry for our regular attendees, to provide
automated reminders. Please ask if you wish to be added to this.

8. AOB
- ------

Best wishes,

Jeremy Bennett
Vice chair



More information about the Embench mailing list