[Embench] Embench meeting/call 11 May 2020

Jeremy Bennett jeremy.bennett at embecosm.com
Sun Jun 14 17:13:37 CEST 2020

Hash: SHA1

Hi all,

Our next regular meeting of the Embench group will be today Monday, 15
June, 8-10am California time.

We will meet using the same Zoom details as before. Please email me if
you do not have them - I am reluctant to post details on an open
mailing list.


1. Actions from previous meeting
- --------------------------------

ACTION: Jeremy Bennett. Update London Meetup slides and make available
to group.
- - Complete, but needs updating as discussed below.

ACTION: Jeremy Bennett to complete work on the embench-iot-results
- - Complete, see pull request 3.

ACTION: Paolo Savini to distribute his slide.
- - In progress.

ACTION (continued): Jeremy Bennett to update the documentation with
more details to make the board easier to find in future.
- - Complete, see pull request 67.

ACTION: David Patterson/Jeremy Bennett. Chase up what is happening for
Arm TechCon.
- - Complete. Proposal submitted.

ACTION: Paolo Savini to explore suitable BEEBS floating point benchmarks.
- - In progress.

ACTION: Jeremy Bennett. Set up a shared Google doc for people to
record suggestions.
- - Complete, see

ACTION: All. Reach out to friends in embedded computing for
suggestions, including DSP suggestions.
- - In progress.

2. GitHub pull requests
- -----------------------

One pull request needs discussion.

- From embench-iot:

- - Add --baselinedir and --baseline-output options
- - For discussion
- - Thanks to Ed  Jones for providing this.

- - Always log compiler and linker output, even if successful
- - Merged
- - Issue 66 raised to add --quiet and --verbose flags
- - Thanks to Ed Jones for providing this.

- - Document how to obtain the baseline platform
- - Merged
- - Thanks to Jeremy Bennett for providing this.

- From embench-iot-results:

- - The rework to just hold Embench 0.5 results
- - Merged as a matter of urgency
- - Tagged embench-0.5
- - Thanks to Jeremy Bennett for providing this.

3. New GitHub issues
- --------------------

- From embench-iot:

- - Difficulty recreating results
- - Given some guidance, encourage to join call and mailing list
- - Thanks to joeski65 for raising this.

- - Issue of cubic using long double
- - Explained rationale (we have discussed this before)
- - Thanks to Matteo Perotti for raising this

- - Add --verbose and --quiet flags
- - Enhancement request (see also pull request 64 above)
- - Thanks to Jeremy Bennett for raising this

- From embench-iot-results:

- - Not clear what libraries are used for results
- - Latest commit clarifies this.
- - Thanks to Matteo Perotti for raising this.

- - Reference data is now wrong.
- - Latest commit resolves this.
- - Thanks to Matteo Perotti for raising this.

4. Update on benchmarks
- -----------------------

The embench-iot-results repository now captures the Embench 0.5
results for Arm Cortex M4 and RISC-V RI5CY.  Pull requests for other
architectures very welcome.

Roger Shepherd has raised the issue of widening results to increase
credibility.  He writes:

"I think both the credibility and usefulness of benchmarks goes up as
the result base widens. I think we should encourage the generation of
results for a range of processor/platforms - even if they from systems
that are not the target of our benchmarks. I’ll volunteer to produce
some over the summer on platforms I have easy access to - e.g. ST dev
boards, Macs, Pi’s."

5. Publicity
- ------------

David Patterson has made a submission to Arm TechCon for this year.

We now have a Wikipedia page: https://en.wikipedia.org/wiki/Embench.
Please edit to improve.

Jeremy Bennett's slides from the London meetup in January are now
online for others to use. NOTE they predate the release of 0.5 and the
Arm Cortex M4 baseline, so the data needs updating. See
Google presentations struggle with error bars on graphs, so the graphs
are all images copied from LibreOffice calc.

Report on other conferences upcoming.

6. Embench 0.6
- --------------

At our last meeting we all took an action to consider new benchmarks
that could be included in a floating-point Embench suite.  There is a
shared document to add these:

An opportunity to discuss.

A reminder that we need volunteers to lead particular aspects of the

7. Next meeting
- ---------------

We currently meet regularly on the third Monday of each month 8-10am
California time. Our future meetings will be. Note that 2 dates have
been changed to accommodate upcoming conferences/holidays.
- - Mon 20 Jul 2020
- - Mon 10 Aug 2020 (changed)
- - Mon 21 Sep 2020
- - Mon 19 Oct 2020
- - Mon 16 Nov 2020
- - Mon 14 Dec 2020 (changed)

I've created a calendar entry for our regular attendees, to provide
automated reminders. Please ask if you wish to be added to this.

8. AOB
- ------

Best wishes,

Jeremy Bennett
Vice chair



More information about the Embench mailing list