[Embench] Embench meeting/call 19 October 2020

Jeremy Bennett jeremy.bennett at embecosm.com
Sun Oct 18 18:28:46 CEST 2020

Hash: SHA1

Hi all,

Our October regular meeting of the Embench group will be tomorrow,
Monday, 19 October, 8-9am California time. We shall meet using the
same Zoom details as before.

Please email me if you do not have the Zoom meeting details - I am
reluctant to post details on an open mailing list.


1. Actions from previous meeting
- --------------------------------

ACTION: Jeremy Bennett, Stephan Wallentowitz to complete interview blog
post on website.
- - in progress

ACTION: Jeremy Bennett to circulate new generic slide deck
- - complete

ACTION: David Patterson to prepare draft archive paper
- - abstract prepared, more discussion, see agenda item 5

ACTION: Jeremy to add tutorial example to reproduce baseline results
in documentation.
- - captured as GitHub issue 68
- - in progress

ACTION (ongoing): Roger Shepherd to provide Embench results for ST dev
boards, Apple Macs and Raspberry Pi.
- - in progress

ACTION: Jeremy Bennett/Paolo Savini to attempt to get results for
Atmel AVR using Arduino.
- - in progress

ACTION: Jeremy Bennett to explore alternatives to Google Docs for
- - Open Document Format version prepared
- - hieroglyph version in progress

ACTION (ongoing): Ray Simar to set class project to come up with new
benchmarks based on the TinyML book.
- - Ray to report progress in agenda item 4

ACTION: Jeremy Bennett to update documentation to advise that Python
minimum version 3.6 is required.
- - in progress

2. GitHub pull requests
- -----------------------

- From embench-iot:


- From embench-iot-results:


3. New GitHub issues
- --------------------

- From embench-iot:

- - Does Wikisort need to use double precision constants
- - responses from Roger Shepherd and Jeremy Bennett
- - thanks to user axiomlegend for raising this
- - thanks to all who provided responses

- - Explanation / Origin of benchmarks
  (beebs.eu link broken, due to a failure to renew domain)
- - website moved to https://beebs.mageec.org/
- - marked as bug, assigned to Jeremy Bennett to fix links
- - thanks to Nagendra Gulur for raising this

- - Issue Running Embench
- - advice provided by Roger Shepherd, Jeremy Bennett and Paolo Savini
- - marked as an enhancement for documentation assigned to Paolo Savini
- - thanks to George Onwubuya for raising this
- - thanks to all who provided responses

- - Help needed for Cortex-m55f
- - initial advice from Jeremy Bennett
- - asssigned to Paolo Savini for further advice
- - thanks to user sheenagoel for raising this

- From embench-iot-results:


4. Update on benchmarks, including Embench 0.6
- ----------------------------------------------

Roger Shepherd to report on his progress with benchmarking other

Ray Sinar to report on progress with his class at Rice University on
Embench 0.6. I anticipate we may have attendees from a masters project
at Southampton University who wish to pick up on the work with TinyML
as a benchmark,

Some of you may have seen the Antmicro blog post about automated CPU
testing using Embench:


I've reached out (at the very last minute), to see if one of the team
can join us to present the work.

A reminder that we need volunteers to lead particular aspects of
current and future efforts.

5. Publicity
- ------------

Recent talks where Embench has been presented as part of the talk:

3rd Workshop on RISC-V Activities held by Edacentrum, Germany. The
talk will be looking at Embench within the context of the RISC-V
Software Ecosystem

VSDOpen. The talk considered Embench as an example of software that
bridges into the hardware world as part of a talk on the history of
open source software and hardware.

As noted in the previous item, Antmicro have published a blog post on
automated CPU testing with Embench

Following on from the suggestion of a reference arXiv paper on
Embench, David Patterson has put together an abstract:

"Embench, which stands for embedded benchmark, was developed by a
group of academics and practitioners following modern principles of
benchmark design. Embench 0.5 is a suite of 19 small integer programs
targeting embedded systems with memory sizes of 32 KiB that was
released in February 2020. Embench 0.6 will include larger programs,
aimed at embedded systems with memory sizes of 1 MiB. The programs
will use floating-point data for digital signal processing and machine
learning. We intend to release Embench 0.6 in 1H2021. This paper
covers the motivation, current status, and future directions of Embench."

We have data for soft cores in cycle accurate modeling and FPGA, but
we need data on real ASIC cores (other than the baseline Cortex-M4 data).

Stefan Wallentowitz to report on progress with his interview style
blog post about Embench, based on interview responses from Dave
Patterson and Jeremy Bennett

The new generic presentation on Embench was circulated with the
minutes of the last meeting, but still lacks the following
- - consistent performance data on RV32I, RV32IC, RV32IM and RV32IMC
- - sufficient historical record of LLVM performances
- - consistent data on ARC and AVR

6. Next meeting
- ---------------

We currently meet regularly on the third Monday of each month 8-9am
California time. Our future meetings will be. Note that the Decmember
date has been changed to avoid the week before Christmas.
- - Mon 16 Nov 2020
- - Mon 14 Dec 2020 (changed)

The following dates are proposed for 2021

- - Mon 11 Jan 2021 (week 2 to keep clear of Martin Luther King Jr day)
- - Mon 15 Feb 2021
- - Mon 15 Mar 2021
- - Mon 19 Apr 2021
- - Mon 17 May 2021
- - Mon 21 Jun 2021
- - Mon 19 Jul 2021
- - Mon 16 Aug 2021
- - Mon 21 Sep 2021
- - Mon 18 Oct 2021
- - Mon 15 Nov 2021
- - Mon 13 Dec 2021 (week 2, to keep clear of Christmas)

I've created a calendar entry for our regular attendees, to provide
automated reminders. Please ask if you wish to be added to this.

7. AOB
- ------

Best wishes,

Jeremy Bennett
Vice chair



More information about the Embench mailing list