[Embench] Notes from Embench call 10 August 2020

Jeremy Bennett jeremy.bennett at embecosm.com
Mon Aug 10 18:19:52 CEST 2020

Hash: SHA256

Hi all,

These are the notes from today's meeting, summarizing actions.

1. Actions from previous meetings
- ---------------------------------

ACTION: Jeremy Bennett to add tutorial example to reproduce baseline
results in documentation.
- - captured as GitHub issue 68
- - in progress

ACTION (ongoing): Roger Shepherd to provide Embench results for ST dev
boards, Apple Macs and Raspberry Pi.
- - in progress
- - see agenda item 4.

ACTION: Jeremy Bennett/Paolo Savini to attempt to get results for
Atmel AVR using Arduino.
- - in progress

ACTION: Jeremy Bennett to explore alternatives to Google Docs for
- - in progress

ACTION (ongoing): Ray Simar to set class project to come up with new
benchmarks based on the TinyML book.

ACTION (ongoing): Ray Simar to continue work on BDTI benchmarks,
including how they would be licensed (anything GPL compatible is fine).

ACTION: Jeremy to set up repository, with Ray as co-owner for FP Embench
- - complete

ACTION: Jeremy to reissue 10 August invitation.
- - complete

2. GitHub pull requests
- -----------------------

- From embench-iot:

- - adds missing prerequsite to documentation
- - merged.
- - thanks to Jeremy Bennett for the fix

- - remove Python 3.6 dependency when checking version
- - merged
- - thanks to Nidal Faour for the fix

- - make scripts work with older versions of Python
- - rejected - previous design decision to use Python 3.6
- - thanks to Nidal Faour proposing this solution

- - resolves issue #84
- - under revision
- - thanks to Roger Shepherd for the fix

- From embench-iot-results:


Thank you to everyone who contributed pull requests.

3. New GitHub issues
- --------------------

- From embench-iot:

- - parameter value priority order is unclear
- - the structure of config files needs cleaning up
- - thanks to Roger Shepherd for raising this

- - -gc-sections is -dead-strip in the Apple linker
- - this issue will largely go away when we get rid of dummy libs
- - any residual documentation should be updated.
- - thanks to Roger Shepherd for raising this

- - documentation incorrectly says --chip and --board are mandatory
- - bug in the documentation to be fixed.
- - thanks to Roger Shepherd for raising this

- - native target truncates time measurements
- - appears to be a bug in run_native.py
- - thanks to Samuel Obuch for raising this

- - error in logging failed command when running benchmarks
- - but benchmark_speed.py
- - thanks to Samuel Obuch for raising this

- - unclear naming convention for Python modules in benchmark_speed.py
- - bug in the documentation
- - thanks to Roger Shepherd for raising this

- From embench-iot-results:


Thanks to all those who raised issues and provided responses.

4. Update on benchmarks
- -----------------------

Roger Shepherd gave his experience of benchmarking Apple Mac
- - see issues and pull requests

Roger obtained initial speed results. Sizing ran into issues with
Mach-O format rather than ELF. Library identified that can solve this

5. Publicity
- ------------

We were unfortunately not accepted for Arm TechCon online. Cesare
noted it is a different event - it is just for developers this year.

Jeremy Bennett is speaking at VSDOpen in October, a conference for
students in India. Although the talk has quite a broad abstract, it
will use Embench as an example of the bridge between hardware and
software in open source. David Patterson may be speaking at the same

It was noted that Huawei and ETH Z├╝rich have used Embench in a recent
paper on code size optimizations for the RISC-V architecture.

Stefan Wallentowitz has offered to help improve the website.

6. Embench 0.6
- --------------

There is an ongoing shared document to add potential programs for a
new floating point benchmark suite:


This has been updated with the candidate programs from BEEBS.

Ray Sinar at Rice University is leading the effort to develop the FP
benchmark suite with his students during the coming semester.

A reminder that we need volunteers to lead particular aspects of
current and future efforts.

Cesare Garlati sees lot of interest coming back to 16 register RISC-V
(RV32E) from Huawei, Alibaba and others to get closer to Arm Thumb.

HEX-Five about to announce complete open source stack for IoT devices
based on both RISC-V and Arm. Could use the nano-kernel as the basis
of an updated context-switching benchmark.

Volunteers needed to pick up this work. Attendees are encouaged to
reach out to Universities who are looking for student projects at this
time of year.

PolarFire chip from MicroSemi is now available for benchmarking on
development board in a month or so. Good for application class embench.

Erin Olson from Seagate has parallelized running Embench (for use with
simulated RTL). Erin was congratulated for her efforts and encouraged
to submit pull request.

New build system from Western Digital is still on the workplan, but a
bit delayed. Embench is being pushed as the official benchmark for
code size.

7. Future meetings
- ------------------

We currently meet regularly on the third Monday of each month 8-10am
California time. Our future meetings will be. Note that the December
date has been changed to avoid the week before Christmas.

- - Mon 21 Sep 2020
- - Mon 19 Oct 2020
- - Mon 16 Nov 2020
- - Mon 14 Dec 2020 (changed)

At our next meeting, I shall propose dates for 2021.

I've created a calendar entry for our regular attendees, to provide
automated reminders. Please ask if you wish to be added to this.

8. AOB
- ------

ACTION: Jeremy Bennett to update documentation to advise that Python
minimum version 3.6 is required.

Best wishes,

Vice Chair
- -- 
Cell: +44 (7970) 676050
SkypeID: jeremybennett
Twitter: @jeremypbennett
Email: jeremy.bennett at embecosm.com
Web: www.embecosm.com
PGP key: 1024D/BEF58172FB4754E1 2009-03-20


More information about the Embench mailing list