benchmark test: HaWe brickbench – for C# / Mono?

ps,
link to the source code libs seemed to work faulty – this one should work:
http://www.mindstormsforum.de/viewtopic.php?f=71&t=8095&start=60#p64499

hello,
thank you very much, I gladly take the Mono C# results up into my table.

Please notice that according to Copyright restrictions it is not allowed to publish your own tables:

© Helmut Wunder 2013,2014.
HaWe brickbench test Comparison tables/speadsheets and results are copyright protected and may not be freely created, copied and / or distributed. Additions and / or changes in the tables require written permission by the author.
HaWe brickbench test – Vergleichstabellen und Ergebnisse sind urheberrechtlich geschützt und dürfen nicht frei erstellt, kopiert und/oder verbreitet werden. Ergänzungen und/oder Veränderungen der Tabellen bedürfen der schriftlichen Genehmigung durch den Autor.
http://www.mindstormsforum.de/viewtopic.php?f=71&t=8095&start=60#p64772

I also obeserved that you did not include the Copyright rules for the code itself:

// hw brickbench
// benchmark test for NXT/EV3 and similar Micro Controllers
// PL: . . . . . . . . .
// Autor: (C) Helmut Wunder 2013,2014
// Ported to . . . . . by . . . . . .
// freie Verwendung für private Zwecke
// für kommerzielle Zwecke nur nach Genehmigung durch den Autor.
// protected under the friendly Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License
// http://creativecommons.org/licenses/by-nc-sa/3.0/
// version 1.08

I gladly publish both your C# code and your C# results in my results table, and of course you may link to my results tables in my Thread, but please delete your table which you have refered to in your own link.

Please understand that these precausions unfortunately had to be taken to prevent faulty and/or uncornfirmed data to be disseminated.

Please send me a link with all published and needed files in raw text code format (just blanks, no tabs), compressed in a zip folder. Github is far too confusing, additionally I need to have the access to the files in my own thread.

Last but not least a couple of questions:
– why do you have different “runs” – what does it mean? How have to be interpreted the divergent data ?
– what about the display bench mark? (As you might observe e.g., for RobotC and Java, grafic benchmark partially have a big influence on the overall benchmark)
– also some data are needed for memory specs provided by C#:
runt. code mem.,
variable mem.
BT raw master+slave
RS485/i2c raw master+slv.
BT, +MC I/O rem.protocol
RS485, +MC I/O rem.prot.
USB MC Daisy-Chain
WiFi MC I/O rem.protocol
feat. double=64bit float
feat.>=4D array/matrix oper.
Multitthreading
API complete (sensor+motor+screen)?

Thank you very much again for your input!

Vlad,
thank you very much for your input and for your cooperation – I was afraid you might perceive my post as sort of rude, I apologize if you might have had this impression – but it was only meant to avoid obsolete data to misrepresent the real comparison at any later date which would be unfair to the other tests (not sure if I could express my intention correctly). Of course you are free to publish your own results which is in your own responsibility – and I certainly would appreciate if you’d publish them singularily without showing different platform data!

Indeed some changes meanwhile have been made in the RobotC VM which lead to a quicker runtime performance.
The code and the test have been written and performed by Xander Soldaat, and he wrote me that especially the sort algorithm is meanwhile replaced in an internal build by an intrinsic test with the speed comparable to Linux executables.
As soon as the new RC build has been released he surely will publish the updated results.
If you like, you certainly may perform the RC test and publish them, too (with a date time and a RC version stamp), and I then will gladly always update the newest results in the comparison chart. As I don’t have RC, I can’t do the tests by myself.
BTW, there are additional issues about the leJOS benchmarks and I’m waiting for Andy Shaw’s fixes, too, to include them as well, just post them please directly here in the forum again if you wish!

About Mono/C#:
Now I also understand the loop thing in your table. It appears to be something similar to the Java JIT (just-in-time) compiler. Actually it’s complicated to unite all those runtime data for each VM and each loop in my spreadsheet table – finally it would be bursting at the seams. Maybe just loops #1 + #3 would give a representative cross-section about this, I’ll add them ASAP.

Just to outline what the rest of the benchs is about (of course I know that you won’t know all the details without exception):
I would appreciate to get to know the matrix and the sort tests, possibly also the text_out. If there is no graphic lib, we’ll have to skip that thing (nxtOSEK also provides no graphic lib, but Martin Aumair wrote and published his own lib so that it could be performed).
About memory: is there a memavail() function where you can read this SRAM mem during runtime? Is it up to 64MByte like gpp C or even more, maybe up to all built-in 264 MByte?
Is the code size limited in either way?
About BT or daisy-chaining or multi-robot-communication and all that: Do you have an idea if direct I/O control is possible? Is there something written in the documentation? (The Lego FW provides 1+3 for USB daisy chaining and 1+7 for raw BT comm).
The multi-dim array thing is about how to manipulate multi-dim arrays, e.g. after beeing passed to functions. If you wish to compare the gpp C code to the RobotC or the nxtOSEK code you’ll recognize what I mean:
by gpp C, you can access both rows and columns individually/seperately, in other cases this is not possible, one has to multiply the rows by number of columns to access the cell in a virtual linearized array.
The first feature is the better one, of course, and this is what this point is about.
The last things about int, long, and float already have been answered in our German forum (you are welcome to join it!): http://www.mindstormsforum.de/viewtopic.php?f=71&t=8095&p=65299#p65297

The last line is about API completeness:
– can motors be commanded to run certain degrees by certain pwm? Is there a PID control to approach encoder targets?
– can all sensors of Lego EV3 and NXT kits be polled? Are there general i2c functions to poll 3rd party devices, too (even chained ones by reading/writing different single addr and data registers)?
– Screen API is complete AFAIK for writing text to any (x,y) position, graphics are not available as already mentioned.

Anyway, thanks again for your interest, your input and thanks for sharing your code! 🙂

hi Vlad,
I’ll add what you reported to the spreadsheet!
(actually Xamarin is not quite “small and made-to-measure” though, IMO, compared to BricxCC, RobotC, or the Sketch IDE)

About RobotC, I’m curious where the code is to be found.
As I don’t use it – can you please post the benchmark sourcecode here please or link to a zip file instead?
I don’t have access to the file.
Is it still part of the 4.25 RobotC example files?

thanks for your input!
Helmut

(edited)
thank you!
about the 4.25 samples files this is correct, it’s probably then on Xander’s homepage.
It actually was not meant to be distributed with RobotC releases because this is a commercial product.
Private distribution for private purposes of course is ok.