[dart-misc] Dart Code Snippet Benchmarks

[Hans Van den Keybus]

Hi All,

Just wanted to let you know that I started a library to benchmark several
coding styles / code snippets.
The project can be found here:
https://github.com/dotdotcommadot/dotdotcommadot_dart_benchmarks

The goal is to find out which writing style is faster than the other.

A few remarkable conclusions I can already make are
- Foreach() loops are about 2x slower than using a for(var i...) loop
- When compiled to JavaScript, Operator Overloading works 8x (!!) faster
than calling a regular method
- In the Dart VM, String Interpolation ('$hello $world') is 3x faster than
the + operator (hello + ' ' + world)
However compiled to JavaScript, this makes no difference.

More tests will be coming up.
Feel free to contribute or make suggestions!

[Joel Trottier-Hébert]

That's interesting! Thanks for the share :)

Post by Hans Van den Keybus
Hi All,
Just wanted to let you know that I started a library to benchmark several
coding styles / code snippets.
https://github.com/dotdotcommadot/dotdotcommadot_dart_benchmarks
The goal is to find out which writing style is faster than the other.
A few remarkable conclusions I can already make are
- Foreach() loops are about 2x slower than using a for(var i...) loop
- When compiled to JavaScript, Operator Overloading works 8x (!!) faster
than calling a regular method
- In the Dart VM, String Interpolation ('$hello $world') is 3x faster than
the + operator (hello + ' ' + world)
However compiled to JavaScript, this makes no difference.
More tests will be coming up.
Feel free to contribute or make suggestions!
--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
To unsubscribe from this group and stop receiving emails from it, send an

[Andrew Skalkin]

I think this is a great idea. In particular, for number crunching, it would
be great to see the performance impact for cases when Dart compiler
identifies a field as a non-nullable one, and therefore gets rid of null
checking when accessing the fields/variables.

Post by Hans Van den Keybus
Hi All,
Just wanted to let you know that I started a library to benchmark several
coding styles / code snippets.
https://github.com/dotdotcommadot/dotdotcommadot_dart_benchmarks
The goal is to find out which writing style is faster than the other.
A few remarkable conclusions I can already make are
- Foreach() loops are about 2x slower than using a for(var i...) loop
- When compiled to JavaScript, Operator Overloading works 8x (!!) faster
than calling a regular method
- In the Dart VM, String Interpolation ('$hello $world') is 3x faster than
the + operator (hello + ' ' + world)
However compiled to JavaScript, this makes no difference.
More tests will be coming up.
Feel free to contribute or make suggestions!

['Bob Nystrom' via Dart Misc]

On Mon, Jun 29, 2015 at 6:37 AM, Hans Van den Keybus <

Post by Hans Van den Keybus
- Foreach() loops are about 2x slower than using a for(var i...) loop

Your forEach() benchmark is doing something fundamentally different than
the for loop one. The former is iterating over a list and the latter is
just incrementing a number. You should walk a list with for as well to get
an equal comparison. (Because real world code sure as hell should *not* be
creating a list of integers and iterating over them just to do a numeric
loop!)

Also, the bodies of the loops are empty. If you aren't careful, the
optimizer will notice that and completely eliminate your loop. Benchmarking
modern language implementations with complex JITs and optimizers is hard.

For what it's worth, this script:

const trials = 10;
const iterations = 10000000;

void time(String name, benchmark()) {
var start = new DateTime.now();

for (var i = 0; i < trials; i++) {
benchmark();
}

var elapsed = new DateTime.now().difference(start).inMilliseconds /
trials;
print("$name: $elapsed ms/trial");
}

void main() {
var list = [];
for (var i = 0; i< iterations; i++)
{
list.add(i);
}

time("forEach()", () {
var sum = 0;
list.forEach((i) {
sum += i;
});

if (sum == 1000) print("prevent dead code elimination");
});

time("for()", () {
var sum = 0;
for (var i = 0; i < list.length; i++) {
sum += i;
}

if (sum == 1000) print("prevent dead code elimination");
});

time("while()", () {
var sum = 0;
var iterator = list.iterator;
while (iterator.moveNext()) {
sum += iterator.current;
}

if (sum == 1000) print("prevent dead code elimination");
});
}


On my Mac laptop outputs this when run on the VM:

forEach(): 46.5 ms/trial
for(): 6.6 ms/trial
while(): 15.7 ms/trial


I don't promise that my script isn't flawed either. :)

- bob

[Yissachar Radcliffe]

Shouldn't your for loop do:

sum += list[i];

For a fair comparison? Currently foreach and while perform list access, while for does not.

On my machine this only adds 3ms to the for loop time, so the overall conclusion remains the same. Though for some reason my while speeds are much closer to the foreach speeds than yours are:

foreach(): 52.6 ms/trial
for(): 9.6 ms/trial
while(): 43.1 ms/trial

[Don Olmstead]

What about a for in loop? I had thought those would be the same as the for
though not positive.

Post by Yissachar Radcliffe
sum += list[i];
For a fair comparison? Currently foreach and while perform list access, while for does not.
On my machine this only adds 3ms to the for loop time, so the overall
conclusion remains the same. Though for some reason my while speeds are
foreach(): 52.6 ms/trial
for(): 9.6 ms/trial
while(): 43.1 ms/trial
--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
To unsubscribe from this group and stop receiving emails from it, send an

['Bob Nystrom' via Dart Misc]

Post by Yissachar Radcliffe
sum += list[i];

Oops, yes! :)

- bob

[Hans Van den Keybus]

Thanks all for the very valid reply!
It is true that some for loops resulted in an empty body, when compiled to
JS.
Also, the loops now perform similar functionality.

This has been fixed.

@Andrew Skalkin: Can you give me an example of what you mean?

Grts,
Hans

['Andreas Kirsch' via Dart Misc]

Are you sure that the operator overloading version works correctly? Maybe
there is a bug and dart2js is optimizing away your expression?

Post by Hans Van den Keybus
Thanks all for the very valid reply!
It is true that some for loops resulted in an empty body, when compiled to
JS.
Also, the loops now perform similar functionality.
This has been fixed.
@Andrew Skalkin: Can you give me an example of what you mean?
Grts,
Hans
--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
To unsubscribe from this group and stop receiving emails from it, send an

[Hans Van den Keybus]

@Andreas:
I was just looking into it myself.
I see something very strange going on here.
The generated JS code, is:

NoOperatorOverloading: {
"^": "BenchmarkBase;myClazzes,myClazz,name,emitter",
setup$0: function() {
var t1, i;
for (t1 = this.myClazzes, i = 0; i < 100000; ++i)
t1.push(new N.Clazz(i));
},
run$0: function() {
var t1, t2, i, t3;
for (t1 = this.myClazzes, t2 = this.myClazz, i = 0; i < t1.length;
++i) {
t3 = t1[i];
t2.myInteger = t2.myInteger + t3.myInteger;
}
},
teardown$0: function() {
}
},
OperatorOverloading: {
"^": "BenchmarkBase;myClazzes,myClazz,name,emitter",
setup$0: function() {
var t1, i;
for (t1 = this.myClazzes, i = 0; i < 100000; ++i)
t1.push(new N.Clazz(i));
},
run$0: function() {
var t1, t2, i, t3;
for (t1 = this.myClazzes, t2 = this.myClazz, i = 0; i < t1.length;
++i) {
t3 = t1[i];
t2.myInteger = t2.myInteger + t3.myInteger;
}
},
teardown$0: function() {
}
},

So both are exactly the same!
Only the results are wide apart: 20551ms against 3036.

Weird stuff going on...

['Andreas Kirsch' via Dart Misc]

Are you running both from the same browser/tab/main? Swap the two around
and look at the results. Maybe there is some cross-method optimization
happening because both JavaScript objects are the same...

In general, you want to start with a cold environment for every benchmark
probably.

Post by Hans Van den Keybus
I was just looking into it myself.
I see something very strange going on here.
NoOperatorOverloading: {
"^": "BenchmarkBase;myClazzes,myClazz,name,emitter",
setup$0: function() {
var t1, i;
for (t1 = this.myClazzes, i = 0; i < 100000; ++i)
t1.push(new N.Clazz(i));
},
run$0: function() {
var t1, t2, i, t3;
for (t1 = this.myClazzes, t2 = this.myClazz, i = 0; i < t1.length;
++i) {
t3 = t1[i];
t2.myInteger = t2.myInteger + t3.myInteger;
}
},
teardown$0: function() {
}
},
OperatorOverloading: {
"^": "BenchmarkBase;myClazzes,myClazz,name,emitter",
setup$0: function() {
var t1, i;
for (t1 = this.myClazzes, i = 0; i < 100000; ++i)
t1.push(new N.Clazz(i));
},
run$0: function() {
var t1, t2, i, t3;
for (t1 = this.myClazzes, t2 = this.myClazz, i = 0; i < t1.length;
++i) {
t3 = t1[i];
t2.myInteger = t2.myInteger + t3.myInteger;
}
},
teardown$0: function() {
}
},
So both are exactly the same!
Only the results are wide apart: 20551ms against 3036.
Weird stuff going on...
--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
To unsubscribe from this group and stop receiving emails from it, send an

[Ahmet A. Akın]

Honestly, I would not trust micro benchmarks. Yes they are fun, but there
are too many pitfalls to actually measure platform speed correctly.

Post by Hans Van den Keybus
Hi All,
Just wanted to let you know that I started a library to benchmark several
coding styles / code snippets.
https://github.com/dotdotcommadot/dotdotcommadot_dart_benchmarks
The goal is to find out which writing style is faster than the other.
A few remarkable conclusions I can already make are
- Foreach() loops are about 2x slower than using a for(var i...) loop
- When compiled to JavaScript, Operator Overloading works 8x (!!) faster
than calling a regular method
- In the Dart VM, String Interpolation ('$hello $world') is 3x faster than
the + operator (hello + ' ' + world)
However compiled to JavaScript, this makes no difference.
More tests will be coming up.
Feel free to contribute or make suggestions!