[dart-misc] HTTP server running in parallel via Isolate

[Michele Costa]

Hi,

i am the maintainer of a micro-benchmark branch for several different
languages: i test a plain "Hello World" HTTP server by using the `wrk`
loading tool.

I've recently added Dart to the benchmark, by relying on the async HTTP
server implementation
<https://github.com/costajob/app-servers/blob/master/servers/dart_server.dart>
i've found on official tutorials.
The resulting throughput is not disappointing, but nonetheless far from the
Node.js one, which uses the cluster library to pre-fork several processes.

Dart HTTP server implementation does not run in parallel, as confirmed by
inspecting the CPUs usage via Activity Monitor.

I read that Dart deliver parallelism on a multi-core server via the Isolate
library.
Unfortunately i have found no concrete examples of how to use Isolate on
the standard APIs and the code-snippets i found online are not relevant
and/or obsolete.

Any suggestions on how to implement a HTTP server by using Isolate.spawn
will be really appreciated.

Many Thanks
Mike

['Kevin Moore' via Dart Misc]

Here you go! Using pkg/shelf – but you could do your own thing. Setting
`shared` to true is the key bit.

import 'dart:io';
import 'dart:isolate';
import 'package:shelf/shelf.dart' as shelf;
import 'package:shelf/shelf_io.dart' as io;

shelf.Response _echoRequest(shelf.Request request) =>
new shelf.Response.ok('Request for "${request.url}"');

main() async {
var isolates = 4;

for (int i = 1; i < isolates; i++) {
Isolate.spawn(_startServer, []);
}
_startServer();
}

_startServer([args]) async {
var server =
await HttpServer.bind(InternetAddress.LOOPBACK_IP_V4, 8080, shared: true);
io.serveRequests(server, _echoRequest);
}

Post by Michele Costa
Hi,
i am the maintainer of a micro-benchmark branch for several different
languages: i test a plain "Hello World" HTTP server by using the `wrk`
loading tool.
I've recently added Dart to the benchmark, by relying on the async HTTP
server implementation
<https://github.com/costajob/app-servers/blob/master/servers/dart_server.dart>
i've found on official tutorials.
The resulting throughput is not disappointing, but nonetheless far from
the Node.js one, which uses the cluster library to pre-fork several
processes.
Dart HTTP server implementation does not run in parallel, as confirmed by
inspecting the CPUs usage via Activity Monitor.
I read that Dart deliver parallelism on a multi-core server via the
Isolate library.
Unfortunately i have found no concrete examples of how to use Isolate on
the standard APIs and the code-snippets i found online are not relevant
and/or obsolete.
Any suggestions on how to implement a HTTP server by using Isolate.spawn
will be really appreciated.
Many Thanks
Mike

['Zach Anderson' via Dart Misc]

There's no need to use shelf.
https://github.com/costajob/app-servers/pull/23

On Tue, Oct 3, 2017 at 10:35 AM, 'Kevin Moore' via Dart Misc <

Post by 'Kevin Moore' via Dart Misc
Here you go! Using pkg/shelf – but you could do your own thing. Setting
`shared` to true is the key bit.
import 'dart:io';
import 'dart:isolate';
import 'package:shelf/shelf.dart' as shelf;
import 'package:shelf/shelf_io.dart' as io;
shelf.Response _echoRequest(shelf.Request request) =>
new shelf.Response.ok('Request for "${request.url}"');
main() async {
var isolates = 4;
for (int i = 1; i < isolates; i++) {
Isolate.spawn(_startServer, []);
}
_startServer();
}
_startServer([args]) async {
var server =
await HttpServer.bind(InternetAddress.LOOPBACK_IP_V4, 8080, shared: true);
io.serveRequests(server, _echoRequest);
}

Post by Michele Costa
Hi,
i am the maintainer of a micro-benchmark branch for several different
languages: i test a plain "Hello World" HTTP server by using the `wrk`
loading tool.
I've recently added Dart to the benchmark, by relying on the async HTTP
server implementation
<https://github.com/costajob/app-servers/blob/master/servers/dart_server.dart>
i've found on official tutorials.
The resulting throughput is not disappointing, but nonetheless far from
the Node.js one, which uses the cluster library to pre-fork several
processes.
Dart HTTP server implementation does not run in parallel, as confirmed by
inspecting the CPUs usage via Activity Monitor.
I read that Dart deliver parallelism on a multi-core server via the
Isolate library.
Unfortunately i have found no concrete examples of how to use Isolate on
the standard APIs and the code-snippets i found online are not relevant
and/or obsolete.
Any suggestions on how to implement a HTTP server by using Isolate.spawn
will be really appreciated.
Many Thanks
Mike

--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups
"Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an

['Kevin Moore' via Dart Misc]

On Tue, Oct 3, 2017 at 12:21 PM, 'Zach Anderson' via Dart Misc <

There's no need to use shelf. https://github.com/
costajob/app-servers/pull/23

Absolutely true! Sorry. I'm biased:
https://github.com/dart-lang/shelf/commit/8fd1b3f1ac920cdf67a34364ef8593e4a46af1c3

On Tue, Oct 3, 2017 at 10:35 AM, 'Kevin Moore' via Dart Misc <

Post by 'Kevin Moore' via Dart Misc
Here you go! Using pkg/shelf – but you could do your own thing. Setting
`shared` to true is the key bit.
import 'dart:io';
import 'dart:isolate';
import 'package:shelf/shelf.dart' as shelf;
import 'package:shelf/shelf_io.dart' as io;
shelf.Response _echoRequest(shelf.Request request) =>
new shelf.Response.ok('Request for "${request.url}"');
main() async {
var isolates = 4;
for (int i = 1; i < isolates; i++) {
Isolate.spawn(_startServer, []);
}
_startServer();
}
_startServer([args]) async {
var server =
await HttpServer.bind(InternetAddress.LOOPBACK_IP_V4, 8080, shared: true);
io.serveRequests(server, _echoRequest);
}

Post by Michele Costa
Hi,
i am the maintainer of a micro-benchmark branch for several different
languages: i test a plain "Hello World" HTTP server by using the `wrk`
loading tool.
I've recently added Dart to the benchmark, by relying on the async HTTP
server implementation
<https://github.com/costajob/app-servers/blob/master/servers/dart_server.dart>
i've found on official tutorials.
The resulting throughput is not disappointing, but nonetheless far from
the Node.js one, which uses the cluster library to pre-fork several
processes.
Dart HTTP server implementation does not run in parallel, as confirmed
by inspecting the CPUs usage via Activity Monitor.
I read that Dart deliver parallelism on a multi-core server via the
Isolate library.
Unfortunately i have found no concrete examples of how to use Isolate on
the standard APIs and the code-snippets i found online are not relevant
and/or obsolete.
Any suggestions on how to implement a HTTP server by using Isolate.spawn
will be really appreciated.
Many Thanks
Mike

--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups
"Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an

--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to a topic in the
Google Groups "Dart Misc" group.
To unsubscribe from this topic, visit https://groups.google.com/a/
dartlang.org/d/topic/misc/Ju4f2d5ziwE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to

[Michele Costa]

Hi all,

thanks for your support: i was now able to test versus a parallel server.
I am not aware it is so easy using the Isolate library.

Throughput is now better and distributed on available CPUs.

Best
Mike

Post by 'Zach Anderson' via Dart Misc
There's no need to use shelf.
https://github.com/costajob/app-servers/pull/23
On Tue, Oct 3, 2017 at 10:35 AM, 'Kevin Moore' via Dart Misc <

Post by 'Kevin Moore' via Dart Misc
Here you go! Using pkg/shelf – but you could do your own thing. Setting
`shared` to true is the key bit.
import 'dart:io';
import 'dart:isolate';
import 'package:shelf/shelf.dart' as shelf;
import 'package:shelf/shelf_io.dart' as io;
shelf.Response _echoRequest(shelf.Request request) =>
new shelf.Response.ok('Request for "${request.url}"');
main() async {
var isolates = 4;
for (int i = 1; i < isolates; i++) {
Isolate.spawn(_startServer, []);
}
_startServer();
}
_startServer([args]) async {
var server =
await HttpServer.bind(InternetAddress.LOOPBACK_IP_V4, 8080, shared: true);
io.serveRequests(server, _echoRequest);
}

Post by Michele Costa
Hi,
i am the maintainer of a micro-benchmark branch for several different
languages: i test a plain "Hello World" HTTP server by using the `wrk`
loading tool.
I've recently added Dart to the benchmark, by relying on the async HTTP
server implementation
<https://github.com/costajob/app-servers/blob/master/servers/dart_server.dart>
i've found on official tutorials.
The resulting throughput is not disappointing, but nonetheless far from
the Node.js one, which uses the cluster library to pre-fork several
processes.
Dart HTTP server implementation does not run in parallel, as confirmed
by inspecting the CPUs usage via Activity Monitor.
I read that Dart deliver parallelism on a multi-core server via the
Isolate library.
Unfortunately i have found no concrete examples of how to use Isolate on
the standard APIs and the code-snippets i found online are not relevant
and/or obsolete.
Any suggestions on how to implement a HTTP server by using Isolate.spawn
will be really appreciated.
Many Thanks
Mike

--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups
"Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an

['Filip Hracek' via Dart Misc]

Hi Mike, I'd be interested to see how this impacted the results. Will you
be making an update to the table in README?

Post by Michele Costa
Hi all,
thanks for your support: i was now able to test versus a parallel server.
I am not aware it is so easy using the Isolate library.
Throughput is now better and distributed on available CPUs.
Best
Mike

There's no need to use shelf. https://github.com/cost
ajob/app-servers/pull/23
On Tue, Oct 3, 2017 at 10:35 AM, 'Kevin Moore' via Dart Misc <

Post by 'Kevin Moore' via Dart Misc
Here you go! Using pkg/shelf – but you could do your own thing. Setting
`shared` to true is the key bit.
import 'dart:io';
import 'dart:isolate';
import 'package:shelf/shelf.dart' as shelf;
import 'package:shelf/shelf_io.dart' as io;
shelf.Response _echoRequest(shelf.Request request) =>
new shelf.Response.ok('Request for "${request.url}"');
main() async {
var isolates = 4;
for (int i = 1; i < isolates; i++) {
Isolate.spawn(_startServer, []);
}
_startServer();
}
_startServer([args]) async {
var server =
await HttpServer.bind(InternetAddress.LOOPBACK_IP_V4, 8080, shared: true);
io.serveRequests(server, _echoRequest);
}

Post by Michele Costa
Hi,
i am the maintainer of a micro-benchmark branch for several different
languages: i test a plain "Hello World" HTTP server by using the `wrk`
loading tool.
I've recently added Dart to the benchmark, by relying on the async
HTTP server implementation
<https://github.com/costajob/app-servers/blob/master/servers/dart_server.dart>
i've found on official tutorials.
The resulting throughput is not disappointing, but nonetheless far from
the Node.js one, which uses the cluster library to pre-fork several
processes.
Dart HTTP server implementation does not run in parallel, as confirmed
by inspecting the CPUs usage via Activity Monitor.
I read that Dart deliver parallelism on a multi-core server via the
Isolate library.
Unfortunately i have found no concrete examples of how to use Isolate
on the standard APIs and the code-snippets i found online are not relevant
and/or obsolete.
Any suggestions on how to implement a HTTP server by using
Isolate.spawn will be really appreciated.
Many Thanks
Mike

--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to
http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google
Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send

--

For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups
"Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an

[Michele Costa]

Hi Filip,

i've updated the README: throughput using all the cores is now on par with
Ruby, about 27k reqs/sec.
Memory consumption was the same as before, but CPUs usage is now 240%
(before was 110%).

Numbers testing the server with WRK on localhost are higher, but so are for
all of the others languages.
The fact that Dart can use threads to deliver parallelism is a win-win to
me, compared to the pre-forking model of Node, Ruby or Python, all faster
to be fair...

Post by 'Filip Hracek' via Dart Misc
Hi Mike, I'd be interested to see how this impacted the results. Will you
be making an update to the table in README?

Post by Michele Costa
Hi all,
thanks for your support: i was now able to test versus a parallel server.
I am not aware it is so easy using the Isolate library.
Throughput is now better and distributed on available CPUs.
Best
Mike

Post by 'Zach Anderson' via Dart Misc
There's no need to use shelf.
https://github.com/costajob/app-servers/pull/23
On Tue, Oct 3, 2017 at 10:35 AM, 'Kevin Moore' via Dart Misc <

Post by 'Kevin Moore' via Dart Misc
Here you go! Using pkg/shelf – but you could do your own thing. Setting
`shared` to true is the key bit.
import 'dart:io';
import 'dart:isolate';
import 'package:shelf/shelf.dart' as shelf;
import 'package:shelf/shelf_io.dart' as io;
shelf.Response _echoRequest(shelf.Request request) =>
new shelf.Response.ok('Request for "${request.url}"');
main() async {
var isolates = 4;
for (int i = 1; i < isolates; i++) {
Isolate.spawn(_startServer, []);
}
_startServer();
}
_startServer([args]) async {
var server =
await HttpServer.bind(InternetAddress.LOOPBACK_IP_V4, 8080, shared: true);
io.serveRequests(server, _echoRequest);
}

Post by Michele Costa
Hi,
i am the maintainer of a micro-benchmark branch for several different
languages: i test a plain "Hello World" HTTP server by using the `wrk`
loading tool.
I've recently added Dart to the benchmark, by relying on the async
HTTP server implementation
<https://github.com/costajob/app-servers/blob/master/servers/dart_server.dart>
i've found on official tutorials.
The resulting throughput is not disappointing, but nonetheless far
from the Node.js one, which uses the cluster library to pre-fork several
processes.
Dart HTTP server implementation does not run in parallel, as confirmed
by inspecting the CPUs usage via Activity Monitor.
I read that Dart deliver parallelism on a multi-core server via the
Isolate library.
Unfortunately i have found no concrete examples of how to use Isolate
on the standard APIs and the code-snippets i found online are not relevant
and/or obsolete.
Any suggestions on how to implement a HTTP server by using
Isolate.spawn will be really appreciated.
Many Thanks
Mike

--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to
http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google
Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send

--

For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups
"Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an

[Patrice Chalin]

Hi Mike: the README refers to version 1.2.4 of Dart. I suspect that you
probably meant 1.24.2, right? --Patrice

Post by Michele Costa
Hi Filip,
i've updated the README: throughput using all the cores is now on par with
Ruby, about 27k reqs/sec.
Memory consumption was the same as before, but CPUs usage is now 240%
(before was 110%).
Numbers testing the server with WRK on localhost are higher, but so are
for all of the others languages.
The fact that Dart can use threads to deliver parallelism is a win-win to
me, compared to the pre-forking model of Node, Ruby or Python, all faster
to be fair...

Post by 'Filip Hracek' via Dart Misc
Hi Mike, I'd be interested to see how this impacted the results. Will you
be making an update to the table in README?

Post by Michele Costa
Hi all,
thanks for your support: i was now able to test versus a parallel server.
I am not aware it is so easy using the Isolate library.
Throughput is now better and distributed on available CPUs.
Best
Mike

Post by 'Zach Anderson' via Dart Misc
There's no need to use shelf.
https://github.com/costajob/app-servers/pull/23
On Tue, Oct 3, 2017 at 10:35 AM, 'Kevin Moore' via Dart Misc <

Post by 'Kevin Moore' via Dart Misc
Here you go! Using pkg/shelf – but you could do your own thing.
Setting `shared` to true is the key bit.
import 'dart:io';
import 'dart:isolate';
import 'package:shelf/shelf.dart' as shelf;
import 'package:shelf/shelf_io.dart' as io;
shelf.Response _echoRequest(shelf.Request request) =>
new shelf.Response.ok('Request for "${request.url}"');
main() async {
var isolates = 4;
for (int i = 1; i < isolates; i++) {
Isolate.spawn(_startServer, []);
}
_startServer();
}
_startServer([args]) async {
var server =
await HttpServer.bind(InternetAddress.LOOPBACK_IP_V4, 8080, shared: true);
io.serveRequests(server, _echoRequest);
}

Post by Michele Costa
Hi,
i am the maintainer of a micro-benchmark branch for several different
languages: i test a plain "Hello World" HTTP server by using the `wrk`
loading tool.
I've recently added Dart to the benchmark, by relying on the async
HTTP server implementation
<https://github.com/costajob/app-servers/blob/master/servers/dart_server.dart>
i've found on official tutorials.
The resulting throughput is not disappointing, but nonetheless far
from the Node.js one, which uses the cluster library to pre-fork several
processes.
Dart HTTP server implementation does not run in parallel, as
confirmed by inspecting the CPUs usage via Activity Monitor.
I read that Dart deliver parallelism on a multi-core server via the
Isolate library.
Unfortunately i have found no concrete examples of how to use Isolate
on the standard APIs and the code-snippets i found online are not relevant
and/or obsolete.
Any suggestions on how to implement a HTTP server by using
Isolate.spawn will be really appreciated.
Many Thanks
Mike

--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to
http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google
Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send

--

For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to
http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google
Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send

[Michele Costa]

Right! Just changed that, thanks!

Post by Patrice Chalin
Hi Mike: the README refers to version 1.2.4 of Dart. I suspect that you
probably meant 1.24.2, right? --Patrice

Post by Michele Costa
Hi Filip,
i've updated the README: throughput using all the cores is now on par
with Ruby, about 27k reqs/sec.
Memory consumption was the same as before, but CPUs usage is now 240%
(before was 110%).
Numbers testing the server with WRK on localhost are higher, but so are
for all of the others languages.
The fact that Dart can use threads to deliver parallelism is a win-win to
me, compared to the pre-forking model of Node, Ruby or Python, all faster
to be fair...

Post by 'Filip Hracek' via Dart Misc
Hi Mike, I'd be interested to see how this impacted the results. Will
you be making an update to the table in README?

Post by Michele Costa
Hi all,
thanks for your support: i was now able to test versus a parallel server.
I am not aware it is so easy using the Isolate library.
Throughput is now better and distributed on available CPUs.
Best
Mike

Post by 'Zach Anderson' via Dart Misc
There's no need to use shelf.
https://github.com/costajob/app-servers/pull/23
On Tue, Oct 3, 2017 at 10:35 AM, 'Kevin Moore' via Dart Misc <

Post by 'Kevin Moore' via Dart Misc
Here you go! Using pkg/shelf – but you could do your own thing.
Setting `shared` to true is the key bit.
import 'dart:io';
import 'dart:isolate';
import 'package:shelf/shelf.dart' as shelf;
import 'package:shelf/shelf_io.dart' as io;
shelf.Response _echoRequest(shelf.Request request) =>
new shelf.Response.ok('Request for "${request.url}"');
main() async {
var isolates = 4;
for (int i = 1; i < isolates; i++) {
Isolate.spawn(_startServer, []);
}
_startServer();
}
_startServer([args]) async {
var server =
await HttpServer.bind(InternetAddress.LOOPBACK_IP_V4, 8080, shared: true);
io.serveRequests(server, _echoRequest);
}

Post by Michele Costa
Hi,
i am the maintainer of a micro-benchmark branch for several
different languages: i test a plain "Hello World" HTTP server by using the
`wrk` loading tool.
I've recently added Dart to the benchmark, by relying on the async
HTTP server implementation
<https://github.com/costajob/app-servers/blob/master/servers/dart_server.dart>
i've found on official tutorials.
The resulting throughput is not disappointing, but nonetheless far
from the Node.js one, which uses the cluster library to pre-fork several
processes.
Dart HTTP server implementation does not run in parallel, as
confirmed by inspecting the CPUs usage via Activity Monitor.
I read that Dart deliver parallelism on a multi-core server via the
Isolate library.
Unfortunately i have found no concrete examples of how to use
Isolate on the standard APIs and the code-snippets i found online are not
relevant and/or obsolete.
Any suggestions on how to implement a HTTP server by using
Isolate.spawn will be really appreciated.
Many Thanks
Mike

--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to
http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google
Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it,

--

For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to
http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google
Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send

[Joe Conway]

There is still a lot left on the table here. On a four core machine, you can use 5 isolates before you get diminishing returns (here is a graph: https://aqueduct.io/docs/http/threading/). The Dart sample here uses three.

I'll put up a PR when I have a moment, or if someone can beat me to it: note that the sample put up oddly subtracts one from the number of isolates in the for loop, so just updating that isolate count variable isn't accurate.