[dart-misc] request for comment: cross-boundary messaging API

Discussion:

Istvan Soos

2017-03-20 09:29:57 UTC

Hi Dartisans,

I've drafted a high-level messaging API that could work in various
environments that need some kind of boundary cross (wire format and
object serialization, e.g. between web- and service workers, web
sockets...). My desire is that instead of sending a blob through
postMessage() and rolling your own parsing on the other side, most
applications could define their own data types and communication model
in a more apparent way.

The main idea is that each side of the pipe provides a 'hub':
https://github.com/isoos/message_hub/blob/master/lib/message_hub.dart#L5

You register adapters that handle the serialization and deserialization:
https://github.com/isoos/message_hub/blob/master/lib/message_hub.dart#L7

You register message handlers (a service that responds to requests):
https://github.com/isoos/message_hub/blob/master/lib/message_hub.dart#L16

And the rest of the API is either generic stream (sendMessage,
onMessage) or request-reply model (call, invoke).

I've also drafted three "transport handler" for web_worker, web_socket
and service_worker, which gives you an idea how this may work on
different transports:
https://github.com/isoos/message_hub/tree/master/lib

Please, please, please, let me know what you think about it! I'd like
to have a polished API that may be useful for different projects. It
is not used anywhere yet, and it is possible to change it in any way
that provides an improvement.

Thanks,
Istvan

--
For other discussions, see https://groups.google.com/a/dartlang.org/

For HOWTO questions, visit http://stackoverflow.com/tags/dart

To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to misc+***@dartlang.org.

j***@gmail.com

2017-03-20 15:47:27 UTC

Permalink

For the cross-wire format, wouldn't gRPC be a good option?

On Mon, Mar 20, 2017 at 11:30 AM +0200, "Istvan Soos" <***@gmail.com> wrote:

Hi Dartisans,

I've drafted a high-level messaging API that could work in various
environments that need some kind of boundary cross (wire format and
object serialization, e.g. between web- and service workers, web
sockets...). My desire is that instead of sending a blob through
postMessage() and rolling your own parsing on the other side, most
applications could define their own data types and communication model
in a more apparent way.

The main idea is that each side of the pipe provides a 'hub':
https://github.com/isoos/message_hub/blob/master/lib/message_hub.dart#L5

You register adapters that handle the serialization and deserialization:
https://github.com/isoos/message_hub/blob/master/lib/message_hub.dart#L7

You register message handlers (a service that responds to requests):
https://github.com/isoos/message_hub/blob/master/lib/message_hub.dart#L16

And the rest of the API is either generic stream (sendMessage,
onMessage) or request-reply model (call, invoke).

I've also drafted three "transport handler" for web_worker, web_socket
and service_worker, which gives you an idea how this may work on
different transports:
https://github.com/isoos/message_hub/tree/master/lib

Please, please, please, let me know what you think about it! I'd like
to have a polished API that may be useful for different projects. It
is not used anywhere yet, and it is possible to change it in any way
that provides an improvement.

Thanks,
Istvan
--
For other discussions, see https://groups.google.com/a/dartlang.org/

For HOWTO questions, visit http://stackoverflow.com/tags/dart

To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups "Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to misc+***@dartlang.org.

Istvan Soos

2017-03-20 16:09:52 UTC

Permalink

Post by j***@gmail.com
For the cross-wire format, wouldn't gRPC be a good option?

Interesting suggestion, but I'm not sure the scope of it. What part of
the gRPC wire format do you refer to as adopting? AFAICT it is mostly
HTTP2 headers + protobuf binary (or JSON proxy for REST-like
services).

Nevertheless, it would be interesting to see if we could use the `rpc`
package (or any of the similar rest-like codegen packages) on pub to
define the API and generate code for various environments...

Thanks,
Istvan

'Florian Loitsch' via Dart Misc

2017-03-20 17:59:56 UTC

Permalink

I didn't have the time to look in detail, but this looks similar to what I
tried to do with Mojo a year ago. [0]
Mojo has since been deprecated and been replaced with fidl [0], and someone
(I?) would need some time and effort to update my work from then. From what
I have heard the API hasn't changed that much, so I guess it wouldn't be
too bad.

Contrary to your (Istvan) approach, fidl requires a separate interface
language and a compiler. In return you get the advantage of fitting into
Fuchsia and lots of tools, like IDE support.

[0]
https://docs.google.com/document/d/1IVr0X8UkDE1tOyiDMJsBKx_wTspO_IlBLHqdgGJdHro
[1] https://fuchsia.googlesource.com/fidl/

Post by Istvan Soos
Hi Dartisans,
I've drafted a high-level messaging API that could work in various
environments that need some kind of boundary cross (wire format and
object serialization, e.g. between web- and service workers, web
sockets...). My desire is that instead of sending a blob through
postMessage() and rolling your own parsing on the other side, most
applications could define their own data types and communication model
in a more apparent way.
https://github.com/isoos/message_hub/blob/master/lib/message_hub.dart#L5
https://github.com/isoos/message_hub/blob/master/lib/message_hub.dart#L7
https://github.com/isoos/message_hub/blob/master/lib/message_hub.dart#L16
And the rest of the API is either generic stream (sendMessage,
onMessage) or request-reply model (call, invoke).
I've also drafted three "transport handler" for web_worker, web_socket
and service_worker, which gives you an idea how this may work on
https://github.com/isoos/message_hub/tree/master/lib
Please, please, please, let me know what you think about it! I'd like
to have a polished API that may be useful for different projects. It
is not used anywhere yet, and it is possible to change it in any way
that provides an improvement.
Thanks,
Istvan
--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups
"Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an

Istvan Soos

2017-03-22 09:36:12 UTC

Permalink

Thanks Florian,

I don't mind using a separate interface language and compiler, so fidl
is a great candidate for using it here.

I've been digging into what others are using, I've encountered WAMP.
It is a WebSocket sup-protocol, with all of the features I've been
looking for, and it has support on many platforms:
http://wamp-proto.org/

Does anybody have experience with it?

Thanks,
Istvan

On Mon, Mar 20, 2017 at 6:59 PM, 'Florian Loitsch' via Dart Misc

Post by 'Florian Loitsch' via Dart Misc
I didn't have the time to look in detail, but this looks similar to what I
tried to do with Mojo a year ago. [0]
Mojo has since been deprecated and been replaced with fidl [0], and someone
(I?) would need some time and effort to update my work from then. From what
I have heard the API hasn't changed that much, so I guess it wouldn't be too
bad.
Contrary to your (Istvan) approach, fidl requires a separate interface
language and a compiler. In return you get the advantage of fitting into
Fuchsia and lots of tools, like IDE support.
[0]
https://docs.google.com/document/d/1IVr0X8UkDE1tOyiDMJsBKx_wTspO_IlBLHqdgGJdHro
[1] https://fuchsia.googlesource.com/fidl/

--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups
"Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an

Istvan Soos

2017-05-17 14:47:08 UTC

Permalink

After a few more iterations, I've came up with a version that can act
as a low-level transport layer for a higher-level API like fidl or
other RPC that gets compiled from an IDL.

This is my cross-isolate example with complex objects that need some
serialization:
https://github.com/isoos/message_hub/blob/master/example/isolate_factorization.dart

There is a simpler echo service example alongside. If you have spare
time during IO, please take a look and let me know what you think
about it (and the direction I was taking).

Thanks,
Istvan

Post by Istvan Soos
Thanks Florian,
I don't mind using a separate interface language and compiler, so fidl
is a great candidate for using it here.
I've been digging into what others are using, I've encountered WAMP.
It is a WebSocket sup-protocol, with all of the features I've been
http://wamp-proto.org/
Does anybody have experience with it?
Thanks,
Istvan
On Mon, Mar 20, 2017 at 6:59 PM, 'Florian Loitsch' via Dart Misc

--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups
"Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an

'Florian Loitsch' via Dart Misc

2017-05-17 15:01:31 UTC

Permalink

Looks good.

You could consider letting the caller (the one that initiates everything)
to be able to set up the callee. Something like:

mainHub.bindService(
"factorization",
const FacturizationService());

class FacturizationService implements HubService {
const FacturizationService();

void run() { ... }
Map encodeRequest(Request x) { ... }
Request decodeRequest(...) { ... }
Map encodeResponse(Response x) { ... }
Response decodeResponse(...) { ... }
}

This way, you could dispatch to an isolate without needing lots of work on
each side. Basically, you could spawn a generic isolate, and then register
services from your side. This would keep the code a bit more grouped.

Post by Istvan Soos
After a few more iterations, I've came up with a version that can act
as a low-level transport layer for a higher-level API like fidl or
other RPC that gets compiled from an IDL.
This is my cross-isolate example with complex objects that need some
https://github.com/isoos/message_hub/blob/master/example/isolate_factorization.dart
There is a simpler echo service example alongside. If you have spare
time during IO, please take a look and let me know what you think
about it (and the direction I was taking).
Thanks,
Istvan

Post by 'Florian Loitsch' via Dart Misc
I didn't have the time to look in detail, but this looks similar to

what I