[dart-misc] How does the Dart VM work?

[Benjamin Summerton (@def-pri-pub)]

I posted this question here on the GitHub repo for the SDK a while back
<https://github.com/dart-lang/sdk/issues/26967>, but they redirected me
here to ask it.

So, how exactly does the Dart VM work (under the hood)? I've read a few of
the documents on isolates and whatnot, but I'm wondering about things like
how does my code get JIT'd at runtime (for platforms like x86 or ARM),
possible performance hangups, etc. So maybe I should ask, "Hoes does my
'Hello World' program go from source code to stdout in the terminal?" How
does it run through the Dart VM?

[Filipe Morgado]

Hey :)

I have nothing to add, except more doubts.

I've been using the VM without knowing exactly what's going on (lack of
time/incentive to investigate).

Sorry to hijack the thread, but there has been a lot of stories.

Implemented in Dart, We have dart2js, DevCompiler
<https://github.com/dart-lang/dev_compiler> and Kernel
<https://github.com/dart-lang?query=kernel>. It seems Kernel is a
foundation to target any external environment, specially JS, still being
working on.
DevCompiler, base on Kernel?, may supplant dart2js.

Then we have the regular VM and AoT compilation implemented in C++. Are
those always bundled together?

When I run the VM, it seems to JITs my code directly to unoptimized machine
code, then it optimizes code for hot loops. It runs well.

If my server-side VM bundles with AoT compilation, can can I take advantage
of it, considering I'm (currently) on a x64?

Is all the work being invested in Kernel and AoT ever gonna produce
performance results in a regular VM?

Thank you :)

On Friday, 30 September 2016 22:28:10 UTC+1, Benjamin Summerton

Post by Benjamin Summerton (@def-pri-pub)
I posted this question here on the GitHub repo for the SDK a while back
<https://github.com/dart-lang/sdk/issues/26967>, but they redirected me
here to ask it.
So, how exactly does the Dart VM work (under the hood)? I've read a few
of the documents on isolates and whatnot, but I'm wondering about things
like how does my code get JIT'd at runtime (for platforms like x86 or ARM),
possible performance hangups, etc. So maybe I should ask, "Hoes does my
'Hello World' program go from source code to stdout in the terminal?" How
does it run through the Dart VM?

[Joao Pedrosa]

Hi,

Post by Benjamin Summerton (@def-pri-pub)
I posted this question here on the GitHub repo for the SDK a while back
<https://github.com/dart-lang/sdk/issues/26967>, but they redirected me
here to ask it.
So, how exactly does the Dart VM work (under the hood)? I've read a few
of the documents on isolates and whatnot, but I'm wondering about things
like how does my code get JIT'd at runtime (for platforms like x86 or ARM),
possible performance hangups, etc. So maybe I should ask, "Hoes does my
'Hello World' program go from source code to stdout in the terminal?" How
does it run through the Dart VM?

The Dart VM borrowed many ideas from the JavaScript VM used by Chrome
called V8. Recall that the Dart VM
was supposed to be included in the browser too, so it had to support
features that were friendly to the browser
and that is why it ended up sharing some of the same concerns of the JS VM.

For example as you mentioned the isolates is an idea from the browser. At
the beginning, they wanted to
support lightweight isolates in case Dart needed even more concurrency. But
once async got in, they seemed
to settle on heavier isolates with the idea of managing at least 1 isolate
per CPU thread. The isolates used
more memory as a result, so if say we wanted thousands of them, we could
run into memory issues instead.

Once the Dart developers took up async, they started adding support for
Zones. Where each isolate was also
a Zone that handled the async transparently. One of the ideas they
introduced later with Zone was that they
supported a capturing of the async exception at the last minute to help
developers handle it and potentially
survive the exception. With async screwing up the stacktraces, they had
other projects that helped to build
a more developer friendly stacktrace to help with debugging, but I think it
used a separate package for the
frontend.

In the JS VM, they used as a first step the compilation to unoptimized
native code. I think the Dart VM did
the same. For example, the Dart VM has only recently started adopting some
more interpretation to help
Dart in devices that disallow JIT or have less memory for the JIT'ed code.
So that the Dart VM has started
to add more layers to its execution.

One of the major differences between the JS and Dart optimization
strategies revolves around the static
methods. In Dart, classes and objects don't change at runtime, so once
compiled the calling the methods
is close to the optimized native speed. This trade-off allows Dart to be
quicker with less work for the runtime.

The Dart VM seems to be register based. Dart has some optimizations for
each CPU that it targets,
with some layer of assembly code to make it faster I think. Otherwise, a
lot of the higher level APIs are either
written in C++ or in Dart itself.

Since Dart code is compiled to native code right away making it fast enough
out of the box, further optimizations
would come from being able to remove dead code, inlining method calls or
moving code out of the hot loops
if possible (hoisting.) And so on? :-) Those extra steps can be done by the
JIT after it notices that code
has become hot after say being called hundreds of times. Say call a
function 100 times and the JIT may
try to optimize it further. If the JIT decided that it messed it up, it
could try to deoptimize it by running the
old code that it had just compiled to native code when it loaded up the
source code. Ah! This reminds me
though. In JS they seem to keep the JS source code around in case they need
to visit it again after an
optimization or deoptimization call. Dart may be similar, it is possible
that the VM keeps the Dart's AST
around in case it needs it later. One of the differences between a bytecode
and a source code VM is that
the source code VM may be visiting the code again and again since it
provides more information for the
compilers.

That brings us to where Dart has been heading, though. The Dart VM is
changing. Dart has started to adopt
ahead-of-time (AoT) compilation as another deployment strategy helpful for
the mobile platforms. Also, Dart
still needs the interpreter just in case, perhaps. There are developers
working on something in Dart called
the Kernel. The Kernel would be a reduced version of Dart that would be a
better target for compilation.

So, how would a Hello World type of application go?

1. $ dart hello_world.dart
2. Dart VM parses the AST and compiles it to native code, wrapping it
inside the isolate and Zone.
3. Dart VM runs the code's main method. Sees the print("Hello World!")
command which would
be in native code at this point. This would be a call to the C++ print API
with the static string converted
to a C++ string by now. Runs it and exits the main method and the program.

When Dart code interfaces with C++ code, it is possible that some
transformation may be necessary.
The compiler can spare some of it when it does its job. That's one of the
differences between an
interpreter and a compiler. But JS by comparison is dynamic, and in the
JS's case it is possible that
some code could have changed the print API (console.log) so they have to
guard it against changes
when they are running it, whereas in Dart changing the APIs at runtime is
largely forbidden. So that's
why Dart lends itself to compiling a bit better than JS does.

How does the JIT work? I think that on function calls (native code) they
have a counter variable for each
function. If the function gets called say 100 times, they could run the JIT
on it. But before they do, they
would need to take the time it took to run the function and store it for
later, and when they run its
JIT version they can compare it to decide whether they need to deoptimize
it and try again.

The Dart VM was supposed to be much simpler than the JS one. Since some of
the core Dart VM
developers also worked on the JS one and learned from it. Unfortunately the
Dart VM didn't make
it into the browser though. And on mobile it seems that concerns like
battery life, memory usage
and security create a hassle for VMs.

Cheers,
Joao

[Benjamin Summerton (@def-pri-pub)]

Thanks! That's pretty interesting. Is there a Dart VM developer we could
ping to add some more to the discussion?

[Joao Pedrosa]

Hi,

Post by Benjamin Summerton (@def-pri-pub)
Thanks! That's pretty interesting. Is there a Dart VM developer we could
ping to add some more to the discussion?

They hang around here. I'm not aware of any articles or videos talking
about the Dart VM in detail, though.
You could try to compare what they have done in the Dart VM by studying up
the JS VM which seems to
be talked about more.

On the JS VM I have watched some recent videos on youtube:
https://www.youtube.com/user/blinkontalks/videos

Specifically this one about the new interpreter of JS called Ignition:

Like I said before, the Dart VM and the JS VM share many properties and
concerns. They have started
diverging more as of late once the JS VM introduced a new engine called
TurboFan which is said to
deal with a Sea of Nodes. But TurboFan was more targeted at high
performance math and will have to
catch up to the "traditional" engine used by both JS and Dart that does a
better job at "normal" code. If
you watch the video about Ignition, you will know that the challenge they
have is kind of interesting.

The Dart VM had maybe around 5 core designers. A couple of them may be left
maintaining it now. They
may be helped by some new developers though. Since the interest in the Dart
VM died down a little once
they could not have it on the browser, some of them may have moved on a
little bit. They tried their hands
at other VMs instead like Dartino for IoT which didn't pan out.

As Anders Hejlsberg from Microsoft alluded to in a recent video when
talking about modern day compilers,
some of these compilers are not found in the books yet. They are being
invented by these top technological
companies. When Chrome innovated the JS VM, all the other browser companies
took notice and started
borrowing ideas so that they could improve their own engines too. These
days, Firefox and Edge (IE) have
caught up. That may partly explain why these developers are not in the
teaching business.

If you want, you can try to talk to one of the JS and Dart VM developers
over on Twitter:
https://twitter.com/mraleph

The JS TurboFan developer is this one:
https://twitter.com/bmeurer

Beware though that the VM has both the lower level tidbits and the higher
level stuff to make it work. A single
developer may not cover all of it.

I share your curiosity though. On the NodeJS project, they have debugging
tools that help to explain how it
all works. The Dart VM may have some debug APIs too. You could try to find
some of those. Be on the
lookout for material explaining how the JS VM JIT works. It should be
similar.

Here's a curiosity. The Dartino VM used bytecodes:
https://github.com/dartino/sdk
But the Dartino VM was still interpreted. So it was relatively slow. But
Dart kind of needed an Interpreter.
The Ignition VM for JavaScript is also based on bytecodes and interpreted.
JavaScript also needs an
interpreter for devices that don't allow for JIT or that have restrictions.
An interpreter supposedly helps with
the faster startup too. With the Ignition VM, they will use the TurboFan
compiler to generate optimized
code. Maybe they wanted to have an extra layer of optimization on top of
Dartino too, but they didn't get
to that point unfortunately. My feeling is that they are on that treadmill
of we have 5 standards, we need to
fix them, we create a new standard and we now have 6 standards instead. :-)

Cheers,
Joao

[Daniel Morilha]

This question comes once in a while and unfortunately I have never found a
good high level document explaining the vm internals.

Dart is more than its VM. It is a programming language project led by
google.

As pointed out, the original vm was originally designed to replace
javascript on the browser, the language however is purpose free. Dartino
implements a very different vm yet relying on the same language.

The dart vm which runs on the server and most recently has been adapted to
run on mobile consists of a C++ runtime (memory manager, scheduler,
profiler, snapshots, libraries...) and the compiler part (parser, compiler,
optimizer, assembler). The core of the VM[1] can be easily embedded with
only limited dependencies into their host OS. When running on the server
(aka standalone[2]) dart exposes additional libraries to deal with IO.
There is a considerable amount of dart written libraries as well[3].
Libraries written in dart use the same infrastructure as the code that the
user writes.

the DartVM implements AOT, JIT and a Dart Byte Code (DBC) interpreter for a
mix of architectures. The language was designed to rely on JIT techniques
so performance for AOT may not compare. However keep in mind AOT part of
the project is new and work has been done to speed it up[4]. In fact the
whole introduction for AOT started when the project started pursuing the
mobile platform rather than web.

The compiler infrastructure within the VM is robust and massive. IMO it is
what makes reading the source code complicated. If you are not into
compilers, it is very difficult to keep up with all the concepts
(intrinsics, arity, canonicalization, class hierarchy, different call
conventions). Nevertheless it is a very well designed project yet pragmatic
given all it does.

The hello world program, gets parsed and compiled to the machine's
architecture. Your console calls tie with the libraries for your operating
system. In the end it is just the same kernel call you would see with your
C code. There are lots of abstractions and layers for that to happen. The
key for performance is to remove the abstraction as much as possible for
heavily executed code. Once you have machine code the C++ runtime jumps
into it and your compiled code takes over the CPU, for certain operations
it goes back to the C++ runtime or even to something external to the vm.

There is a short video which helps to understand the single compiler[5].
Bear in mind that at that time there was a lot of comparisons with
javascript, which IMO should not make sense at this point given their
different goals.

HTH

[1]: https://github.com/dart-lang/sdk/tree/master/runtime/vm
[2]: https://github.com/dart-lang/sdk/tree/master/runtime/bin
[3]: https://github.com/dart-lang/sdk/tree/master/sdk/lib
[4]: https://codereview.chromium.org/2379733002/#msg11
[5]:

Post by Joao Pedrosa
Hi,

Post by Benjamin Summerton (@def-pri-pub)
Thanks! That's pretty interesting. Is there a Dart VM developer we
could ping to add some more to the discussion?

They hang around here. I'm not aware of any articles or videos talking
about the Dart VM in detail, though.
You could try to compare what they have done in the Dart VM by studying up
the JS VM which seems to
be talked about more.
https://www.youtube.com/user/blinkontalks/videos
http://youtu.be/r5OWCtuKiAk
Like I said before, the Dart VM and the JS VM share many properties and
concerns. They have started
diverging more as of late once the JS VM introduced a new engine called
TurboFan which is said to
deal with a Sea of Nodes. But TurboFan was more targeted at high
performance math and will have to
catch up to the "traditional" engine used by both JS and Dart that does a
better job at "normal" code. If
you watch the video about Ignition, you will know that the challenge they
have is kind of interesting.
The Dart VM had maybe around 5 core designers. A couple of them may be
left maintaining it now. They
may be helped by some new developers though. Since the interest in the
Dart VM died down a little once
they could not have it on the browser, some of them may have moved on a
little bit. They tried their hands
at other VMs instead like Dartino for IoT which didn't pan out.
As Anders Hejlsberg from Microsoft alluded to in a recent video when
talking about modern day compilers,
some of these compilers are not found in the books yet. They are being
invented by these top technological
companies. When Chrome innovated the JS VM, all the other browser
companies took notice and started
borrowing ideas so that they could improve their own engines too. These
days, Firefox and Edge (IE) have
caught up. That may partly explain why these developers are not in the
teaching business.
If you want, you can try to talk to one of the JS and Dart VM developers
https://twitter.com/mraleph
https://twitter.com/bmeurer
Beware though that the VM has both the lower level tidbits and the higher
level stuff to make it work. A single
developer may not cover all of it.
I share your curiosity though. On the NodeJS project, they have debugging
tools that help to explain how it
all works. The Dart VM may have some debug APIs too. You could try to find
some of those. Be on the
lookout for material explaining how the JS VM JIT works. It should be
similar.
Here's a curiosity. The Dartino VM used bytecodes: https://github.com/
dartino/sdk
But the Dartino VM was still interpreted. So it was relatively slow. But
Dart kind of needed an Interpreter.
The Ignition VM for JavaScript is also based on bytecodes and interpreted.
JavaScript also needs an
interpreter for devices that don't allow for JIT or that have
restrictions. An interpreter supposedly helps with
the faster startup too. With the Ignition VM, they will use the TurboFan
compiler to generate optimized
code. Maybe they wanted to have an extra layer of optimization on top of
Dartino too, but they didn't get
to that point unfortunately. My feeling is that they are on that treadmill
of we have 5 standards, we need to
fix them, we create a new standard and we now have 6 standards instead. :-)
Cheers,
Joao
--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups
"Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an

--
Daniel Morilha (***@gmail.com)

['Vyacheslav Egorov' via Dart Misc]

If you are curious about how Dart VM compiles your Dart code into native
code then you can start by watching my talk "Building an Optimizing (JIT)
Compiler for Dart" from Strange Loop 2013:

Video: https://www.infoq.com/presentations/dart-compiler
Slides: http://mrale.ph/s3/StrangeLoop2013.pdf

It outlines most important ideas that Dart VM uses to JIT compile Dart
code. (spoiler alert: most of these ideas are coming from Self VM from 90s).

AOT is a completely different story. There will be a talk about Dart VM AOT
mode on the Dart Summit this year. Make sure you see it - if you are
interested in the challenges that AOT compilation is facing.

Cheers,

Post by Daniel Morilha
This question comes once in a while and unfortunately I have never found a
good high level document explaining the vm internals.
Dart is more than its VM. It is a programming language project led by
google.
As pointed out, the original vm was originally designed to replace
javascript on the browser, the language however is purpose free. Dartino
implements a very different vm yet relying on the same language.
The dart vm which runs on the server and most recently has been adapted to
run on mobile consists of a C++ runtime (memory manager, scheduler,
profiler, snapshots, libraries...) and the compiler part (parser, compiler,
optimizer, assembler). The core of the VM[1] can be easily embedded with
only limited dependencies into their host OS. When running on the server
(aka standalone[2]) dart exposes additional libraries to deal with IO.
There is a considerable amount of dart written libraries as well[3].
Libraries written in dart use the same infrastructure as the code that the
user writes.
the DartVM implements AOT, JIT and a Dart Byte Code (DBC) interpreter for
a mix of architectures. The language was designed to rely on JIT techniques
so performance for AOT may not compare. However keep in mind AOT part of
the project is new and work has been done to speed it up[4]. In fact the
whole introduction for AOT started when the project started pursuing the
mobile platform rather than web.
The compiler infrastructure within the VM is robust and massive. IMO it is
what makes reading the source code complicated. If you are not into
compilers, it is very difficult to keep up with all the concepts
(intrinsics, arity, canonicalization, class hierarchy, different call
conventions). Nevertheless it is a very well designed project yet pragmatic
given all it does.
The hello world program, gets parsed and compiled to the machine's
architecture. Your console calls tie with the libraries for your operating
system. In the end it is just the same kernel call you would see with your
C code. There are lots of abstractions and layers for that to happen. The
key for performance is to remove the abstraction as much as possible for
heavily executed code. Once you have machine code the C++ runtime jumps
into it and your compiled code takes over the CPU, for certain operations
it goes back to the C++ runtime or even to something external to the vm.
There is a short video which helps to understand the single compiler[5].
Bear in mind that at that time there was a lot of comparisons with
javascript, which IMO should not make sense at this point given their
different goals.
HTH
[1]: https://github.com/dart-lang/sdk/tree/master/runtime/vm
[2]: https://github.com/dart-lang/sdk/tree/master/runtime/bin
[3]: https://github.com/dart-lang/sdk/tree/master/sdk/lib
[4]: https://codereview.chromium.org/2379733002/#msg11
[5]: http://youtu.be/bNESI-d-tBQ
Hi,
Thanks! That's pretty interesting. Is there a Dart VM developer we could
ping to add some more to the discussion?
They hang around here. I'm not aware of any articles or videos talking
about the Dart VM in detail, though.
You could try to compare what they have done in the Dart VM by studying up
the JS VM which seems to
be talked about more.
https://www.youtube.com/user/blinkontalks/videos
http://youtu.be/r5OWCtuKiAk
Like I said before, the Dart VM and the JS VM share many properties and
concerns. They have started
diverging more as of late once the JS VM introduced a new engine called
TurboFan which is said to
deal with a Sea of Nodes. But TurboFan was more targeted at high
performance math and will have to
catch up to the "traditional" engine used by both JS and Dart that does a
better job at "normal" code. If
you watch the video about Ignition, you will know that the challenge they
have is kind of interesting.
The Dart VM had maybe around 5 core designers. A couple of them may be
left maintaining it now. They
may be helped by some new developers though. Since the interest in the
Dart VM died down a little once
they could not have it on the browser, some of them may have moved on a
little bit. They tried their hands
at other VMs instead like Dartino for IoT which didn't pan out.
As Anders Hejlsberg from Microsoft alluded to in a recent video when
talking about modern day compilers,
some of these compilers are not found in the books yet. They are being
invented by these top technological
companies. When Chrome innovated the JS VM, all the other browser
companies took notice and started
borrowing ideas so that they could improve their own engines too. These
days, Firefox and Edge (IE) have
caught up. That may partly explain why these developers are not in the
teaching business.
If you want, you can try to talk to one of the JS and Dart VM developers
https://twitter.com/mraleph
https://twitter.com/bmeurer
Beware though that the VM has both the lower level tidbits and the higher
level stuff to make it work. A single
developer may not cover all of it.
I share your curiosity though. On the NodeJS project, they have debugging
tools that help to explain how it
all works. The Dart VM may have some debug APIs too. You could try to find
some of those. Be on the
lookout for material explaining how the JS VM JIT works. It should be
similar.
https://github.com/dartino/sdk
But the Dartino VM was still interpreted. So it was relatively slow. But
Dart kind of needed an Interpreter.
The Ignition VM for JavaScript is also based on bytecodes and interpreted.
JavaScript also needs an
interpreter for devices that don't allow for JIT or that have
restrictions. An interpreter supposedly helps with
the faster startup too. With the Ignition VM, they will use the TurboFan
compiler to generate optimized
code. Maybe they wanted to have an extra layer of optimization on top of
Dartino too, but they didn't get
to that point unfortunately. My feeling is that they are on that treadmill
of we have 5 standards, we need to
fix them, we create a new standard and we now have 6 standards instead. :-)
Cheers,
Joao
--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups
"Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an
--
--
For other discussions, see https://groups.google.com/a/dartlang.org/
For HOWTO questions, visit http://stackoverflow.com/tags/dart
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups
"Dart Misc" group.
To unsubscribe from this group and stop receiving emails from it, send an

--
// Vyacheslav Egorov

[kc]

I made a similar point here:

https://groups.google.com/a/dartlang.org/d/msg/misc/hVUGMpf9x9w/yvlv4w-3AgAJ

The Dart VM is way too much of a black box.

Some info on how the the Dart VM is engineered - especially to interact
with the Flutter engine (with a nice surface syntax) - would help make the
sale.

K.


On Sunday, October 2, 2016 at 1:48:22 AM UTC+1, Benjamin Summerton

Thanks! That's pretty interesting. Is there a Dart VM developer we could
ping to add some more to the discussion?