[dart-misc] Dart Language and Library Newsletter (2017-09-15)

['Florian Loitsch' via Dart Misc]

Github link:
https://github.com/dart-lang/sdk/blob/master/docs/newsletter/20170915.md
Earlier newsletters:
https://github.com/dart-lang/sdk/tree/master/docs/newsletter

Dart Language and Library Newsletter

Welcome to the Dart Language and Library Newsletter.
<https://github.com/dart-lang/sdk/blob/master/docs/newsletter/20170915.md#did-you-know>Did
You Know

In this (hopefully) recurring section, we will show some of the lesser
known features of Dart.
<https://github.com/dart-lang/sdk/blob/master/docs/newsletter/20170915.md#labels>
Labels

Dart's semantics introduces labels as follows:

A label is an identifier followed by a colon. A labeled statement is a
statement prefixed by a label L. A labeled case clause is a case clause
within a switch statement (17.9) prefixed by a label L. The sole role of
labels is to provide targets for the break (17.14) and continue (17.15)
statements.

Most of this functionality is similar to other languages, so most of the
following sections might look familiar to readers. I believe, Dart's
handling of continue in switch statements is relatively unique, so make
sure you read that section.
<https://github.com/dart-lang/sdk/blob/master/docs/newsletter/20170915.md#loops>
Loops

Labels are most often used as targets for break and continue inside loops.

Say you have nested loops, and want to jump to break or continue to the
outer loop. Without labels this wouldn't (easily) possible.

The following example uses continue label to jump from the inner loop
directly to the next iteration of the outer loop:

/// Returns the inner list (of positive integers) with the smallest
sum.List<int> smallestSumList(List<List<int>> lists) {
var smallestSum =0xFFFFFFFF; // The lists are known to have smaller sums.
var smallestList = null;
outer:
for (var innerList in lists) {
var sum = 0;
for (var element in innerList) {
assert(element >= 0);
sum += element;
// No need to continue iterating over the inner list. Its sum is already
// too high.
if (sum > smallestSum) continue outer; // <===== continue to label.
}
smallestSum = sum;
smallestList = innerList;
}
return smallestList;
}

This function runs through all lists, but stops adding up variables, as
soon as the sum is too high.

The same technique can be used to break out of an outer loop:

var firstListWithNullValues = null;
outer:for (var innerList in lists) {
for (var element in innerList) {
if (element == null) {
firstListWithNullValues = innerList;
break outer; // <====== break to label.
}
}
}// Now continue the normal work-flow.if (firstListWithNullValues != null) {
...
}

<https://github.com/dart-lang/sdk/blob/master/docs/newsletter/20170915.md#breaking-out-of-blocks>Breaking
out of Blocks

Labels can also be used to break out of blocks. Say we want to treat an
error condition uniformly, but have multiple conditions (potentially deeply
nested) that reveal the error. Labels can help structure this code.

void doSomethingWithA(A a) {
errorChecks:
{
if (a.hasEntries) {
for (var entry in a.entries) {
if (entry is Bad) break errorChecks; // <===== break out of block.
}
}
if (a.hasB) {
var b = a.b;
if (b.inSomeBadState) break errorChecks; // <===== break out of block.
}
// All looks good.
use(a);
return;
}
// Error case:
print("something bad happened");
}

A break to a block makes Dart continue with the statement just after the
block. From a certain point of view, it's a structured goto, that is only
allowed to jump to less-nested places that are after the current
instruction.
<https://github.com/dart-lang/sdk/blob/master/docs/newsletter/20170915.md#labels-in-switch>Labels
in Switch

Labels can also be used inside switches. They allow programs to continue with
another case clause. In its simplest form this can be used as a way to fall
through to the next clause:

void switchExample(int foo) {
switch (foo) {
case 0:
print("foo is 0");
break;
case 1:
print("foo is 1");
continue shared; // Continue at the clause that is marked `shared`.
shared:
case 2:
print("foo is either 1 or 2");
break;
}
}

Interestingly, Dart does *not* require the target of the continue to be the
clause that follows the current case clause. Any case clause with a label
is a valid target. This means, that Dart's switch statements are
effectively state machines.

The following example demonstrates such an abuse, where the whole switch is
really just used as a state machine.

void runDog() {
int age = 0;
int hungry = 0;
int tired = 0;

bool seesSquirrel() => new Random().nextDouble() < 0.1;
bool seesMailman() => new Random().nextDouble() < 0.1;

switch (0) {
start:
case 0:
print("dog has started");
continue doDogThings;

sleep:
case 1: // Never used.
print("sleeping");
tired = 0;
age++;
// The inevitable... :(
if (age > 20) break;
// Wake up and do dog things.
continue doDogThings;

doDogThings:
case 2: // Never used.
if (hungry > 2) continue eat;
if (tired > 3) continue sleep;
if (seesSquirrel()) continue chase;
if (seesMailman()) continue bark;
continue play;

chase:
case 3: // Never used.
print("chasing");
hungry++;
tired++;
continue doDogThings;

eat:
case 4: // Never used.
print("eating");
hungry = 0;
continue doDogThings;

bark:
case 5: // Never used.
print("barking");
tired++;
continue doDogThings;

play:
case 6: // Never used.
print("playing");
tired++;
hungry++;
continue doDogThings;
}
}

This function jumps from one switch clause to the next simulating the life
of a dog. In Dart, labels are only allowed on caseclauses, so I had to add
some case lines that will never be reached.

This feature is pretty cool, but it has been used extremely rarely. Because
of the added complexity for our compilers, we have frequently discussed its
removal. So far it has survived our scrutiny, but we might eventually
simplify our specification and make users add a while(true) loop (with a
label!) themselves. The dog example could be rewritten as follows:

var state = 0;
loop:while (true)
switch (state) {
case 0:
print("dog has started");
state = 2; continue;

case 1: // sleep.
print("sleeping");
tired = 0;
age++;
// The inevitable... :(
if (age > 20) break loop; // <===== break out of loop.
// Wake up and do dog things.
state = 2; continue;

case 2: // doDogThings.
if (hungry > 2) { state = 4; continue; }
if (tired > 3) { state = 1; continue; }
if (seesSquirrel()) { state = 3; continue; }
...

If the state values were named constants this would be as readable as the
original version, but wouldn't require the switchstatement to support state
machines.
<https://github.com/dart-lang/sdk/blob/master/docs/newsletter/20170915.md#synchronous-async-start>Synchronous
Async Start

This section discusses our plans to make async functions start
synchronously. This change is planned for Dart 2.0.
<https://github.com/dart-lang/sdk/blob/master/docs/newsletter/20170915.md#motivation>
Motivation

The current Dart specification requires that async functions are delayed:

If f is marked async (9), then a fresh instance (10.6.1) o implementing the
built-in class Future is associated with the invocation and immediately
returned to the caller. The body of f is scheduled for execution at some
future time.

For example:

Future<int> foo(x) async {
print(x);
return x + 1;
}
main() {
foo(499).then(print);
print("after foo call");
}

When this program is run, it emits the following output:

after foo call
499
500

The specification doesn't explain what precisely "at some future time"
means, but in practice async functions use scheduleMicrotask to start their
body.

There are some benefits to delaying the execution of async function bodies:

- It ensures that other code has the time to run. This helps to avoid
some race conditions: by delaying the execution of the body, it's harder to
accidentally interfere with the caller.
- Seeing an async keyword made it easy to detect that a function would
yield. This way async is mostly similar to await.

However, this approach also comes with drawbacks:

- Users tend to avoid async because it introduces latency.
- Users of APIs start to rely on the async modifier, which is an
implementation detail and should not be seen as part of the signature.
- Many users don't expect that the function is delayed.
- async functions cannot be used in many use-cases.

<https://github.com/dart-lang/sdk/blob/master/docs/newsletter/20170915.md#latency-issues>Latency
Issues

When programs need to fetch data from the server they often use async. This
makes sense: XMLHttpRequests are asynchronous, and waiting for them in an
async function is the easiest way to deal with the corresponding futures.
Often, programs start by fetching their resources as early as possible, so
that work is done in parallel with the request.

Some Googlers noticed big latency issues when using this approach. Because
of the immediate yield of async functions, these requests weren't sent
immediately, but the function was just bumped back in the microtask queue.
Only later, when the microtask queue was finally executing the body, did it
do the request. Often this delay was significant and noticeable.
<https://github.com/dart-lang/sdk/blob/master/docs/newsletter/20170915.md#relying-on-async>Relying
on async

Dart considers async an implementation detail. That is, as a user of an API
it doesn't matter if a function body is implemented with async or without.
This is the reason for having the async keyword after the function's
signature, and not as part of it. Since async is not part of the type /
signature, users may override async functions with synchronous functions,
use closures of either implementation approach interchangeably, or refactor
functions from one async to non-async or the inverse. In general, Dart
wants our users to see async functions similar to non-async functions (from
a user's point of view).

Despite these efforts, we see users that take the async as part of the
signature. Specifically, knowing that asyncimmediately returns, is used as
a part of the contract of a function. This is counter to how we envision
async to be used: since async is not part of the signature / type, a user
should be allowed to change the body from async to non-async.
<https://github.com/dart-lang/sdk/blob/master/docs/newsletter/20170915.md#expected-behavior-is-not-to-yield>Expected
Behavior is Not to Yield

During readability reviews we have seen code where the authors clearly
didn't expect the async function to yield. For example, we saw code like
the following:

class A {
bool isDoingRequest = false;

Future<String> doRequest(Uri link) async {
isDoingRequest = true;
return (await rpcCall()).data;
}

Future foo() async {
if (!isDoingRequest) {
var str = await doRequest(...);
}
}
}

In this example, some other function is testing for the value of the
isDoingRequest. If that field is set to false, it invokes doRequest, which,
in the first line, sets the value to true. However, because of the
asynchronous start, the field is not set immediately, but only in the next
microtask. This means that other calls to foo might still see the
isDoingRequest as falseand initiate more requests.

Running synchronously also brings Dart in line with other languages, like
Ecmascript. <footnote>C# also executes the body of async functions
synchronously. However, C# doesn't guarantee, that await always yields. If
a Task (the equivalent class for Future) is already completed, C# code may
immediately continue running at the await point.`
<https://github.com/dart-lang/sdk/blob/master/docs/newsletter/20170915.md#required-changes>Required
Changes

Switching to synchronous starts of async functions require changes in the
specification and in our tools.

The tool changes are relatively small, since few code touches the async/
await functionality. A prototype CL for the VM and dart2js can be found
here: https://dart-review.googlesource.com/c/sdk/+/5263

The specification has already been updated with
https://github.com/dart-lang/sdk/commit/2170830a9e41fa5b4067fde7bd44b76f5128c502
<https://github.com/dart-lang/sdk/blob/master/docs/newsletter/20170915.md#migration>
Migration

Running async functions synchronously is a subtle change that might break
programs in unexpected ways. Most programs don't depend on the additional
yield on purpose, but some may depend on it by accident. We are aware that
this change has the potential to cause big headaches.

Once the patch is complete we intend to roll it out behind a flag. This
way, users can start experimenting without being forced to switch in one
go. With a bit of luck, most programs just continue working (or the reason
for failures is obvious).

If necessary, a full program search-and-replace can also bring back the old
behavior:

// Before:Future foo() async { doSomething(); }Future bar() async =>
doSomething();// After:Future foo() async { await null; doSomething();
}Future bar() async { await null; return doSomething(); }

This transformation is purely syntactic, and preserves the old behavior if
done at the same time as the switch to the new semantics. Note that a
slightly more advanced transformation would pay attention not to return a
void value in the bar case above. However, it would be probably easier to
just fix those by hand.

Depending on the feedback and our own experience of migrating Google's
whole codebase, we could also add a temporary flag to our tools that
maintains the old behavior.