How do tests work, when we need to rely on resources?

[sethladd]

I just tried upgrading from unittest to test. I'm now getting this error:

00:00 +18 -4: test/html_generator_test.dart: Templates function                                     
  Invalid argument(s): package:dartdoc/templates/index.html does not exist, tried /var/folders/00/0v8ph000h01000cxqpysvccm003d2t/T/dart_test_6OtYRh/packages/dartdoc/templates/index.html
  package:dartdoc/resource_loader.dart 120:5     _readFile.<async>

I am running this test via pub run test, as instructed.

Where did this temp folder come from? And how can I get the entire contents of my lib/ folder into that temp folder?

Specifically, I have:

lib/
  templates/
    index.html

Thanks!

[kevmoo]

How are you calculating your path?

Via script?

Directory.current should be the root of your project.

[sethladd]

Where does the temp folder come from, and how do my files get into that temp folder?

[sethladd]

Directory.current should be the root of your project.

Unfortunately, the resource loading logic can't make an assumption that a particular working directory is ever set. The resource loading logic needs to work with snapshots, pub run, pub serve, or from the command line. The platform.script is the "anchor", and from there I can look inside of packages/ for my resources.

[sethladd]

here is the logic for trying to find resources, compensating for all the ways we can run Dart code. Maybe it needs to compensate for the new test runner's conditions, too. https://github.com/dart-lang/dartdoc/blob/master/lib/resource_loader.dart

This is ugly because Dart doesn't make guarantees here, unfortunately.

[kevmoo]

@sethladd The temp dir is where we create the funky script that loads your test in an isolate. It's all internal to the test runner.

[sethladd]

What exactly is put into the temp dir? Just a "funky script"? Where can I learn more about this "funky script" ?

Is there a document on how the test runner works (and what guarantees it makes, where files are put, how isolates are involved, etc) ?

tldr: please help me understand what I can rely on, so I can find files in my lib/ dir of my project. The root issue is: my app needs to load files from within itself (currently, lib/). Thanks!

[nex3]

How do tests work, when we need to rely on resources?

Currently, browser tests can get resources from packages/<package>/<path>. VM tests can use a clever combination of Platform.script, Platform.packageRoot, and dart:mirrors to figure out where they exist on disk.

This is awkward and ad-hoc, which is why everyone's so excited about Lasse's proposal for a more uniform system. Such a system, supported by the core libraries, is necessary for having a means of loading resources that works well across all environments.

Unfortunately, the resource loading logic can't make an assumption that a particular working directory is ever set. The resource loading logic needs to work with snapshots, pub run, pub serve, or from the command line. The platform.script is the "anchor", and from there I can look inside of packages/ for my resources.

In general, Platform.script on its own isn't a reliable way to locate anything. It's too dependent on the way a program is used, which is never going to be entirely in the hands of the author. The same main() method may be invoked by running it directly with the dart executable, running a snapshotted version, running it over HTTP, running it in an isolate, or running another library that imports it and wraps its functionality.

Is there a document on how the test runner works (and what guarantees it makes, where files are put, how isolates are involved, etc) ?

The code itself is documented. The exact manner in which the test file is invoked is considered an implementation detail, but roughly speaking:

• The test runner decides it wants to load the tests in a given file.
• It creates a Dart file in a temporary directory that imports the test file and wraps its main() method with some logic to extract the tests.
• It starts an isolate running the newly-created Dart file.
  The VM recently landed support for running data: URIs directly, so we'll probably switch to that eventually instead of creating a temporary file.

We don't document that Platform.script is changed because it's not reliable in the first place, for reasons I outlined above. We expect a Dart script to be resilient to being invoked in multiple different ways by default.

[zoechi]

packages imports force the files into the lib directory. test could provide some information about the environment to the tests it runs including the project directory it was invoked from.

[sethladd]

It creates a Dart file in a temporary directory that imports the test file and wraps its main() method with some logic to extract the tests.

I wrote up a quick SO question/answer for how to do it here: http://stackoverflow.com/questions/30465381/how-can-i-dynamically-construct-a-dart-script-for-spawnuri/30465382#30465382

[nex3]

@zoechi That's covered by #110.