Ticket #24 (assigned enhancement)

Reporter


David Herron
Opened: 03/8/11
Last modified: 05/9/11
Status: assigned
Type: enhancement

Owner


Nicholas C. Zakas
Target Release: NEXT
Priority: P3 (normal)
Summary: Suggestions on YUITest for Node cli.js
Description:

I've been packaging this for internal use at Yahoo and have a couple suggestions for the cli.js that will make it more directly usable.

a) rename --options to --resultsformat (or some similar option name)

b) add --results <filename> ... that will specify a file the results should go to, rather than scribbling them on the console

c) yuitest-node has support for writing a code coverage report, but cli.js doesn't include such support. A pair of options, --coverageformat and --coverage, would enable gathering code coverage data

d) cli.js has require('./lib/yuitest-node.js') which won't work when cli.js is installed as bin/yuitest (as its package.json says to do). It should instead require('yuitest') and the yuitest-node.js
file should be installed in lib/yuitest/index.js along with a package.json

The last point is because it's redundant for a Node program to require a module whose name includes the string "node". There's a caution about this inside the NPM documentation.

Type: enhancement Observed in Version: development master
Component: YUI Test for Node.js Severity: S3 (normal)
Assigned To: Nicholas C. Zakas Target Release: NEXT
Location: Library Code Priority: P3 (normal)
Tags: Relates To:
Browsers: N/A
URL:
Test Information:

Change History

Nicholas C. Zakas

YUI Developer

Posted: 03/9/11
  • location changed to Library Code
  • milestone changed to NEXT
  • priority changed to P3 (normal)
  • status changed from new to accepted

For (a), I'm assuming you mean --output instead of --options. I actually had this on my TODO list to change to --format.

I'll take a look at the others as well. I had planned on adding code coverage support, I've just not had time to get to it yet.

Nicholas C. Zakas

YUI Developer

Posted: 03/15/11

(a) and (d) are complete and checked in.

I'm holding off on (b) for now pending some other changes. For the time being, you can redirect output using > output.txt

(c) will be included in a future edition.

David Herron

Posted: 03/30/11

I just checked out the latest build and see the --format option. Positive improvement.

The only quibble over redirecting output to a file is that it prevents the test author from having debugging output, which is a common usage. Yeah, looking forward to (b) being implemented. BTW it appears the code is using stderr.write and sys.print in different places and I don't know if that's on purpose or accidental.

David Herron

Posted: 03/30/11

It appears that the results get truncated.

I have a test suite(s) of a couple hundred tests run this way: yuitest --format junitxml test.js test_cache.js

I've arranged it so all of my tracing output goes to a separate file leaving the console for the test results (see previous comment for preferred behavior). The XML of the results doesn't fully print, instead it stops in mid-tag and my shell prompt gets printed. Hence the sys.print in YUITest.Node.CLI.Format must not be printing everything it gets.

As a quick hack idea I changed that to console.log and it had essentially the same behavior.

Another idea I had was this:

if (!process.stdout.write(format(results) + "n")) {
while (!process.stdout.write(" ")) { }
}

But it just got into a really long pause obviously due to the obvious infinite loop. It tells me that what's going on is stdout.write is returning false and according to the node.js documentation this means the output hasn't been fully flushed. Somehow what's needed is to pause and allow the output to drain...?

David Herron

Posted: 03/31/11

Spent awhile working on debugging this...

There's a note in the nodejs@googlegroups.com list about output being cut off if there's a process.exit(1) (the thread title is "redirecting output & exiting the process")

I found that hacking the handler in YUITest.Node.CLI.XUnit to call process.stdout.flush() several times, then all the output would go out.

I found that among the output not occurring is a COMPLETE_EVENT handler like this where it's using openSync, writeSync, closeSync to write the output. Somehow this handler never got called. Perhaps because of the handler in cli.js - which I've commented out. The test suite has failed tests, and that handler process.exit(1)'s if there are failures. Clearly doing exit(1) is useful but it needs to be coordinated with ensuring all the output occurs.

YUITest.TestRunner.subscribe(
YUITest.TestRunner.COMPLETE_EVENT,
function(event) {
log("test_cache " + event.type);
if (event.results) {
var res = YUITest.TestFormat.JUnitXML(event.results);
log('RESULTS: results-test_cache.xml');
trace.writeLogFile("results-test_cache.xml", res);
}
}
);

Nicholas C. Zakas

YUI Developer

Posted: 05/9/11