-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathNOTE-reporters
66 lines (54 loc) · 2.94 KB
/
NOTE-reporters
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
==========================================================================
Sisyphus Reporters
==========================================================================
Sisyphus is a rather unique testing library. Assertions are just normal
functions that return a boolean result (in other words, "predicates"). I
don't know of any other testing library that uses plain predicates as
assertions. The trade for this simplicity, however, is that Sisyphus
reporters take on some extra complexity.
The complexity is not so much in the interface for reporters, but in
understanding how they work. Ultimately all a reporter needs to do is:
1) receive a message and modify it (if desired); then,
2) return a reporter for the next message.
In other words, a reporter is just a function that takes a message,
processes it, and returns another reporter (in the simplest case, itself).
So the complexity is not really in the design of reporters. It's in their
implementation. Functions that return functions in a (theoretically
endless) chain are sort of wonky and hard to wrap your head around. (These
are also sometimes known as 'generators' or 'iterators', although most
generator or iterator interfaces you can probably call to mind are
object-based, using classes, rather than function-based.)
The complexity really arises from this "message" concept. Basically,
reporters intercept "messages" from Sisyphus assertions. In other words,
Sisyphus streams information and results to the reporter while it is
evaluating a test suite.
Messages occur in predictable sequences. So, a reporter can take advantage
of its knowledge of the message types to behave appropriately. In
Sisyphus, the pattern looks like this:
1. For each defined test suite:
send 1 'test:suite' message
2. For each test in the suite:
send 1 'test:prerun' message
3. For each assertion in the test:
intercept the assertion predicate; this will send either:
- 2 messages of the following pattern:
- 1 'intercept/target:{assertion-name}/arguments'
- 1 'intercept/target:{assertion-name}/result'
- or, ≥ 3 messages of the following pattern:
- ≥ 1 sequences of:
- 1 'intercept/target:{assertion-name}/arguments'
- 1 'intercept/target:{assertion-name}/continuation'
- 1 terminating 'intercept/target:{assertion-name}/result'
4. Then, once the test is complete:
send 1 'test:result' message
So, a reporter is expected be:
1. Suite receiver
2. Test prerun receiver
3. Assertion interceptor receiver
4. Test result receiver
Conducting itself appropriately according to the message received, and
erroring if any of the sequence constraints described above is violated.
(For example, a reporter should fail fatally if it receives 'test:prerun'
twice in a row without receiving a 'test:result' in between.)
A basic reporter is implemented at `lib/reporter.mjs`, which can be used
as a reference for additional implementations.