Pedro Moura called my attention to the distinction, only implicit in the tutorial, between evaluating a new application (to see it was worth while pursuing) and evaluating a particular application for which there was already consensus that it would be useful. (The example involving Resnik's multilingual gisting vs. e.g. MT in general).
António Colaço pointed the lack of clarity in the foil about Hindle & Rooth's work, namely instead of "followed by preposition" one should have written "NPs followed by preposition".
Alexsandro Soares pointed out that in the neural network community three kinds of corpora are standardly used: as well as pointed me to ftp://ftp.sas.com/pub/neural/FAQ.html#A_data, from where I reproduced the following excerpt:
There is no book in the NN literature more authoritative than Ripley (1996), from which the following definitions are taken (p.354):
He also mentioned that it might be interesting to compare evaluation contests within NLP with programming language contests like the ones in http://cristal.inria.fr/ICFP2001/prog-contest/, held in connection with the ICFP 2001, International Conference on Functional Programming.
Also, Alexsandro called my attention to the EAGLES Final Report at http://issco-www.unige.ch/ewg95/ewg95.html, of which I was unfortunately unaware during the tutorial's preparation.
I also came to read and consider highly relevant the discussion on cost-benefit analysis and technology assessment in general in Frederick Ferré's book on the philosophy of technology. (See the References page.)