...

We follow the prize committee's terminology in using the terms `confederate', `contestant', and `judge' for the computer program entrants, the humans being compared against, and the human interrogators performing the evaluation, respectively. We use the term `agent' for both confederates and contestants.

...

The confederate room referees, in addition to myself, were Susan Cole Dranoff, an attorney at the firm of Ropes and Gray, and Dr. Burton Dreben, Edgar Pierce Professor of Philosophy Emeritus, Harvard University. The judge room referees were Ned Block, Professor of Philosophy, Massachusetts Institute of Technology, Robert W. Furlong, patent attorney, and Dr. Robert Harford, Professor of Radiology, Thomas Jefferson University. Dr. Thomas Sheridan, Professor of Engineering, MIT, served in the auditorium.

...

All excerpts given below are taken verbatim from electronic transcripts of the competition provided by and copyright 1991 of the Cambridge Center for Behavioral Studies. No changes were made except for the adjustment of line breaks. In particular, spelling errors and extraneous characters were let stand.

...

This is not the only case in which exception has been taken to the appropriateness of the Turing test as a barometer of intelligence. See the discussion in the next section.

...

Daniel Dennett, the head of the prize committee, has himself argued against placing ``tacit restrictions on the lines of questioning of the judges'', calling this a ``a common misapplication of the sort of testing exhibited by the Turing test that often leads to drastic overestimation of the powers of actually existing computer systems.'' [emphasis in original]dennett-85

...

Dennett [6] uses the term ``parrying'' for the Eliza-like technique of randomly generating a canned response as an option of last resort, a key tool for implementers of PARRY's finesse.

...

Dennett [6] discusses this and other problems with the PARRY tests. Arbib [1] presents a contravening view, rejoined by Weizenbaum [30].

...

In fact, other limited Turing tests have been carried out as well. See the discussion by Moor [page 1129-30]moor-encyc for some examples.

...

``The flight [of the Gossamer Condor] has shown that, with what appears to be a comparatively unsophisticated design, controlled man-powered flight over a reasonable distance is possible.'' [page 341]reay

...

Several other factors markedly differentiate the Kremer and Loebner prizes. First, whereas the committee administering the Kremer prize consisted primarily of scientists specializing in the engineering of human-powered aircraft, it has been observed that current researchers in artificial intelligence, computational linguistics, and natural-language processing are conspicuous by their absence from the Loebner prize committee. (This problem has since been corrected.) Second, competition for the Kremer prize was on an as-needed, as opposed to regular, basis, and no prize was awarded until the prize test was completed in the presence of a qualified judge certified by the prize committee. Finally, the successful participants in the human-powered flight competitions were uniformly groups with strong backgrounds in the component technologies. In the case of the Loebner prize, the participants were almost without exception amateurs.

...

Dreyfus [7] provides pertinent examples.

...

Dr. Dennett has, on behalf of the Loebner prize committee, demanded that the advertising claim be discontinued, at peril of lawsuit, and Weintraub has apparently complied.

...

Robert Epstein has claimed that ``We have changed the Turing test as Turing would have if he were alive.'' [27] But it seems likely that Turing would have appreciated that the limitations imposed on the test by the Loebner committee invalidate it as even a sufficient criterion for intelligent behavior, and would not have sanctioned such gross modifications. An anonymous reviewer notes that ``none of the conditions assumed by Turing are redundant for a meaningful test - not the unlimited domain, not the unlimited time, not the interactive nature of the test, not the interrogator's full awareness that one of the respondents is a machine.''

...

Although the limitations and evaluation methods may be more sophisticated, the use of such task-limited evaluations to guide scientific research may be no more beneficial. (See the next section.)

...

Hubert Dreyfus [page 100]dreyfus has made a similar analogy of climbing trees to reach the moon.

...

Prize committee member Weizenbaum places the state of AI technology a bit later in his analogy with Newtonian physics [page 199]weizenbaum, Dreyfus a bit earlier in his analogy with alchemy [8]. Neither writer is, of course, sanguine about the prospects for progress in the coming centuries.

...

It is interesting to compare the Loebner prize with the Leibniz award for automatic theorem proving, endowed in 1983 by the Fredkin Foundation and administered by Carnegie-Mellon University. Like the Loebner prize, the Leibniz award offers $100,000 on the basis of an extremely difficult task; it is to be conferred on the occasion of the first major new mathematical theorem whose proof is found with essential contributions by automatic theorem proving. However, there are important differences. Awarding of the Leibniz prize is at the discretion of the Committee on Automatic Theorem Proving of the American Mathematical Society; it is therefore a subjective test, as it must be to decide issues such as the suitability of the theorem that was proved. In the interim, until the Leibniz prize is awarded, intermediate awards are occasionally (not annually) presented. The Milestone and Current Awards are conferred, respectively, for ``foundational work in automatic theorem proving'' and for ``ongoing research that shows promise'', again at the recommendation of the committee. The Current Award, as an award for present developments rather than past achievement, is therefore structured in much the same way as the present proposal.