[Termtools] A proposal for the next competition (Scoring)

[Johannes Waldmann]

Koprowski, A. wrote:

> But how about instead of rewarding fast responses 
> reward solving difficult problems?
> For instance we could give bonus points 
> for solving TRSs that no other tool could solve. 

The SAT and CASC competitions have rules for that,
see the URL I posted http://www.satcompetition.org/2005/whatsnew05.html


Scoring Rationale

Each problem (benchmark) has a "prize" for solving it, or a "purse" as
they say in horse racing. The standard purse is 1000 points.

If k solvers solve the problem within the time limit, they split the
purse k ways.

This provides a natural way to give more credit for solving "hard"
problems without introducing arbitrary weights to measure "hardness" --
the winners simply split the purse among themselves.

A state-of-the-art contributor (SOTAC) solver in the CASC terminology is
one that solved some problem that no one else solved. That is an
automatic 1000 and is likely to be worth more than being great at
"stealing candy from babies", i.e., knocking off all the easy problems.
-- 
-- Johannes Waldmann -- Tel/Fax (0341) 3076 6479/80 --
---- http://www.imn.htwk-leipzig.de/~waldmann/ -------

_______________________________________________
Termtools mailing list
Termtools at serveur-listes.lri.fr
http://serveur-listes.lri.fr/mailman/listinfo/termtools