[Termination tools] Competition 2005 is over !

[Claude Marche]

Dear all,

Please allow me to write a few comments about the competition
2005. Since we have no meeting to discuss about the competition, like
last year, I encourage people to write their own comments to this list.

First, I would like to thank the new entrants, Jambox, TEPARLA and
TPA, which performed very well, and contributed to make the
competition more interesting.

I hope that each participant enjoyed the competition. I hope that the
results will be analyzed in details, and I hope it will encourage more
research in termination techniques.

Among the satisfactory points I see :

1) The system of "secret" problems given just before the competition
   seems good to me. It is clear that this collection reveals the
   respective strengths and weaknesses of the tools.

2) I feel that tools made good progress with respect to last year. In
   particular AProVE solves an impressive pourcentage of the
   TPDB. Good progress have been made also for the 'NO' answers. 
   But still, there is a quite large number of problems not solved by
   any tool.

3) A particular satisfactory point: the secret problem cime1.trs was
   an open problem to me and its original author Eduardo Bonelli. But
   TEPARLA solved it ! I'm sure that Eduardo will be very interested
   in that proof. 

But I see also some unsatisfactory points :

1) First of all, the LP category was not run this year, due to lack of
   participants. 

2) In the rewriting categories, there are existing tools which did not
   participate: Cariboo, Muterm, Termptation.

3) I find the 1 minute timeout a bit to low. I would like to suggest,
   for the next competition, a larger one, say 5 minutes. 

4) Only the 'standard rewriting' sub-categories have a real interest:
   the others have too few participants. And nobody proposed "secret"
   problems for those... (except one, cime6.trs, which was not
   difficult anyway)

I think we have to think about the next competition as soon as
possible :

1) Will the participants of this year still want to participate ?

2) When should it take place ? If it was much before WST (or RTA), we
   could have an opportunity the insert a report on the competition in
   WST (RTA ?) proceedings.

3) How can we attract participants for LP categories ?

4) Should we add new categories ? higher-order rewriting ?

5) How could we enhance the tpdb ? Are there volunteers to work on it ?

Please comment !

- Claude


-- 
| Claude Marché           | mailto:Claude.Marche at lri.fr |
| LRI - Bât. 490          | http://www.lri.fr/~marche/  |
| Université de Paris-Sud | phoneto: +33 1 69 15 64 85  |
| F-91405 ORSAY Cedex     | faxto: +33 1 69 15 65 86    |