Dear fellow cryptologists,
I turn to the cryptologic public, because I experienced unfair and
incorrect review procedures at IACR sponsored workshops, and the IACR
board of directors seems to be unwilling or unable to react adequately
to my formal complaints.
I submitted two papers to CHES 2006, and these two papers got three
reviews each. There is firm evidence that none of the six reviewers had
bothered to read completely the paper they had to review . It is
obvious that it is impossible to judge the merits of a scientific paper
one has not read.
How can I know the reviewers did not read the papers? This is evident
from the reviews. Let me give an example. In the submission
"Correlation Power Analysis and a Fast Algorithm for Optimal Key
Search", I use an algorithm which uses a list C. In each step of the
algorithm, the minimum element of C has to be determined. One of the
reviewers blamed my algorithm for being very inefficient, because he
assumed that one has to search C sequentially each time. If he had read
the paper, he would have known that one implements C as an AVL-tree,
which allows to find the minimum element in O(log n) steps if there are
n elements in the list.
I put the reviews, together with my comments, on my website
http://irregular.npx.de/
. Check for yourself that it is evident that
none of the reviewers bothered to read the papers. I did not put my
papers on the website, because I want to publish them elsewhere. But I
will send preprints to anybody who asks.
Now it could theoretically be that just I had bad luck with the
reviewers my papers got.
But if 6 out of 6 reviewers do not read the papers, but just skim over
them, this is statistically very significant evidence that a high
percentage of the reviews was done in a very careless and superficial
way.
I find this behaviour of the reviewers bad enough, but I think that the
responsibles for the review procedure, the program chairs, are much
more to blame. The program chairs for CHES 2006 were Louis Goubin and
Mitsuru Matsui. They are responsible for the choice of the reviewers,
and they should have supervised their work. However, even when I
pointed out to them that the reviews had obviously been incorrect,
because none of the reviewers had read the papers completely, they just
answered that the decision of the program committee was unchangeable.
Clearly, their attitude, to base the selections of papers for a
conference on reviews, from which they know that they were incorrect,
is irresponsible and against scientific morale.
On July 13 2006 I sent a formal complaint about this to all directors
of IACR, with the request to investigate the review procedure for CHES
2006. Up to the present day (November 15 2006), I did not get any
response to these emails. I find this behaviour of the IACR board of
directors unacceptable.
Let me give another example where the review procedure for an
IACR-sponsored conference failed very badly.
At FSE 2005, there was a paper "Unbiased Random Sequences from
Quasigroup String Transformations" by Smile Markovski, Danilo
Gligoroski, and Ljupco Kocarev. The authors suggest a bijective, easily
invertible function for postprocessing biased random numbers. Clearly,
this does not make sense at all. I just wonder how bad the errors in a
paper must be in order to be noticed by the reviewers. The theorems of
the paper are based on faulty mathematics, but the incorrect proofs are
in the appendix, which the reviewers are not required to read. But I
would have hoped that it is obvious to any member of a program
committee for a crypto conference that applying a bijective mapping to
biased random data can not increase the entropy of the output, and
hence does not make any sense.
It is unavoidable in any field of science that errors happen, and that
incorrect results are published. It is an indispensable part of the
general scientific approach, that such errors are corrected as soon as
possible.
I submitted a paper to FSE 2006 which described the problems of the
quasigroup random number post processing paper from FSE 2005. My paper
also described a sma****ng attack on the quasigroup postprocessing
scheme. None of the reviewers doubted the correctness of my results,
but anyway my paper was rejected. In my opinion, it is a severe breach
of scientific morale to publish a very wrong paper in a prestigious
series of publications as Springer Lecture Notes in Computer Science,
but to suppress the publication of another paper, which corrects these
errors, at the same place. I can only consider the reply I got from the
program chair, Matt Robshaw, "Certainly I hope that you find an equally
good channel for your paper in the near future" as very cynical.
Clearly, FSE is the only reasonable place to publish papers about wrong
FSE papers.
It is often suggested to use cryptographic algorithms which were
published and for which for some years no attacks by the cryptographic
community turn up. This approach becomes pointless, if the IACR
proceeds like in the case of the quasigroup random number
postprocessing, where an attack was found, but the FSE program
committee chooses to have this attack not published.
I wrote an email about this matter to the president of the IACR, Andy
Clark. He promised to bring it up at the meeting of the IACR board of
directors at Eurocrypt 2006. About one month after Eurocrypt, I asked
him by email about the outcome. He answered, that he did not have the
minutes of the meeting at hand, but would answer until the end of the
week or early nexst week. This was 4 months ago, but I am still waiting
for an answer.
Clearly, the IACR board of directors is unwilling or unable to
establish regular and correct review procedures for IACR sponsored
workshops. I think that it does not make sense just to be angry
privately about these unfair and incorrect review procedures, but that
this should be discussed publicly. Therefore I encourage all that have
suffered from incorrect IACR reviews, and I am certain that there are a
lot of us, to speak up publicly!
Yours sincerely
Markus Dichtl
|
