This note was originally posted on Facebook on January 25, 2019. There have been some minor changes. I got some great feedback from Pauline J. on that post, in case readers are interested in pursuing what other people think. These notes are based on my experiences with journals, editors and reviewers over the last ten years or so. I want to emphasize here that I greatly appreciate the time that editors and reviewers put into their jobs.
The Power of Editors
I believe that an editor of a linguistics journal has the ability to significantly increase or decrease the chances of a paper getting published independently of the contents of the paper. Here are some of the relevant powers that editors have which lead me to think this way. I do not mention any particular journal or editor names, since it is not really relevant to the general points I am making.
(a) The editor decides whether or not to review the paper. The editor can say that the paper does not fit the parameters of the journal. Of course, there may be many completely unobjectionable uses of this power. But some particular cases of the use of this power have seemed subjective to me. And in fact, from experience, I know that editors can decline to review for other reasons as well. They can tell you that you are barred from submitting for some specified period of time (e.g., two years).
Of course, if they decide not to review the paper, you can ask for the reasons why, and sometimes they will allow you to make changes to the paper to make it more consistent with how they see their journal.
(b) The editor decides who to send the paper to for review. If the editor sends a paper to a person whose position is being argued against (as evident from the in-text citations, and the references), then that works against the author (since people's standards go way up when it comes to arguments against their own positions). In an ideal world, this would not be so. People would always be helpful and not mind criticism of their papers, and might even like their name in the spotlight. But it is clear we are not living in that world.
Some journals allow you to suggest reviewers that should not be sent the paper. This is also a mainstream
practice on NSF grant applications. I think it is a good idea, and should be adopted by all journals.
(c) The editor decides what counts as a rejection. How strong do the reviews have to be and how many have to be strong for a paper to get accepted (or to get "revise and resubmit")? In many cases, reviewers do not even say "accept" or "reject" or "revise and resubmit". They just write down their thoughts, which have no strong negative or positive orientation. In this case (which happens more frequently than you would think), the editor is left to read between the lines of the review to formulate a decision.
The editor also has the ability to ignore a positive review, if they feel a negative one outweighs it. They will say things like "While review A is positive, review B brings up serious points that we concur with...." (or something to that effect).
(d) The editor decides how many reviews will there be. At least one major journal now seems to only require two reviews. Other journals are more rigorous and require three. The fewer reviews, the more power the editor has. See (e) below.
(e) The editor often intervenes with the crucial deciding vote. That is, in many cases the editors submit an additional review that gets counted in the decision. But the editor's review is not blind, so that could lead to potential problems (e.g., various kinds of implicit bias).
In the case of two reviews that are split, you can ask the editor for adjudication. This is something that I only recently learned about. Until very recently, I have always accepted editorial decisions as the final unquestionable last word. That is, the editor will find a third reviewer, and have them decide between the two reviews. However, even in this case, the editor has a huge amount of power in choosing the third reviewer (for example,
is the third reviewer a close colleague of the person who wrote the negative review).
(f) The editor decides when the process stops. How many "revise and resbumits"
can there be before a paper is finally clearly rejected? Some editors are more willing to work with a paper that has an interesting idea in order to see it through to the light of day (e.g., two or three cycles of "revise and resubmit"). Other editors will cut you off after one "revise and resubmit". That is, after a "revise and resbumit" judgment, if you resubmit and get another "revise and resubmit" judgment, the editor cuts it off.
(g) The editor decides what subjective criteria (over and above contribution to scientific knowledge) are important and they prioritize these (probably implicitly and unconsciously). These criteria play a role in their editorial decisions. For example, an important consideration with certain journals is that papers must contextualize their results by giving lengthy discussions of existing treatments. A paper can actually be declined because of lack of contextualization.
Reviewers and editors have also been known to put forth straw man proposals (that nobody has ever published or even entertained) and to demand that the author address them. And what is worse, editors can decline a paper because the author has not addressed such straw man theories.
As another example, every paper in linguistics has the potential of branching off in a large number of ways, and it is easy for a reviewer (or editor) to say that the paper fails to explore one of those branches.
The problem with (g) is that no matter how good a paper is, no matter how striking the result, it is always possible to find something to criticize in it in order to reject it. I believe that such factors should never lead to the rejection of a paper, although authors could be asked to address them in the final version.
I am not criticizing any particular editors or journals. I am simply pointing out that editors have a huge amount of power in deciding whether or not a paper gets published. My recommendations based on this discussion are the following:
(a) Editors should seek highly qualified, but disinterested reviewers. Do not send the review to the person whose views are being criticized. Allow authors to give a small list of people who should not review the paper.
(b) Editors should try to get three reviews. Two reviews can easily split. The flip side of this is that people should help editors get reviewers in a speedy manner. For example, if you are unable to review a paper, suggest two or three other people who would be able to do it (and provide their e-mail addresses), and do this in a prompt manner (within 24 hours of the request).
(c) Editors should refrain from reviewing as much as possible. For an editor to review a paper violates the standards of double blind reviews in the field, and could lead to all kinds of cases of implicit bias (this paper should be accepted, because it is written by my close colleague's top student).
(d) Editors should try to be open minded about content. Do not try to remake the journal in your research image. There are different points of view out there, some of which may contain a seed of truth (or illuminate part of the elephant).
(e) Your job is not just to efficiently cut out the chaff but it is also to help authors develop papers, in order that good ideas (that lead to scientific progress) can see the light of day.
The Power of Editors
I believe that an editor of a linguistics journal has the ability to significantly increase or decrease the chances of a paper getting published independently of the contents of the paper. Here are some of the relevant powers that editors have which lead me to think this way. I do not mention any particular journal or editor names, since it is not really relevant to the general points I am making.
(a) The editor decides whether or not to review the paper. The editor can say that the paper does not fit the parameters of the journal. Of course, there may be many completely unobjectionable uses of this power. But some particular cases of the use of this power have seemed subjective to me. And in fact, from experience, I know that editors can decline to review for other reasons as well. They can tell you that you are barred from submitting for some specified period of time (e.g., two years).
Of course, if they decide not to review the paper, you can ask for the reasons why, and sometimes they will allow you to make changes to the paper to make it more consistent with how they see their journal.
(b) The editor decides who to send the paper to for review. If the editor sends a paper to a person whose position is being argued against (as evident from the in-text citations, and the references), then that works against the author (since people's standards go way up when it comes to arguments against their own positions). In an ideal world, this would not be so. People would always be helpful and not mind criticism of their papers, and might even like their name in the spotlight. But it is clear we are not living in that world.
Some journals allow you to suggest reviewers that should not be sent the paper. This is also a mainstream
practice on NSF grant applications. I think it is a good idea, and should be adopted by all journals.
(c) The editor decides what counts as a rejection. How strong do the reviews have to be and how many have to be strong for a paper to get accepted (or to get "revise and resubmit")? In many cases, reviewers do not even say "accept" or "reject" or "revise and resubmit". They just write down their thoughts, which have no strong negative or positive orientation. In this case (which happens more frequently than you would think), the editor is left to read between the lines of the review to formulate a decision.
The editor also has the ability to ignore a positive review, if they feel a negative one outweighs it. They will say things like "While review A is positive, review B brings up serious points that we concur with...." (or something to that effect).
(d) The editor decides how many reviews will there be. At least one major journal now seems to only require two reviews. Other journals are more rigorous and require three. The fewer reviews, the more power the editor has. See (e) below.
(e) The editor often intervenes with the crucial deciding vote. That is, in many cases the editors submit an additional review that gets counted in the decision. But the editor's review is not blind, so that could lead to potential problems (e.g., various kinds of implicit bias).
In the case of two reviews that are split, you can ask the editor for adjudication. This is something that I only recently learned about. Until very recently, I have always accepted editorial decisions as the final unquestionable last word. That is, the editor will find a third reviewer, and have them decide between the two reviews. However, even in this case, the editor has a huge amount of power in choosing the third reviewer (for example,
is the third reviewer a close colleague of the person who wrote the negative review).
(f) The editor decides when the process stops. How many "revise and resbumits"
can there be before a paper is finally clearly rejected? Some editors are more willing to work with a paper that has an interesting idea in order to see it through to the light of day (e.g., two or three cycles of "revise and resubmit"). Other editors will cut you off after one "revise and resubmit". That is, after a "revise and resbumit" judgment, if you resubmit and get another "revise and resubmit" judgment, the editor cuts it off.
(g) The editor decides what subjective criteria (over and above contribution to scientific knowledge) are important and they prioritize these (probably implicitly and unconsciously). These criteria play a role in their editorial decisions. For example, an important consideration with certain journals is that papers must contextualize their results by giving lengthy discussions of existing treatments. A paper can actually be declined because of lack of contextualization.
Reviewers and editors have also been known to put forth straw man proposals (that nobody has ever published or even entertained) and to demand that the author address them. And what is worse, editors can decline a paper because the author has not addressed such straw man theories.
As another example, every paper in linguistics has the potential of branching off in a large number of ways, and it is easy for a reviewer (or editor) to say that the paper fails to explore one of those branches.
The problem with (g) is that no matter how good a paper is, no matter how striking the result, it is always possible to find something to criticize in it in order to reject it. I believe that such factors should never lead to the rejection of a paper, although authors could be asked to address them in the final version.
I am not criticizing any particular editors or journals. I am simply pointing out that editors have a huge amount of power in deciding whether or not a paper gets published. My recommendations based on this discussion are the following:
(a) Editors should seek highly qualified, but disinterested reviewers. Do not send the review to the person whose views are being criticized. Allow authors to give a small list of people who should not review the paper.
(b) Editors should try to get three reviews. Two reviews can easily split. The flip side of this is that people should help editors get reviewers in a speedy manner. For example, if you are unable to review a paper, suggest two or three other people who would be able to do it (and provide their e-mail addresses), and do this in a prompt manner (within 24 hours of the request).
(c) Editors should refrain from reviewing as much as possible. For an editor to review a paper violates the standards of double blind reviews in the field, and could lead to all kinds of cases of implicit bias (this paper should be accepted, because it is written by my close colleague's top student).
(d) Editors should try to be open minded about content. Do not try to remake the journal in your research image. There are different points of view out there, some of which may contain a seed of truth (or illuminate part of the elephant).
(e) Your job is not just to efficiently cut out the chaff but it is also to help authors develop papers, in order that good ideas (that lead to scientific progress) can see the light of day.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.