|Prof Barron: I would like to open the panel for discussion. As I said earlier this is a very unique event. Please do not hesitate to make use of it to ask any question you have about publishing.
Thank you for a wonderful lecture for us. Could you tell us what some of the factors are that would make you decide not to send the paper for peer review?
Dr Abraham: For us there are really, I suppose, 3 reasons and maybe more. One is, as I mentioned the issue with case reports, so articles that fall outside the focus of the journal if there is very little chance that the article will be accepted, then I think the fairest thing for the authors, is if we send it back very, very quickly. I try and provide a reason for that, too, such as that it is not appropriate for the journal. Sometimes we get an article, for example, that really is on cardiology, well that is not an appropriate article for a respiratory journal, at least for the Blue Journal. So I will send it back and say that. Sometimes there are quality issues as well. So if the paper really has major problems, I know the reviewers will note that, and the paper will not get a sufficient priority to allow revisions. So again I would try to do the authors a service, and send it back fast enough so they do not waste a month or longer. I always try and provide a reason. I think it is very important for us to have a dialogue with the authors, and not to just sort of imperiously send it back. There are certain journals, for example, where you pay a submission fee, you send in the article, 3 days later you get a rejection, your money is gone, you do not know why the paper was rejected, and you just have to move on. I think that is not fair. So the main reasons for us are quality and focus more than anything else.
Dr Rabe: May I just quickly add something to that. If we make a priori rejections at the editor’s level, the luxury that we have 2 people, so what we try to do is to get this not based on one single opinion, because sometimes you are uncertain. There is always a grey area. Something that is quite clear is a trial that has an uncontrolled nature, uncontrolled trials for example. This is something that we think that is unacceptable. I mean that is something that is a quality that we do not want. The second one is clinical material. I guess, at all of the journals we get a lot of case presentations and case reports and we have different ideas on that. For the ERJ I personally review all case reports. I do them all myself. What I do is I have a list of cases that we have had over the last 15 years as case presentations or cases for diagnoses, and I just looked on that I accept or I reject and I have not got a complaint letter so far, I tell people give me a new disease, give me a new gene, give me a new side effect, then you are in. But I have about 300 cases submitted but only 12 slots per year. Usually people accept the decision if they get the reason and I think that is something we are all saying. If you write a letter and take the time to tell an author, why it just did not make it, people understand this if the response comes in as quickly as possible. I think that is the second part. I think a rather clean, quick rejection is less of a worry than 3 months wiggling about with bad reviews and finally you do not make it. I mean that is something that people do not like, and I accept that.
Dr Irwin: I would add 2 other things to what has already been said about us being able to make a rapid decision without sending the manuscript out. That would be when IRB approval is clearly required but was not obtained, and a phase 3 trial that was not registered in an independent registry. Those would just automatically be rejected . We would get back in touch with the authors, usually our staff would first un-submit them and say did you forget to mention that you had IRB approval, or did you forget to enter the registry number. If we do not hear back, and they just submit again then we will reject without sending it out.
Dr Thompson: I would like to endorse all of those comments. Our editorial office screens every manuscript to make sure they have complied with instructions to authors. It might sound a very basic comment, but you would be surprised how many authors submit articles missing their ethics approval and so on. They are just automatically screened for all of those criteria and will be sent back automatically to be rectified. If our editorial staff cannot follow the English, so just a basic English screening test at that level, it will go back as well. So we are not talking about sophisticated English, we are just talking basic English that our staff can follow. Then as Klaus said, for all the clinical studies that come to me first, particularly the case reports, we reject about half of the case reports up front. In our instructions to authors it clearly says that case reports will be assessed on their originality for new mechanisms, new treatments, or if it is a timely review of an unusual but important condition. I guess that is a bit vague, the other 2 are very clear. So all of those are reasons why we might reject right at the beginning. Initially not complying with instructions to authors does not mean that they cannot be resubmitted of course.
Prof Barron: Thank you. Any other questions?
Dr Rabe: I could probably add a little comment that I think we have different ways of how we deal with papers where the research and/or the author was supported by the tobacco industry. Respiratory journals have failed in the past to uniformly, I guess, formulate a policy on that. We have always been thinking that should not happen, but we will within a certain period of time strongly discourage research that has been funded by the tobacco industry. The Blue Journal has done this years ago. Not all of the journals have a policy on authors being funded by the tobacco industry to submit papers to the journal. As of next year we will not publish papers that are coming in from that sort of funding. I think it is worth discussing amongst our colleagues.
So you would not send it to the reviewer in that case?
Dr Abraham: Yes, and for us it is the American Thoracic Society’s position to absolutely not publish an article that has been supported by the tobacco industry. So we will not consider it all. It will be turned down by the office.
Dr Irwin: Same for Chest. The only way it conceivably might happen is if the authors never disclose that information, and that has happened. But in the spirit of full disclosure that kind of information, as described in our instructions to authors, is supposed to be disclosed.
Prof Barron: Can we go to the next question, please?
Thank you very much. If we have a letter from the Editor in Chief saying that the paper has been rejected, could it be there are any “code words” in the letter that indicate that the paper is completely rejected?
Prof Barron: You mean any words that indicate that a resubmission would not be welcome.
Dr Abraham: I think I understand the question. We will sometimes send a letter and say that there are multiple problems with the paper, and though we find the idea interesting there are too many problems to ask for a revision. However, if you do additional crucial experiments or provide additional crucial data, we will reconsider the manuscript. It is not often, but once in a while we will open the door to that. Then an other thing that sometimes happens, although it is very rare, is we do have a dialogue, and if there are really major problems that were shown by the reviews we considered it a reject. But a lot of times letters come in about a paper that has been rejected, and they say they disagree with the reviewers, but that is really not enough. I mean there are differences of opinion, but you have to really show that the reviewer made a major mistake for us to reconsider. But we are open to that.
Prof Barron: If you are trying to indicate that the problems are such you would not welcome a resubmission. Would you put in something like, the paper does not have sufficient priority? Would that convey a meaning that a resubmission is not welcome?
Dr Abraham: We are even more blunt. We actually say that we will not consider a resubmission of the manuscript. It is very, very clear. Sometimes the authors do not understand, even though we say that, they will write back in a few months and say they fixed everything the reviewers asked for, all their criticisms, and now we are resubmitting. The answer is, you cannot resubmit in that case. I will not even send it out for review. That is another example of a manuscript that does not go out for review again.
Prof Barron: This young lady has been waiting a long time.
Do you believe in the saying publish or perish? And if so, how can a young researcher advance ones academic career, say for applying for grants, through means other than purely the number of papers you have published?
Dr Irwin: It would depend upon your situation. So you would have to define what it would take in your mind to be successful. That depends on whether you are a clinician scientist, whether you are a clinician researcher, whether you are a clinician who enjoys teaching. In order to really specifically answer your question, it would be very important for me to know what you would define as being successful. If you wanted to educate the rest of the community you could do a scholarly review of something. You could write textbook chapters. I think it is very important for us to understand the context in which you would define success.
Dr Thompson: I am trying to identify where you are coming from. First of all I think presenting at major meetings is number one. That way you can get feedback on your work. People can become familiar with your work. That then will help facilitate you to have a successful publication and if we believe the analysis that Klaus presented in his presentation this today, then this is almost guaranteed. So that is one point. I think the other is to network with other people, and if you are at a stage where publishing is difficult, one way forward is to join forces with other people who have had success in publishing, and start to interact with other groups, other people. Generally most people are enthusiastic if someone wants to interact and work with them. So those are 2 ways that I would suggest. If you are asking whether publish or perish is just a numbers game or is it a quality game, then my answer is that I think it a bit of both. I think you need numbers to show that you are active. If you do not publish for 3 years and then get your Nature paper, you may have perished in the 3 years beforehand because no one really knows what you are doing. Equally though, if you do not publish quality papers people ultimately will not respect what you are trying achieve.
Dr Rabe: May I add something? I guess there is no uniform answer to that, because it depends on the system in the university that you work. I think all of us who work in a university setting recognize the problem in evaluating the value of a CV, for example. Now I can tell you that in the Netherlands, people have been recognizing this as an increasing problem. What we have been doing is we are trying to start up career tracks for people who do not publish, but teach. So you can have a very good career in an academic institution, which is probably different than 10 years ago, recognizing that your teaching skills and your skills with students and how many courses you have done could weigh a lot against impact factor of publications. So I would hope that your faculty and your university would allow this diversity, because in clinical science and in basic science in the academic career track, teaching is something that is undervalued and there is not even a real career track for that and I hope this is going to change in other countries as well.
Dr Abraham: So let me give a not subtle and more brutal answer to your question. When we look at promotions at the universities in the United States and when we look at grant applications, promotions more than anything, particularly for physician scientists are looked at in terms of having been able to achieve external funding, more than anything else. Publications are a bit of a surrogate, because you are not going to achieve external funding without publications. If you come up for promotion and in particular for us the major promotion point for a 10-year track is between assistant and associate professor, it is really based on your continuity of funding. So publication becomes very important for that in high impact journals that will impress the review groups at the National Institutes of Health. The same kind of thing goes on in Europe, and many places as well, in many countries. I guess the answer here is not so subtle. You have to publish in high impact factor journals that are looked at appropriately by review panels, and ultimately that will leverage yourself into getting a grant, and then getting the grant renewed. So I think it is all tied to together in the process.
Dr Irwin: Again getting back to what I said. If you had said to me that you wanted to be a clinician teacher, again in our institution and many others in the United States, you actually are valued and there is a separate track for doing that. If you said to me you wanted to become a world famous clinician and consultant in certain areas, I think you could get there with writing books, giving talks, doing fewer publications as Dr. Abraham had talked about, and actually having a mentor that helps you along in presenting at national meetings. So understanding what your ultimate goal is, is very important.
Prof Barron: Could we go on to the next questioner because he has been waiting a long time.
I have one quick question about selecting the reviewers. Sometimes reviewers are so nice and very educational, and then following that instruction I got published in one paper in the Blue Journal. At that time the reviewer, one of the reviewers was Dr. Rabe and then I was very impressed about his instruction. But sometimes the reviewer’s comment is very, very short and very low quality. My question is do you have any criteria for selecting the reviewers in general?
Dr Abraham: There are several criteria. We do look at suggested reviewers by the authors and also the reviewers that you say should not look at the paper. We look at those lists and take that into consideration. It does not mean that we will use any of the reviewers you suggest, but we will look at that. We also look reviewers’ performance. So reviewers who are always late or say they will review a paper and then do not review it, we do not go back to. We also look at the quality of the reviews in a consistent way. We have 14, 15 associate editors now and so each of them knows their field and knows who provides good reviews. The other issue that I wanted mention too, is there is a section on our review form for confidential comments. Sometimes the review to the authors can come back and be relatively nice and yet in the confidential comments the reviewer will tell us there is low priority for the article and we will end up rejecting it and then the author will say why did you reject it, the comments were nice? In fact it was because we saw those other comments that were not shared with the author. So those kind of issues also come into play. But in general it is consistency of approach for the reviewer’s expertise in the area, timeliness, and the quality of the reviews.
Who decides if that reviewer is an expert at this?
Dr Rabe: The associate editors will decide. If you are the Chief or the associate editor in every journal you make the selection because that is what you are trusted to be a master in your area. So the selection of the reviewer is something that has been given as a task to the associate editors or those editors handling the manuscript. They are termed differently at different journals. But I think the point you are making is crucial. That is why we just try to get back to that. I mean if you look at journals and I think we have all worked for some one of them, the fact is that every associate editor asks the people around him to review his papers. That is the way it works. You get a lot papers that you need reviewers for, 2, 3, 4 reviewers. I mean there is a finite amount of individuals that you can ask regularly, these need to be people that you know. In selecting those you have to make a very balanced choice. We all know that there are better and worse reviewers. So what you can expect and what you should be expecting and that is the argument of Phil Thompson, I think if you have a very bad review in terms of short, no explanation, if it is very positive or very negative for an editor it has less value. Someone gives no comment and says that is the best thing since sliced bread, something that you cannot do anything with. That is the same thing for the negative decision. So you can expect an editor, - and that is where we have discussions on reviews are very different in quality and we know that. The final letter and decision to you should be based on evaluating the quality, not only the length of it.
Dr Abraham: I just wanted to say one other thing about how manuscripts are handled, at least at the Blue Journal, which I think is somewhat different from the ERJ, for example. In the past the associate editor would make a decision on the manuscripts for acceptance. Those manuscripts that would go out for review. We have now added a second tier to this. So what happens now at the Blue Journal is those manuscripts that each associate editor or deputy editor or myself think should be considered for revisions to be sent back to authors, we actually discuss at a weekly meeting that the deputy editors and myself have, and so we will look at the reviews but also a lot of the articles fall within the areas of expertise of our deputy editors, such as asthma, COPD, critical care. So that provides another level of review and another issue where we are able to consider the issues of priority. So the decision to accept or reject an article, to ask for a revision or not, is not based only on what the reviewers say, but also about the priority, the distribution of articles for the journal, and how we consider it given all the articles that we are looking at over a period of time.
Prof Barron: Okay Richard you have been waiting patiently.
Dr Irwin: I would agree with what everybody else has said. I would just add one other thing. I am sure everybody else does it, I know we do, we actually grade the reviewers. We grade them on timeliness as well as quality of their reviews, and over time scores actually appear next to the names of the reviewers, so that we know whether they have done a good job in the past. That is important, because I am sure the other journals have similarly sized data bases of reviewers, we have a database of 8,000 reviewers. Over the years and certainly since I have been doing this, I have been grading them and the associate editors have been grading them in a very consistent way. You can tell who is a good reviewer or not, based upon the criteria that have already been described.
Dr Thompson: One quick comment. Many journals, including our own, appoint the associate editors or whatever designation they have in their journal, based on their specialty interests as well. Certainly with our journal when the manuscript comes in we will try to send it to an associate editor who already has a specialty interest in that area, and then we are relying, as Klaus has said, on that person knowing people who specialize in the area for refereeing. So by channeling your manuscript right from the start towards an associate editor who has some background in that area, we are hopeful that we get high quality referees in that particular area, and then all the other things that have been said we would try to practice as well. The electronic systems, like Manuscript Central, have systems for storing referees and grading them, which is then available to your editorial staff as well.
Prof Barron: Thank you.
Thank you for giving me the opportunity to listen to your editorial policy. We all are pulmonologists. We have different professional fields such as the scientific assembly. My question is, do you have any idea or a good way to keep balance of the citation ratio between the different professional fields?
Dr Abraham: We actually look at that for various areas. We look at pediatrics, we look at asthma, we look at COPD papers in the various areas, how often are they cited. Now we do not make a decision, I think what Dr. Rabe said was very important. We still make all decisions based on the scientific quality of papers for publication. Although we take into account these other issues. So for example, if it is a wonderful paper on ciliary dysfunction for example, we are going to accept it even though there are very few people working in that area. But if it really moves the field forward we still take the paper. So it is always a question of the best science. In terms of priorities, what field it comes from is a small factor, but that really is reflected by the number of papers that come in in the area. The major criterion for us always is the quality of the paper.
Dr Irwin: I would say that we do not pay any attention at all beyond making certain that the manuscript is in the areas that we are focusing on, we just look at the quality of the paper. We are less interested in enhancing our impact factor than we are serving the people who are sending good papers through us and the readership. I think that is what Ed has also said.
Prof Barron: This may be the last question.
I have a question concerning publication of a case report with research results because you are talking today about positive and negative factors affecting the paper. I know that case reports with original research data appear very rarely, but can you mention a little bit about the positive and negative factors about publishing those. Thank you.
Dr Rabe: I think that is a very interesting question. Thank you for asking this. There is a very, very good case to be made for observational papers. An observational paper couldn’t be an n of one. So a paper that in fact is not so much a case report but actually highlights a new mechanism, even if it is very rare, should be published and we should be smart enough to realize that, because there is only one way to detect rare mechanisms. Those sorts of things that do not happen very frequently. If several people put together very rare events and if they never get published, we will never recognize it. A case that would highlight a mechanism would advance, as Ed Abraham was saying, the field in that, a case can do that. We have had examples where someone by a single case detected a gene defect. I mean, you know, they did a lot of stuff in a single case and identified the locus where things got wrong. This is the ideal case for a case report that we would all like to see. I think none of us that realized it would reject this up front. Unfortunately, this is an extreme minority of cases that you get. What you usually get is something which is relatively rare, reasonably well presented, and is yet another one of a story that has been told before. In our case we only have 12 slots per year, so even though everything is very nicely presented, if it is known to the field, it does not advance it, that would have a negative decision. And we have society journals that can put these in an educational publication like, Breathe is an example of that. Other societies have these as well. But for original publications that would be the limiting factor.
Dr Abraham: I think the other kind of case report is a very interesting negative case report, particularly genetic, where you present a patient who has a classic disease, that is always supposed to be associated with a particular genetic abnormality, and this case does not have it, that is very useful too. That is an n of one that can be quite useful. But most of the case reports are just as we heard, that they are the third, or the twelfth presentation of an unusual disease, that is not going to go forward.
Prof Barron: Time is up so I would like to thank all of the panelists and the societies who very generously supported this event. I have to apologize to Dr. Paul Reynolds who is here to give a presentation about next year’s APSR meeting in the Gold Coast, but we will make another slot later on in the Congress. Thank you all very much for a very interesting event. Thank you.