<p>I am very interested in pursuing graduate school mainly because I prefer to be involved in research setting rather than industry. But one thing that bugs me the most is published papers and just the thought that people get away with making up data. I don’t know how often this happens in the US, but it seems like it happens a lot in other countries. </p>
<p>Have you guys ever encountered peers who got away with frauding their research and made up data, etc…?
It seems like it’s very unlikely that these people get caught since you don’t really submit much to a journal except stating what results you got. I can see those without moral values doing this. I expect this is definitely in the very small minority, but it just bugs me that I may have to compete with these kind of people. </p>
<p>What do you guys think about this? How often do you guys think this happens in engineering?</p>
<p>I’ve never run across it personally. I really don’t think it happens much at all, and when it does it is likely to be fairly quickly debunked when others can’t reproduce the results.</p>
<p>Yeah, I expect that it doesn’t happen much, but today I overheard some graduate students talking about the subject and saying that it does happen. </p>
<p>How do people tell the difference between fake data and results that are wrong, but may be accidentally wrong (maybe due to an incorrect experimental setup or something, etc…)?</p>
<p>I feel like not a lot of researchers go out of their way to reproduce something. I’d say the “fake” results have to be something completely ridiculous to debunk and classified as fake, but if it’s completely ridiculous, then it wouldn’t have gotten published in the first place.</p>
<p>I have run across a few publications from overseas with completely unrepeatable results, but it is always hard to be sure whether or not that is a science problem or a translation problem. Never seen it firsthand from a US publication.</p>
<p>I’d guess it’s more common than most think, since the most research basically goes unread beyond a few editors. However, I’d like to think that the overwhelming majority of papers are in fact based on accurately collected data.</p>
<p>It’s just one of those problems out there that bugs me when I think about it, but there’s nothing people can really do about it unless it’s so obvious. I think most people that make up data are smart enough to make it believable, but just too lazy to go out and do the actual thing.</p>
<p>Maybe I’m too naive, but I doubt it happens very often, at least not in the narrow niche of electrical engineering that I work in, since it is not so difficult to reproduce the research of others. </p>
<p>A much more frequent problem is choosing poor experimental conditions or the exact problem definition, cherry picking so that the proposed solution performs well, even though it wouldn’t work well under more realistic real-world conditions. I think that is what really sets engineering apart from math and the pure sciences, in that engineers are (or should be) more focused on solving real-world problems rather than just theoretical ones.</p>
<p>But how many experiments are reproduced? The vast majority of papers, at least for the geosciences get at most 5 citations, and almost no scientists reproduce the experiments, particularly those which are basically insignificant.</p>
<p>Research results must be able to be duplicated by other scientists. If it can’t be, then a career will be quickly over since future research will be considered unreliable.</p>
<p>Of course it has to be; however, the simple reality is that the vast majority of “insignificant” research is not reproduced, which may increase incentives for scientists (and presumably engineers) to fabricate the data since they can be reasonably sure they won’t get caught.</p>
<p>But if it wasn’t important enough for there to reproduce it, then why are you worrying about it being bad research? If it only gets 5 citations and no one thinks it portent enough, then it really doesn’t matter if it turns out to not be reproducible because it sounds like people are already ignoring it, not to mention they aren’t basing larger, more important research on it.</p>
<p>The bottom line is that this is that fraudulent research is really not a problem in the major journals in a given field. Stick to the journals that you know are reputable and you will be fine, as the vetting process for articles is quite extensive and the known journals don’t need filler articles that are questionable.</p>
<p>Hypothetically speaking, if I try to reproduce someone else’s results and I fail to do so.
How does that mean their results are wrong or not repeatable? </p>
<p>There’s too many factors/excuses that intentional fraudulent data can hide behind.</p>
<p>I really doubt we can easily describe the review process for a major journal here. The point is, it’s difficult to establish credibility for real research, especially if you’re a from a relatively unknown group. If there’s any hint of falsified data, it could easily ruin your reputation with most major journals. Furthermore if you’re caught your research will never be valued again, so it’s not worth the risk. Finally most academics got into the field because they actually want to make a difference, not just a career.</p>
<p>There was a professor at my university who I read about bring outed for publishing fake results. I’m not certain but I believe he was caught for having his studies simply turn out too well after having little success in the past.</p>
<p>It’s actually reasonably easy to describe the journal review process. You submit a paper and it goes to an editor. That editor passes it on to several (usually three) expert reviewers from the same field. Those reviewers do a handful of things: they check for scientific validity, they check for the overall impact of the research, and they check for clarity. Those reviewers come back and either reject, accept, or offer revisions for the article (usually the latter). The authors get the comments and address them in words or through revisions to the manuscript and then send it back to the editor who passes it back to the committee. If they approve it after revisions, it gets sent back to the editor who decides whether or not to publish it (and I’ve never heard of an editor not publishing something that was approved by the reviewers).</p>
<p>That’s already a pretty rigorous review process that a lot of legitimate research fails to make the cut. It is even harder for fraudulent research to make it. It does happen but not often in journals that have any sort of established reputation. That’s part of the reason such weight is placed on the journal’s impact factor.</p>
<p>Straight out fraud is probably unlikely. Poor error analysis, wishful thinking in interpretation, drawing conclusions from data which is very likely coincidental (assisted by cherry picking), etc seems to be everywhere. A lot of experiments are of the style where you tweak it a bunch of times until you get it to finally work. How do you know it worked, though? I’ve had times where I thought I had absolutely beautiful data, and then realized a few weeks later it was due to an experimental error, which, once fixed, made my results not match at all what we were expecting. How would a reviewer catch a mistake like that if I didn’t fess up to it?</p>
<p>Reviewers are there to make sure you have enough impact for a journal and don’t overinterpret your results. Also to check for any sort of fundamental flaws in your experiment.</p>
<p>Cherry picking is also a somewhat gray term in itself. For example, say I run a Vickers hardness test on a sample. My results are 685, 656, 671, 524, 690, 684, 673, and 690. Is it cherry picking to throw out my 524 result and assume it was experimental error (poor contact, slightly tilted surface, some junk on the surface, porosity, etc)? Or could it have been there’s actually a really small fraction of a soft phase? How far do I need to go to prove there isn’t a soft phase?</p>
<p>In my experience though, for the really respected journals, those reviewers are more likely to look at an assumption that you make that might be a bit shaky and call you on it if you can’t produce better supporting evidence. They aren’t going to full rerun your experiment or anything but they generally try to point out shaky conclusions.</p>
<p>Usually the less prestigious the journal, the less picky they are on that sort of thing though.</p>
Hit the nail right on the head. I know that in many very repetitive and not-so-successful experiments, I’ve been tempted to stretch the truth in order to avoid a whole lot of extra work to fix a few relatively small but very significant faults in my conclusions. Thankfully I usually don’t (and I get called on it almost always when I do), but the temptation is always there and a few of those can always slip past the radar.</p>
<p>The review system is there to catch people who slip up like this; that’s one major reason that circumventing it and talking to the media is a taboo in most research.</p>