A note of caution about technical papers (example included)

[DCockey]

Originally Posted by daiquiri

The Appendix A, the way it is written, just shouldn't be there. It has nothing to do with submarines and is physically completely incorrect if it was intended as a case study of a surface vessel. I find it really incredible that the reviewer of the paper (if there was any) didn't notice how inconsistent and disconnected from the reality is the example of that boat.

The report was a student senior research report (research done by an student during last year as an undergraduate) and I'm certain it was reviewed and graded by at least one faculty member. One would hope that someone from the sponsoring agency also read it before it was signed off as "approved". My guess is no one involved was familar with analyzing surface vessels which is an esoteric specialty even within fluid mechanics and CFD.

I agree with "It has nothing to do with submarines and is physically completely incorrect if it was intended as a case study of a surface vessel" but I've seen equally bad mistakes in other technical reports. It's not uncommon for specialists in analysis to misunderstand the physical version of what they are analysing, and also not uncommon for specialists in physical systems to fundamentally mis-use analysis tools and/or mis-understand analysis results.

[DCockey]

I'm not sure the problem of folks using bad information is any worse now than pre-internet. Certainly the internet has made considerably more bad information readily available. But it has also made checking and verifying questionable information much easier. Today an internet search will bring up a number of sources of information about almost any topic - some good, some not.

Pre-internet someone without access to a large, research library with a relevant collection was very limited in their ability to check what ever information they had. For most folks the only available information on technical subjects pre-internet was limited to what their local public library and/or bookstore happened to have on their shelves (which was selected by a librian or bookstore employee who was not knowledgable in the field), and also what was published in magazines and other periodicals.

One fundamental problem is the lack of either the inclination or ability to question if information is reasonable, and to verify it. This is a particular problem when the information is what is desired. In another thread I posted a link to a story of an eminent, distinguised naval architect who apparently got a stability calculation wrong.

Another fundamental problem is when someone, particularly a recognized figure, doesn't recognize what they don't know and then goes on to teach and/or publish about the topic.

[Alik]

David, this is really a problem and this is why we always submit papers for peer-review; it gives fresh opinion and helps at least to clear some omissions or errors.

On other side, we have seen such non-reviewed papers with mistakes used for teaching, and also included in books!

[MikeJohns]

Originally Posted by daiquiri

.......... Please let me repeat the question, because it is a fulcrum of this story: Can you guys tell what's wrong with the case shown in the Appendix A (pages 64 to 70)?

It's obviously a damaged stability err...performance study

[daiquiri]

Originally Posted by MikeJohns

It's obviously a damaged stability err...performance study

Yeah, definitely. A wave-less boat with a submerged transom stern deserves all the scientific study and admiration it can get.

[johneck]

What this really indicates that one should not just look at the pretty pictures in a report. The report was done to determine the suitability of a RANS code to determine submarine hull drag (viscous and pressure drag). There was no interest or intent of looking at wavemaking. The poaper is actually a pretty comprehensive reporting of the process used to perform the calculations, including grid resolution studies and a brief description of turbulence models. They also compared their CFD results with actual test results for the submarine bodies and found that their computations were not very accurate (no real surprise). The author went on to describe plans for future work to try and improve the method. All in all an impressive student paper.

[daiquiri]

Originally Posted by johneck

What this really indicates that one should not just look at the pretty pictures in a report. The report was done to determine the suitability of a RANS code to determine submarine hull drag (viscous and pressure drag). There was no interest or intent of looking at wavemaking. The poaper is actually a pretty comprehensive reporting of the process used to perform the calculations, including grid resolution studies and a brief description of turbulence models. They also compared their CFD results with actual test results for the submarine bodies and found that their computations were not very accurate (no real surprise). The author went on to describe plans for future work to try and improve the method. All in all an impressive student paper.

Definitively yes, but the Appendix A shouldn't be there. It has spoiled the otherwise good job and has brought the doubts about the reviewing process prior to the publication.

[DCockey]

Originally Posted by daiquiri

Definitively yes, but the Appendix A shouldn't be there. It has spoiled the otherwise good job and has brought the doubts about the reviewing process prior to the publication.

Assumptions about review processes may be the source of our different views about this report.

I look the cover and title pages and see a contract report to Defence R&D Canada. Contract reports are the contractor reporting on the results of their work, and are not peer reviewed papers or journal articles. (1) My experience is contract reports can range from exceptionally good and valuable to mediocre to fundamentally flawed.

It is also a report on an undergraduate student research project, and my experience with such reports is they can range from very good to poor. When viewed by itself Appendix A is unfortunate and can be easily misinterpreted, particularly given the first sentence that "A sample yacht hull was modeled in CFX-5 to demonstrate the potential of of CFD in ship design."

---------------------------------

(1) The cover and title pages include the statement "The scientific or technical validity of this Contract Report is entirely the responsibility of the contractor and the contents do not necessarily have the approval or endorsement of Defence R&D Canada".

[BeppeZena]

I'm agree with DCockey! this is not a peer reviewed paper. This is an undergraduate student thesis or project and therefore cannot be considered peer reviewed paper. Probably is in the Canadian database only for founding reason. If you are looking to some technical informations on which you want to base your work I think you should considered only peer reviewed papers! Thesis reports and some time even conference proceedings can be wrong or bad presented.

I'm sorry but I don't agree with ruysg. If a paper is in a journal it went through a serious review process and it can be trusted. Off course also Einstein work has been criticised and modified.

PS - be careful with the Canada R&D database...you can find anything there
cheers to all!

[DCockey]

Originally Posted by BeppeZena

...... If you are looking to some technical informations on which you want to base your work I think you should considered only peer reviewed papers! Thesis reports and some time even conference proceedings can be wrong or bad presented.

Peer reviewed papers may be too limiting in many cases. Are you proposing that the many reports from David Taylor Model Basin (and successor organiztions), NACA/NASA, NPL, Delft, etc which were not subsequently published in peer-reviewed journals should always be ignored?

I'm sorry but I don't agree with ruysg. If a paper is in a journal it went through a serious review process and it can be trusted. Off course also Einstein work has been criticised and modified.

......

I'm not entirely in agreement that papers in journals can always be trusted.

- The amount and depth of the review process varies tremendously between journals.

- The particular peer reviewers selected by the editor of a journal may or may not have expertise in areas peripheral to the main topic of the paper. For example if the report under discussion had been submitted to a journal about numerical analysis the peer reviewers might have beem experts in CFD methods and boundary layer modeling, but not knowledgable about modeling boat and not have recognized the problem daiquiri identified.

- Peer reviewers may let something which is questionable go through because they are friends of the author, or they don't want to make waves, or .....

[Petros]

I once worked in an engineering position where we were doing research on a new aircraft concept. The aero staff wadded through perhaps 50 years worth of research papers, finding errors was not uncommon, even sometimes amateur mistakes from institutions that should know better.

Uncritically relying on other peoples ideas, even experts, is very risky. That is why before you build something based on other peoples work, you still need to verify the claims yourself. This is normal in all costly developmental designs, particularly where lots of money is being invested.

[rwatson]

Originally Posted by johneck

..... All in all an impressive student paper.

hmmm - should I rely on the assessment of someone with an avatar of "non impediti ratione cogi"

[BeppeZena]

Originally Posted by DCockey

Peer reviewed papers may be too limiting in many cases. Are you proposing that the many reports from David Taylor Model Basin (and successor organiztions), NACA/NASA, NPL, Delft, etc which were not subsequently published in peer-reviewed journals should always be ignored?
.....

First of all, sorry for disappearing!
Yes, you are right, I probably push too much; but those examples should not be considered the rule, they are excellent works of good institutions, not the average level of things that you can find online.

My point is that if you look in a peer review journal (you can say a good one, but I think that if you search online usually the best journals are the first in the search page, # of viewers probably ?!?) you can find maybe useless publications but correct. The analysis can be some times pointless and conducted just to publish something, but is not wrongly conducted.

that is my personal point of view.
When I present something, some of the first think people ask is if the approach that I'm using has been published in a peer reviewed journal (people from both academia and industry). That is why I think that referencing to a peer reviewed publication boost the credibility of you work, but (there is always a but ) you should always exercise your judgement.