Peer Review

Discussion in 'Boat Design' started by Tad, Aug 2, 2010.

  1. Tad
    Joined: Mar 2002
    Posts: 2,303
    Likes: 185, Points: 73, Legacy Rep: 2281
    Location: Flattop Islands

    Tad Boat Designer

    Peer review of professional or scholarly work is an accepted practice of long standing in the fields of medicine, mathematics, engineering, architecture, and all areas of science.

    From Wikipedia....."Peer review is a generic term that is used to describe a process of self-regulation by a profession or a process of evaluation involving qualified individuals with the related field. Peer review methods are employed to maintain standards, improve performance, and provide credibility."

    And......"It is difficult for authors and researchers, whether individually or in a team, to spot every mistake or flaw in a complicated piece of work. This is not necessarily a reflection on those concerned, but because with a new and perhaps eclectic subject, an opportunity for improvement may be more obvious to someone with special expertise or who simply looks at it with a fresh eye. Therefore, showing work to others increases the probability that weaknesses will be identified and improved. For both grant-funding and publication in a scholarly journal, it is also normally a requirement that the subject is both novel and substantial."

    It seems to me that, of late, peer review in the field of yacht design has been lacking. We have come to accept this as the norm, we (yacht designers) are expected to tippy-toe around the criticism of others work, perhaps out of fear that others may return that criticism......Print magazines have abandoned the practice, due to what I see as economic pressure.....

    But peer review is vital to the improvement of everyone's work (IMO). Years ago Bill Garden wrote, " With hindsight every design can be improved." I agree.

    In years gone by Yachting World ran a multi-page piece every month, signed by The Walrus. Assumably this was the Editor, but the quasi-anonymity enabled him to write moderately honestly about designs and events without too much fear of being beaten up in the local pub. I believe that designer Douglas Phillips-Birt ran a similar column in the same publication under the pseudonym Argus. John G. Hanna wrote a fiery column for Rudder for many years called The Watch Below. Reading what JG thought of the latest S&S ocean racer was immensely educational, thought provoking for the professional, and also entertaining.

    Bob Perry continues to write a design review section (one of the last left in print) for Sailing Magazine. When he started these in the mid 1970's he didn't mind stepping on toes or giving someone a "nicely done!". Today they have mellowed as their author becomes more tolerant (I guess) or understanding of the complex issue's involved, but they are far less likely to contain any criticism. As Perry himself has written, "If I talk about dogs, guitars, or wine, I probably don't like the boat." This is not real peer review.....

    Who could be more qualified to criticize yacht designs than Yacht Designer's? If we don't do it then who will? Journalists? No.....their understanding of design issues is only as told to them by others. Salesmen "Brokers" are great critic's of the competition's design.....but it usually comes down to features lacking....not particularly technical.

    Yacht design is often rather personal, the creation of one personality with it's style and function very much a part of that personality, and any criticism becomes a personal insult. But for me it is nothing like that, criticism of any design infers no insult on the designer. When I write critically about the latest Shannon or Nordhavn or George Buehler or Brent Swain design it is in the hope that it will educate consumers, and provoke thought among peers....
  2. tom28571
    Joined: Dec 2001
    Posts: 2,472
    Likes: 114, Points: 63, Legacy Rep: 1728
    Location: Oriental, NC

    tom28571 Senior Member

    Tad, We must assume that you want to go somewhere with this post. What is the intent?

    Of course, you are spot on with magazine "reviews". Most appear to be extracted from mfg literature with little variation. Sometimes editors of marine publications do write critical notes but mostly they speak of trends and not specific designs or designers.
  3. Paul B

    Paul B Previous Member

    The BBC program "Top Gear" does, at times, say some pretty harsh things about the cars it reviews.

    The show was going to be re-made here in the USA a few years ago, but of course the sticking point was the inability to tell these sorts of truths when working in a media dependent on advertising dollars. Jay Leno was approached to be one of the hosts and he declined for that very reason.

    So the sailing publications and websites that are dependent on advertising dollars are not going to publish opinions that point out the deficiencies of their advertiser's products, or potetial advertiser's products.
  4. jehardiman
    Joined: Aug 2004
    Posts: 2,541
    Likes: 227, Points: 63, Legacy Rep: 2040
    Location: Port Orchard, Washington, USA

    jehardiman Senior Member

    I think, from a professional position, it all depends on what you mean by "peer review".

    In my office, all design calculations have a peer review. This is to ensure that no major engineering mistake is made because sailors lives depend on the component functioning properly. However, I often do not require changes to the design of componets unless it fails the design requirements or has some other major fabrication or cost flaw. I know of about 10 or more different ways to design a high-speed fairing. If another engineer (or sub-contractor to me) choses a method that is not the one I would chose in that situation, that is not under my purview to review (and to do so is to waste time and money), though I might discuss it with them. If it is a major concern, we put on our pointy hats, grab our "explaining sticks", and go into a private room to bash on each other until there is only one opinion.

    If a design/plan comes in from another office, we look at it, but we assume and expect that the engineering is correct regardless of the politics that lead to that particular design solution. However, on several occasions we have found errors in design, mostly those of ommision of requirements. In these cases we have reported out findings to the paying customer. He has the ultimate responsibility of accepting or rejecting the design.

    In the larger sense, it is the customer who decides the level of engineering review he is willing to accept and/or pay for. It is the responsibility of the Naval Architect/designer to ensure that the engineering work he does is correct to the best of his ability and that he protects the customers interest and/or life. There have been several occasions where I refused to take part design development because the customer had sold expectations that were inherently flawed. Making your position clear on such matters is all you can do when the customer can just to to another office that might not be so scrupulous.

    So in the final anlysis, any design (ship, yacht, pogo stick, etc) is a give and take between the customers requirements and pocketbook and the designers solutions and compromises. Without being in the others shoes, it is too easy to profess "I would have done it better" when in actuality you would only have done it differently.
    Last edited: Aug 2, 2010
  5. peter radclyffe
    Joined: Mar 2009
    Posts: 1,406
    Likes: 59, Points: 58, Legacy Rep: 680
    Location: europe

    peter radclyffe Senior Member

    did you hear the one about the skipper who thought he was a better navigator than columbus, neil armstrong & peter andre combined, for years he wrote books about his wonderful seamanship, books like hand relief & beer,etc, then when he sold his boat, far from being bristol fashion, it was totally rotten,
    but you cant publish these sorts of truths , there the wrong sort of truths
  6. ancient kayaker
    Joined: Aug 2006
    Posts: 3,497
    Likes: 146, Points: 0, Legacy Rep: 2291
    Location: Alliston, Ontario, Canada

    ancient kayaker aka Terry Haines

    While not a yacht designer, I wonder if that is truly the case. When I was in engineering I was usually part of a team so the peer review was at least implicit. Better product usually resulted when two or more people cognisant in the relevant area worked together.

    For me, the exceptions were innovations that needed proof of principle so it could be demonstrated in order to get the necessary support for further development. I always envied individuals who were able to get a company to support something new by talk alone!

    I imagine the majority of significant new boat designs intended for serious production are the output of a team, there is simply too much work to get done in time to catch the market window before the competition.
  7. Tad
    Joined: Mar 2002
    Posts: 2,303
    Likes: 185, Points: 73, Legacy Rep: 2281
    Location: Flattop Islands

    Tad Boat Designer

    Good point Tom.....

    Mostly the OP is a collection of my thoughts after reading though another series of posts on the Origami Boat Construction thread. I see no point in the name calling and derision of others...but I think the process is generally healthy.

    It is a good thing to have heretics among us.....our cages need rattling occasionally......preconceived notions need periodic re-examination. On the other had we need those willing to repeatedly stand up and say, "uh-uh, that's not the case" and then to patiently explain why that is so. Which is the tougher job?.....:?:

    Years ago I was requested by an editor to write a review of Windhorse as well as a statement made (in print) by Steve Dashew in regards to powerboat designers and their negligence of stability issues. I wrote the piece but it was unpublishable (apparently). More than inflammatory statements about stability, I think my comparison of Windhorse and a Nordhavn (big advertiser) 50' got me in trouble.

    Anyway, I don't think we can depend on print or commercial web sites to do any real peer review. So it has to happen on the neutral internet....and is the place for yacht and boat design peer least until this place is bought by some multinational.......:eek:

    This is really intended as encouragement to those who try to write the truth as they see it.....don't give up....but also limit the criticism to what a person has one is served by invention of thoughts or words......
  8. welder/fitter
    Joined: Jun 2008
    Posts: 407
    Likes: 32, Points: 0, Legacy Rep: 423
    Location: Vancouver

    welder/fitter Senior Member

    Is your design your "child"?

    Normally, I wouldn't post to a thread such as this, because I'm not a boat designer. But, I have a question that I believe is best answered by veteran designers.

    When my father decided to retire, he asked me to market his consulting practice for him. One of the reasons he gave for involving me was that he knew that I would be dispassionate, whereas, he felt like he was selling one of his kids. Because most of my working life has been for someone else's business, and the odd business I've had has been relatively short-term, I have never understood what my father was talking about.

    I can appreciate that a designer wouldn't want one of his/her designs criticised, but does it go further than that? Does the average long-time designer define himself/herself by their designs? Is critiquing a design - regardless of how it's done - the same as criticising one of your kids?

    I'm not asking about a specific "designer", like BS. I'm asking you, yourself, as a designer.

    (I hope I haven't asked this before & forgotten)
  9. mcollins07
    Joined: Jan 2006
    Posts: 220
    Likes: 11, Points: 0, Legacy Rep: 166
    Location: Texas

    mcollins07 Senior Member

    On this thread there seems to be three different types of reviews, all called peer review:
    1) Peer review for scientific merit, such as for scientific journals which are marketed to scientist, not the general public.
    2) Peer review during Development, which is reviewing a work-product by members of the same team, reviewing it against a set of requirements.
    3) Product review, where a finished product is reviewed usually for the consumers.

    The way a review is conducted greatly depends on the audience and the purpose of the review. The personality conflicts that exists in these forums often is a result of mixing the intentions. A forum by its nature is public. The audience can not be limited to only the professionals. It seems to me that many professionals would not consider a public forum the appropriate place to discuss merits that only other professionals are really qualified to interpret. Public forums are a mixture of debate techniques and logical arguments. A constructive professional peer review of merits would require a referee to guide the focus on logical arguments, not debate techniques which are very effective but rely on fallacies for their effectiveness.

    A public forum, an exchange of ideas and opinions, can be very useful; however its use for peer review is probably limited. The usefulness of a forum review process might be more effective it the intended purpose, intended audience, and guidelines are spelled out clearly.
    ancient kayaker likes this.
  10. Tad
    Joined: Mar 2002
    Posts: 2,303
    Likes: 185, Points: 73, Legacy Rep: 2281
    Location: Flattop Islands

    Tad Boat Designer


    I was talking with a friend last evening and he related a Phil Bolger statement that shed some light. My friend asked Phil about a particular design and it's attributes....Phil answered something to the effect that, " it hasn't been built yet and it may not be any good at all"......We both laughed at the idea of L. Francis Herreshoff saying something like that about one of his designs.....just not happening........

    Some designers are humble, others are arrogant, most are somewhere in between.......Boat designs are a bit like your children, you're there at conception, but they very quickly take on a life of their own....go out into the world and are successful, or not....but mostly it's beyond your control after the initial effort. Also like children you are proud when they do well, and saddened when they screw up, and always anxious to hear news of them or help in any way......

    I have worked with designer's who could not bear anyone "screwing up" their design, such as an interior designer or exterior stylist, or a builder who wanted to make a change.......and I've always been impressed by other designers who could accept changes with equanimity. I think we need the extreme perfectionists like LFH, where every facet is sacred and you dare not change anything if you want the designer's blessing. But LFH was as capable of making mistakes as anyone, and with hindsight improvements can be made to his designs.......


    In my original post I was specifically addressing peer review aimed at consumers.....the boat buying or building public. Designer's writing about the work of other designer's.......

    Designer's are responsible for design decisions, but they are also responsible for explaining (IMO) those decisions in a manner that Joe pedestrian can understand.........and if a designer sees unexplained problems or possibilities in the work of is his (due to expert knowledge) responsibility to report same......In this way we all benefit from a better informed, more discerning clientele.

    Just writing, "I hate those window shapes" is mildly interesting, but not very helpful........Writing how they could be improved and why is far more interesting and informative....
  11. dskira

    dskira Previous Member

    thank you Tad to start this thread, thanks the other for continuing.
    I like to read what I can't express been not good at writing.
    But some of the thought are quite eye opening.
    I think magazines are gone as for a true and interesting information. Even WoodenBoat is part of some kind of religious group "per se"
    As for Classic Boat, it doesn't know where to go, Yachting World is closer to Vogue every time. The other one, are just brochures for the advertisers.
    Yes the writing and sometime rambling (including mine I hope) of the free of any financial second agenda platform are very useful.
    In my cabin in Maine, way off any people, I really need you all, as bizarre it can sound.
    I get more and more like an hermit.
    I am not a blogaholic. I don't blog. I don't forum.
    But this forum and the other one, Boat-N-Stuff, are the one I go for a while every day. They are just oposite on earth, that's something when you think about.
    I like the exchanges, I learn, I gave my very opinionated opinion and I really enjoy this boating community.
    I learn from you all to temper myself and take a deep breath before posting (not always) and the forum gave me my window to the boating community.
    A great thanks to Jeff and Malasai to allow us to communicate.
  12. mcollins07
    Joined: Jan 2006
    Posts: 220
    Likes: 11, Points: 0, Legacy Rep: 166
    Location: Texas

    mcollins07 Senior Member

    I'm not sure if there is enough interest because I think it would require a significant amount of joint effort. But this forum might be the appropriate format for a Product Review with an unusual depth of comparative analysis.

    Here is my vision. First step is to select 3 to 5 boats, not more than 7, where the designers have stated similar intended use. I think the intended use should be spelled out in detail and used as a ruler for the analysis. Hopefully, this ruler would help prevent direct attacks on the designers. After the boats for comparative analysis were agreed on, the analysis would follow the design spiral. This design spiral could initially be the one proposed by Larsson and Eliasson, and the definition refined as it is used. The steps of comparative analysis would also be instructive as examples of design methodology.

    Can anyone envision this working on the forum?
  13. ThomD
    Joined: Mar 2009
    Posts: 542
    Likes: 13, Points: 18, Legacy Rep: 111
    Location: TO

    ThomD Senior Member

    In Phil's case he was just being honest. Some of his boats where spectacularly unsuccessful, and he then wrote about them.

    The scholarly peer review has several characteristics that are not present in our world. And sometimes not present in the modern scholar's world... The idea is to add to the body of knowledge. To that end they tried not to step all over each other's toes to and all be slaving on the same problem. This is not practical relative to a problem like "most popular boat". Everyone wants in on the same lucrative market (which is more and more true in any form of commercializable scholarly work). The objective though is to increase knowledge, and reveal it publicly;

    The review is then undertaken of the work by one's peers. This normally precedes publication, and the forms the entry into accepted work in the field. In other words it is the act of giving away for free the work product. Of course there may be patents and copyrights, but the info is in the broader world. And a lot of scholarship is of no direct commercial value.

    The reviews far from being in magazines by popular writers, for sport, are often done confidentially without attribution, to save feelings in what may be a small field of study.
    Joined: Oct 2002
    Posts: 4,519
    Likes: 109, Points: 63, Legacy Rep: 1009
    Location: Conn in summers , Ortona FL in winter , with big d

    FAST FRED Senior Member

    "I think the intended use should be spelled out in detail and used as a ruler for the analysis. "

    Like "this vessel is for dockside , lakes bays and protected areas only"?

    "More than inflammatory statements about stability, I think my comparison of Windhorse and a Nordhavn (big advertiser) 50' got me in trouble".

    I'll bet , boats are marketed as DREAM MACHINES , not vessels.

    Reality will always get one in a big heap.

    The TRUTH will not set you Free , where 20% of the boats sales price is marketing.


  15. tom28571
    Joined: Dec 2001
    Posts: 2,472
    Likes: 114, Points: 63, Legacy Rep: 1728
    Location: Oriental, NC

    tom28571 Senior Member

    I think it would be very instructive for any considering pursuing the peer review idea to go back and read ALL of the material on the "Option 1" thread below. For those not familiar with this exercise, it was an attempt to do a cooperative design of a particular type of small powerboat. The first obstacle was to agree on a fixed set of design parameters that the design should adhere to. There was only loose agreement to these and most continued to work to their own variation of these goals.

    Several designs and construction methods and materials were considered. In the end the thread withered and died. I think we all learned quite a bit from each other but we could never agree on what an optimum boat would be for the intended use. As a result, each person went their own way with some actually building a boat to satisfy the intended use from their personal perspective.

    Perhaps a peer review exercise would follow a similar path with divergent opinions complicating the result. The reviewer is seldom going to be able to get inside the designer's head and fully understand what drove the decisions that control the outcome. Pleasure boats are thing of passion and it is extremely difficult (impossible?) to eliminate one's personal tendencies and prejudices from the "dispassionate review" of someone else's work.

    What is wrong with that? Its a good thing that designers don't adhere to the same line of thought or seek the same path, even to a common goal. Sure, there are some real clunkers out there and some of them are very popular with the buying public. There are also some marvelous boats but each will have its supporters and detractors and none are ever perfect. Some, like the NH Alerion I watched being sailed yesterday by an 85 year old friend come very close though.
Forum posts represent the experience, opinion, and view of individual users. Boat Design Net does not necessarily endorse nor share the view of each individual post.
When making potentially dangerous or financial decisions, always employ and consult appropriate professionals. Your circumstances or experience may be different.