OpenCalc open souce framework

Discussion in 'Software' started by Steve Hollister, Sep 12, 2019.

  1. Steve Hollister
    Joined: Sep 2001
    Posts: 59
    Likes: 8, Points: 8, Legacy Rep: 82
    Location: Rhode Island

    Steve Hollister Junior Member

    OpenCalc, a (free) framework for creating and integrating software from all independent software developers is nearing a formal release. It has been in development for almost 6 years and I have given 5 technical papers on the subject and created three working (beta) versions seeded with free copies of my Intact Hydrostatics and Stability, Lackenby Hull Variation, Savitsky Planing Hull Resistance, and Holtrop Ship Resistance programs. Anyone can create their own calculations (free or fee) for the system, but the goal is to make all software and data compatible. This is not my framework for sale, but a free and open source framework for download by anyone. Again, what you add to it may be free or for sale.

    OpenCalc splits traditional "Apps" into three parts: batch Code Engines (CE), common VXML data files, and common User Interface Frameworks (UIF). Included with OpenCalc is a free Excel spreadsheet (UIF) with open source macro code that can launch any CE hidden in the background and provide looping, printing, plotting, and contouring. It looks like the calculations are done inside the spreadsheet. Since CEs are standalone, they can also be wrapped together by users to create completely new solutions with no custom programming. In one of my papers, I showed how I wrapped my Lackenby Hull Variation CE with my Hydrostatics CE and my Holtrop resistance CE to search for an optimum solution - all with no custom programming. This recreated the work for a paper I gave to SNAME 20+ years ago(!), but that was in DOS and all of the code was custom and tied together.

    Many programs are islands of calculation that are difficult to learn and to integrate with other code. There is an open source ship motions program called NEMOH, but it's up to you to figure out how to tie it in with your CAD and other software. OpenCalc provides a framework for everyone to add pieces to tie everything together. Common data definitions will evolve that do not have to be manually translated to and from neutral file definitions, like DXF, IGES, and STEP. Mesh definitions can be generalized and OpenCalc will provide open source I/O code for all programming languages. One could create a NURB to mesh generator CE and then add in my hull variation CE and then perhaps a Michlet or NEMOH CE to automatically search for an optimum shape. Users would be able to "wrap" CE together and launch them from the standard spreadsheet UIF front end.

    OpenCalc is the result of my 45+ years of programming software for the marine market with complex programming and a continually changing technological environment. I still have old DOS code sitting on my computer that does not justify conversion to full Windows apps. (Where is my old DOS NURB surface to Michlet data file code?) However, it will be simple for me to turn them into stand-alone batch CEs. I envision a world where anyone can create compatible CEs with the open source programming templates and then upload the code for download - for free or fee. Some refer to this as coopetition, where we all can work together and compete at the same time. However, we need an open framework to get this done, and OpenCalc will be completely free. It's independent of all of my other software development. I plan on turning the underlying TSPA code into a formal open source project that is independently controlled and funded. This is not a method for me to have an advantage over other programmers.

    This work started in early 2014 when I got a call from Rik van Hemmen where we (again) tried to figure out a way to solve the problem of "lost" programs and changing technology. This initiated a connection with SNAME and the start of what was called "Project 114" which later became OpenCalc ( Home - Project114 ) . Over the years, I generalized the concept to all areas of programming and set up the website for Tri-Split Programming Architecture (TSPA) ( ). This site has complete details on how everything works, including a big discussion on how this work fits into the larger world of cloud programming, outsourcing, common data, and "agile" programming. The "Paper Trail" page gives a complete description of how this all evolved and links to all technical papers, including my last one to Compit 19 in Ireland in March.

    A separate website will be set up for OpenCalc and the download will be free for all. I will likely again seed the framework with some of my software for free. The hydrostatics and stability software and the Lackenby Hull Variation software can work with GHS, SHCP, and LFH station definition files. When it's ready to go, I hope to have a "Parting Shot" column in Professional Boatbuilder to announce it. I will also announce it here.

    For programmers interested in getting ready for this framework, all you have to do is to write your code in a batch form like a big subroutine of input and output variables. The open source templates in C/C++, VBA, Java, Python (and FORTRAN and more later) will handle all of the data file I/O using simple API commands like "Get" and "Put." That's all you have to do. The Excel UIF front end will launch the CE with no programming, and anyone can wrap your CE with others to do more complicated calculations. I'm looking forward to the first wrapping of a NURB surface variation CE (I have to dig up my old NURB surface/Lackenby variation code) with a Mesh Creation CE followed by a CFD program. (OpenFOAM as a CE anyone?) Regular "users" will be able to wrap CEs without a large corporation budget.

    Also, I have shown how my ProSurf CAD program can launch independent CEs and my goal is to get all CAD programs to adopt this open standard. If they do so, then you will be able to write an independent CE and have it work stand-alone with the common Excel front end or with any compatible CAD program with no custom programming. Instead of being tied to one CAD program, your code will run on any. You will write a batch CE once and then the open source framework will allow it to be used in many different ways. See both web sites for complete details.
  2. zukobo
    Joined: Sep 2019
    Posts: 63
    Likes: 15, Points: 8
    Location: taiwan

    zukobo Junior Member

    Very interesting... Thank you for sharing this
  3. Alexanov
    Joined: Feb 2003
    Posts: 235
    Likes: 30, Points: 38, Legacy Rep: 10
    Location: Norway Sola

    Alexanov Senior Member

    Very interesting!
  4. Dejay
    Joined: Mar 2018
    Posts: 721
    Likes: 138, Points: 43
    Location: Europe

    Dejay Senior Newbie

    Thanks this sounds really interesting approach! Although I'm not quite sure how this fits together.
    This all runs on VBA inside spreadsheet software? (MS office?)
    Is this planned to work with "OpenOffice Calc" spreadsheet software?
  5. Steve Hollister
    Joined: Sep 2001
    Posts: 59
    Likes: 8, Points: 8, Legacy Rep: 82
    Location: Rhode Island

    Steve Hollister Junior Member

    Calculations (Code Engines-CE) are written in any programming language using open source templates to create stand-alone batch programs with no user interfaces. These "CEs" exist independently and can be used directly from the command line. However, they can be sequenced or wrapped and combined with any compatible user interface program.

    As an example of one type of generic user interface (UIF), I wrote one in Excel with VBA to launch any external Code Engine with no programming. I hope to have that translated to all other spreadsheets for all to use. However, the open source software shows how to create standard user interfaces in any programming language so that they can launch standard Code Engines. In most cases, there is no reason to write a custom user interface for each "app." A game engine is a form of UIF. I tested my ProSurf CAD program to have it launch external CEs from anyone. I want all CAD programs to be able to do that. In face, any program user interface could be changed to launch external CEs.

    The third part defines one XML text file (like HTML) that can store any program variable or data structure from any program. I am providing open source software in all programming languages that can read/write any data. This means that CEs can store and retrieve common boat/ship data to/from one file, like a database. There are many cases where separate and independent calculations need to be combined to run automatically. Apps with fixed user interfaces prevent that.

    TSPA/OpenCalc is a new open source definition of basic computing components that can be taken off-the-shelf and combined by anyone to create new solutions with little to no extra programming. There is a lot of interest in this work (going far beyond boat/ship design) because it can do things not possible any other way. HoliSHIP is an attempt at this, but it containerizes apps (that have to be converted to batch programs) and the main UIF driving user interface (CAESES) is proprietary. TSPA/OpenCalc is a completely open framework, but you can add in your own proprietary calculations to sell. See for complete details, history, and justification.
    Dejay likes this.
  6. Zeeminner
    Joined: Nov 2019
    Posts: 2
    Likes: 0, Points: 1
    Location: Rotterdam

    Zeeminner New Member

    Interesting! I would like to become a user/contributor. I am awaiting the launch. Any (broad) guess when it will be?
  7. Steve Hollister
    Joined: Sep 2001
    Posts: 59
    Likes: 8, Points: 8, Legacy Rep: 82
    Location: Rhode Island

    Steve Hollister Junior Member

    Even after three working (beta?) versions over the last 5 years, parts that make it useful for all are still not done. I have C/C++ and VBA interfaces done, but I'm working on Java and Python interfaces, and then JavaScript, FORTRAN (for legacy code), and hopefully MATLAB too. The idea is that all you have to do is take one of these program templates, drop in your code, and then your "code engine" batch program will immediately work with the standard Excel and, eventually, other spreadsheets.

    I also hope to have these independent code engines work automatically with any CAD program. I can show how it works with my ProSurf/Pilot3D software, but I want all CAD programs to adopt the open connection. You should be able to write your code engine once, test it out separately, and then have it automatically usable with any compatible user interface - spreadsheet, CAD, or anything else. That's what my release versions have shown, but more tools need to be developed to make it usable by all. Remember that this is an open/free system for anyone to use, but the code engines you write need not be free. My expectation is that since these calculation and data processing tools will be less expensive to create, and since the usable market will be much larger, they will be a lot less expensive. Like Kindle, you should be able to upload your creative work and have it downloaded by others for a fee to use in many different ways, even in combination with their code engines. A paper I gave in 2016 showed how multiple separate code engines were "wrapped" to do hull variation, hydrostatics, and resistance calculation to search for an optimum hull shape with no extra combination programming. There is no other way to do this without having all of the source code and writing a custom UI/app to make it work. HoliSHIP is working towards this end, but you have to custom containerize your code and data file format, and it only works with the proprietary CAESES system.

    On top of all of this is the rapidly moving paradigm of open and collaborative software that lives on the web. Recently, PTC acquired Onshape for $470 million. It's the first web-based full CAD program. It hadn't reached a break-even subscription sales point, but PTC was buying the future. The key to the web are cross-platform, browser-based, collaborative programs where the user interfaces are written in JavaScript. This world is built on evolving open tools from The Khronos Group, like WebGL, Vulkan, OpenXR, and the glTF format. It was in the 1980s that game programming began to drive the technological market and not engineering and CAD. The top end technological web world is still driven by gaming, but it now includes the commerce world, driven by the glTF 3D object and scene rendering file definition. (They like to say that glTF is to 3D as jpeg is to 2D, but glTF includes the actual 3D geometry, not just pixels.) It won't be long before you can buy an inexpensive XR (virtual and augmented reality) headset to see IKEA and Wayfair products sitting in your living room in 3D using the web. Many browsers (Chrome is a leader) can now handle glTF files so that if you upload one that includes a 3D hull model, then a user in a browser can rotate it and visualize it. It's not a video, but a full 3D model that the user controls. You won't need many pictures of your boat on your web site. A user can take your glTF model and do walk-arounds and walk-throughs. Blender rendering software now has glTF input and output.

    What my open source work with TSPA/OpenCalc adds to this is the ability to have common user interfaces that live on the web launch code engines in any programming language and located anywhere on the web. Calculations will be able to use common data files and IO software. No neutral file conversions will be necessary. Rarely is there a need to write custom user interfaces, but if one of the web user interfaces is a spreadsheet, then it can be customized. Also, others could write their own JavaScript UI app to launch code engines. However, most developers will only ever need to do the separate calculations and use common UI front ends. One problem with the Web app model is that people think that the entire app (UI + calcs) have to be written in JavaScript. I have shown how that is not necessary.

    Even though I come from a naval architecture background, my work can be applied to all areas of programming. The work is a moving target especially with the advent of web programming with WebGL. I'm trying to show how this works with browser spreadsheets and I'm converting my CAD software to a WebGL format, but the goal is to get all to adopt this standard.

    So, finally, when will this be ready for official download? On one hand, I want to wait to provide more tools, but on the other hand, some are willing to get ahead of the curve and deal with a moving target. It won't move a whole lot because the concept is simple. All you have to do is to write a batch program using one of my open source templates that does all of the file input/output. If anything has to change, it won't be your calculation code, which is 99% of the work.

    For boat designers, you will be able to download an Excel spreadsheet that links to (seeded with) 5 code engines that I've provided for free. Normally, this won't be the case. Others will provide their own code engines for fee or perhaps free. It's a simple process to add your own code engine to this list, but I have no Kindle-like process set up yet. If you write your own hull shape generator code engine, you will be able to swap it with my Lackenby hull variation program to use the existing hydrostatics and hull resistance wrapped loop. Again, I'm waiting for someone to offer an OpenFOAM code engine and separate meshing tool.

    It now looks like spring before I can get this web site done and the download ready. In the meantime, all you have to do is to separate your code into simple batch programs with no user interface. Hard code in dummy data for testing and that can later be swapped for simple get and put commands to the standard XML data file. I will be making the rounds of all of the boat design and naval architecture schools to promote it. I expect it to be a big hit with students.
    Mikko Brummer likes this.

  8. Andy
    Joined: Aug 2003
    Posts: 280
    Likes: 13, Points: 18, Legacy Rep: 45
    Location: Edinburgh

    Andy Senior Member

    Hi Steven, any progress updates?
Forum posts represent the experience, opinion, and view of individual users. Boat Design Net does not necessarily endorse nor share the view of each individual post.
When making potentially dangerous or financial decisions, always employ and consult appropriate professionals. Your circumstances or experience may be different.