If it was feasible to use a fabric, the mounting would be the greater part of the problems. The faster you go, the more tonnage force is striking the front edge, or radius or whatever is at the leading edge of the device. That means a lot of leverage over its length to counteract, needing strength, reinforcing, and possibly bracing. Then when under way, immediately behind the leading edge, two flat expanses have to resist the water pressure from both sides, as well as the lee pressure; so internal framework, or foam, or fluid pressure. The clamping of a fabric to a rigid leading edge could be simple or complex, and the strength needed means weight; then what holds the trailing edge from distorting under load. The underlying framework would need to be designed so the fabric doesn't wear excessively along any edges when flexing. I can't imagine anything inflatable being useful beyond a few knots, before creasing and folding up. Try holding a stick steady and vertical in the water at say 5 knots.
I think you would need more structure just to make it work, than what would be needed in a normal solid board shape.
I did imagine an electromagnetic fabric which either locks up under power, or goes limp under power, but electricity and water often mix in a bad and unreliable way.
Fabric soaked in resin would work, like electrical insulation board, but defeats the initial idea; its a board.
Hydraulic rubber and steel wire fabric could be fashioned into a 'fabric' centreboard / keel which is flexible to a degree, and rigid when pressurised, but it would be heavy, expensive, need ancillary equipment, and would be hell to make.
Will's idea of: "a short, but full length fabric keel, something only a hand breadth in depth, but the full keel length, might be effective." seems like the most likely method to not fail immediately. Something like that could be powered and become a low speed propulsion system too, sinusoidal oscillating like squid fins.