The only issue is if you are too quick with your wood mix. The individual wood grains in your sawdust must also be saturated. They will take in epoxy from the environment until they are. That might make a starved joint, even if you did everything else well.
Understand, this isn't a dire warning not to use sawdust in your goop. But you asked for possible issue. These are possible issues. If you'd asked me about possible issues relating to sleeping with Charlize Theron, I could provide some there too, but the overall advice would be "Go man go!"