Again, I cannot reproduce the problem. All my tests show that everything is fine, as with the other problem you reported earlier (market creation). When I set the fee to zero, no fees are applied, and when I select the max amount to transact, this works for me.
Something is very strange and I have to think of how to debug it. I am not sure right now what other information to ask you, once I do I will get back to you.
I have now identified a bug that is probably related to causing the “insufficient funds” message. I am still not sure why fees would be applied when you set them to zero. I will look at this again tomorrow.
I have just now released 0.5.3-beta which includes a few bugfixes, including one that I believe was causing the “insufficient funds” message you saw. There is also a new feature, you can now disable buy, sell, and cancel operations individually per market. I was not yet able to identify a condition where fees are applied when they are set to zero.