On Ubuntu 10.04 it wouldn't remove the taskbar button cleanly, so I made it leave it there.
But now that you mention it, it's probably better to have the feature, even if it's messy, than not to have it, though it may confuse a few people when the taskbar button temporarily stays around but disappears if you click on it.
Updated SVN.
Thanks for testing.