A charity has dropped Gregg Wallace as an ambassador after multiple allegations of highly inappropriate behaviour.