Prime Trust has been losing clients and deposits to competitors for weeks amid mounting concerns over its business, a source at one former client told CoinDesk.