People afraid to speak out on refugee housing issue says MacSharry