Outside Phone on Switchvox

I wish to have an outside phone on my switchvox server. If i open port 5060 to include it everyone jumps on and abuses the voip system. if i switch the port i use and redirect then it registers but i hear no one. Any ideas?