Well this is a bit annoying. I had been closely following the various email blasts from VOIP.MS that are intended to let customers like me know how to react to the denial-of-service attack (see recent blog posts). I had gone to quite a bit of trouble to reconfigure several phones and ATAs at several physical locations to make use of the chicago3 server instead of our usual denver2 server.
Then sort of on a whim I happened to click around on the Twitter feed of VOIP.MS. There, sort of as an aside, the people at VOIP.MS happened to let slip that each of their servers that has gotten “hardened” against the DDOS attack, and that now has a green check mark, is no longer supporting encryption. Each of the green-check-box servers is usable only on port 5060, not port 5061, and you can’t use RTSP.
This means that all of the hard work that I did to reconfigure several phones and ATAs at several physical locations to make use of the chicago3 server instead of our usual denver2 server was a waste of time. Those phones and ATAs still will not work because they are all set up to use TLS and RTSP for full encryption of the telephone calls.
Now I get to start all over again, clicking through VPNs and otherwise doing whatever is needed to log in to each of the various phones and ATAs to do about four times as much reconfiguration as I had previously understood to be necessary.
Previously I thought that all I had to do was find the screen or popup window where “denver2.voip.ms” appears and change it to the IP address of the chicago3 server. But now for the first time, only sort of by accident by clicking around in a twitter feed, I have learned that I must also:
- click around to find the screen or popup window to change “5061” to “5060” for the SIP port.
- click around to find the screen or popup window to change “TLS” to “UDP” for the SIP protocol.
- click around to find the screen or popup window to change “SRTP” to “RTP” for the audio transport protocol.
In all of my devices these settings are in three different places — a first place for the server, a second place for the SIP settings, and a third place for the audio protocol settings.
And it is not only me. Each of my staff people is going to have to go through this much more complicated reconfiguration process. Once right now to get to a “green check box” server, and again at some future time when it once again becomes possible to turn the encryption back on and to migrate back to a Denver server.
Oh and not only that. It is going to be necessary right now to turn off encryption for each of our SIP trunks (what VOIP.MS) calls “subaccounts”. And at some future time it will be necessary to turn the encryption back on for the SIP trunks.
When I last looked at IP phones, the several models I reviewed all had some sort of remote management capabilities. Don’t yours, or did you turn that feature off as a security tradeoff?
Oh yes they all have remote management and I have not turned it off. But in each case, I need to be in the same subnet as the phone (or ATA). For most of these subnets, I have some VPN set up so that I can virtually be in the same subnet. But it is still a pain in the neck. I often set up an extra layer so that the VPN can only be reached if I say “captain may I?” first to reduce the chances that a bad person could do a brute-force attack and get into the VPN. And so I have to consult my rarely-consulted notes to remind myself what the “captain may I?” is that lets me log in at the VPN, followed by consulting my rarely consulted notes to recall what the credentials are for the VPN, followed by consulting my rarely consulted notes to recall what the login credentials are for the phone or the ATA. And there is one where I somehow did not seem to do the VPN just right and my punishment for that is I have to get in a car and I have to go in person to the location so that I can be in the same subnet as the device.
Layered on top of this is that we are in a post-Covid world of work-from-home. Some of the phones and devices are in employee homes where the only person who can do the remote administration is the employee himself or herself. My employees are very tech savvy and they will suffer through anything to help keep our office infrastructure working. Having said that, I don’t much like that I already told them to log into their phones and change the address of their SIP server, and then log out again. (Which involves some complicated internal steps such as turning off a router-based VPN and later turning it back on.) And now I have to tell them to do the complicated internal steps again, such as turning off the router-based VPN, and log in to their phones again, and now surf around to find a second UI admin screen to change the SIP port and SIP protocol, and now surf around to find a third UI admin screen to change the audio protocol, and then do a “save and apply”, and then turn the router-based VPN back on again.
And later when the VOIP.MS folks manage to restore the encryption functionality in the hardened (green check box) servers, I have to ask my people to do all of these steps with router-based VPN and the SIP port number and SIP protocol and audio protocol all over again. And turn the router-based VPN on again.
I guess I should have know the problem was not so simple. I find web-based management interfaces to be tedious and error-prone, and I try hard to reject equipment and software that doesn’t allow for convenient scripted management using command-line tools. Though sometimes there’s no good choice that meets that constraint.
Two things:
1) On most VOIP phones, such as Polycom models, you can set up a TFTP server with configuration files for your phone models. Each phone gets set up to look for a certain server address at boot up, and the phone then automatically updates. This way, if you have 100 phones, they can all get updated quickly. But most customers have phones automatically configured by the phone company to get updates from the company’s configuration server, rather than one set up by the customer.
If you’re amazingly successful, you could contact the company and ask them to update their own configuration files to reflect alternative servers. By the time you set up a TFTP server yourself, create the config files, and repoint each phone to the new server, you could just have had your staff update their individual phones…
2) The process you described sounds like a lot of work, and it’s unfortunate that the company isn’t communicating with customers on how to resume operations. Good luck, and thanks for posting. Feel free to e-mail me if you want to go through this process.
Thank you, yes, this would be a nice aspiration. If we had 100 phones, all of the same make and model, then yes it would be maybe a realistic aspiration. The payoff could conceivably be worth the prodigious one-time cost of setting it all up. Of course one would need to have a crystal ball that works well enough to predict which fields in the phones are the ones that will some day turn out to need rush-rush updates like this. In our situation, we have three of one model of VOIP phone made by one company, three of another model of VOIP phone made by that same company, three of yet a different model of VOIP phone made by a different company, and maybe five Cisco ATAs that provide analog telephone adapter functions for fax machines and analog voice phones. The fax machines use their ATAs to connect to fax-specific SIP servers. The other devices connect to voice-specific SIP servers. Of course we would need to construct something like six distinct config files, each of which provides a payoff only about two or three times each.
And one would need to be cautious in the design of each config file. One would want to avoid inadvertently damaging, say, the SIP user ID or SIP password, which is non-identical from one device to the next. In each phone that has a busy-lamp field, the BLF config is non-identical from one phone to the next, so one would want to avoid having this automatic update causing harm to the BLF config.
Sad but true – I won’t even go into the impressively large number of hours I’ve lost setting up Linksys/Cisco SPA122 and SPA3102 gateways, until I created configuration instructions (which are, of course, completely different for the devices). And since your phones and network environments vary, it’s an exercise in futility. I used Vonage’s business service at a prior firm, which was reliable and flexible. But they hide 90% of the advanced features that you’d want to tweak yourself, generously offering you the option of paying for them. P.S. – thanks for inspiring me to venture down the non-billable rabbit hole of encrypted SIP trunking… Impressive stuff. 🙂