Some years ago our law firm migrated away from every old-fashioned landline telephone provider that we used to use, and we moved everything about our telephone service to a Canadian VOIP telephone company called VOIP.MS. I have blogged frequently about my satisfaction generally with the use of VOIP rather than older ways of getting telephone service, and I have blogged frequently about my satisfaction in particular with this company as a provider of such services.
Which then leads to a sense of wonder and frustration to see that somebody has chosen to bring a denial-of-service attack against the VOIP.MS company, and has asked that a bitcoin ransom be paid for the DDOS attack to cease. (See Ars Technica article.) This has led to various disruptions in service for many of the 80,000 or so customers of VOIP.MS, some of which (as for my firm) have been intermittent and some of which (for some customers) have been pretty much continuous.
What can a customer do about this, if anything?
A first thing I suppose is to review briefly the mix of services that one might use to construct a law firm telephone system. This includes:
- DID service
- PBX service
- physical telephones
- outbound calling service
The world of VOIP is sophisticated enough that the customer who wants to pick and choose can use a first company for one of these things, a second company for another of these things, a third company, and a fourth company, and so on.
DID service means “what happens when somebody dials a phone number”. When somebody dials, say, the main telephone number of our law firm, that is DID service. The call goes to whatever company we pay to handle that phone number. The call then passes through a VOIP connect to a PBX and from there to an automated attendant or to voice mail or maybe to a physical phone, depending on how we have configured things.
In our case, we own our own physical telephones, and we program them to connect (via what are called “SIP trunks”) to servers provided by the VOIP.MS company. Some firms choose to bundle a rental of physical phones with other things like the DID service and the outbound calling service, or choose to pay a local service provider for a turnkey solution that includes “all of the above”.
Outbound calling service is what it sounds like. This covers local calls, long-distance calls within one’s own country, and calls to other countries. We choose to get this service from VOIP.MS.
VOIP.MS provides a cloud-based PBX free of charge to its customers, which I think is pretty neat. I have blogged about this.
What exactly is the DDOS attacker doing to try to bring VOIP.MS to its knees? As best I can tell, the attacker is doing two things.
A first thing is that the attacker is trying to make it nearly impossible for anybody to do DNS lookups for anything in the “voip.ms” domain name space. So for example suppose one of our office phones has been configured for many years now that it should “register” its SIP trunk to “denver2.voip.ms”. This means that the phone carries out a registration process once every ten minutes or so. It starts by doing a DNS lookup to find out the IP address for “denver2.voip.ms”. The answer is (usually) 22.214.171.124. The phone then initiates a SIP connection to that IP address. The SIP connection is then used for receiving incoming phone calls and for placing outgoing phone calls.
The VOIP.MS people made a decision, years ago, in their system design to make the TTL (time to live) for DNS lookups quite short — just one hour. The idea was that if one of their servers were to fail, they could change the A record to point to some other server and within an hour, everybody’s phones would have eventually re-registered to the alternative server. They could then do what was needed to repair the broken server and the disruption for customers would not be too terribly lengthy. I think that was quite smart.
Except of course what if a bad person decides to attack the DNS? Then all of a sudden, customer telephones are not able to get the answer to “what is the IP address for denver2.voip.ms”. And customer telephones are unable to place or receive calls.
A similar problem arises if a customer needs to make some change or reconfigure something. The usual way to do this is to log in at the web site www.voip.ms where there is a customer portal. The customer portal is the place for configuring the cloud-based PBX, purchasing new DIDs, and topping up one’s payment account. But if the DNS is disabled, then the customer’s web browser is unable to learn “what is the IP address for www.voip.ms?” and the customer is not able to get anything done.
I have to imagine the attacker is also trying to flood the www.voip.ms web site with TCP SYN packets or some other well-known line of attack that makes it impossible for legitimate users to log in at the web site.
If you click around in Google News you can learn that this is not the first time that some attacker has attacked some VOIP service provider and asked for a ransom. Other VOIP providers in other countries have been attacked in recent months.
So what can or should a customer do if the customer is affected by an attack like this?
One possibility is to reconfigure your phones (or ATAs, or whatever SIP clients you are using) so that they register their SIP trunks using IP addresses instead of domain names. So for example if you usually use “denver2.voip.ms” to identify your SIP server, you would use 126.96.36.199. The advantage to this approach is that you are no longer harmed by the DNS attack. Your phone knows the IP address without having to pose the question to a DNS system that might not be able to answer the question. Of course this approach has the terrible drawback that you will miss out on the automatic redirect to an alternative server that would normally save your service if the usual “denver2” machine were to crash and if the smart people in the VOIP.MS network operations center were to make a plan of redirecting your traffic to a different physical machine.
One colleague of mine wonders if what he should be doing is methodically switching his telephone activity to some other VOIP service provider. He wonders if some other VOIP service provider somehow has a network that is more resistant to DDOS attacks? He wonders if some other VOIP service provider is so small that the attackers will not feel it is worth attacking, and so he should pick that company. He wonders if some other VOIP service provider is so big (he wonders if there are providers who are much bigger than VOIP.MS) that the attackers will not succeed in attacking the much bigger company because they are so much bigger, and so he should pick that company. He wonders if some other VOIP service provider is somehow run by people who are much smarter and somehow could react faster or better to such DDOS attacks?
I do not, of course, know the answers to such questions. But I have lots of guesses. I will share the guesses.
A first guess is that probably every VOIP provider who provides unbundled DID service and unbundled outbound calling service and unbundled PBX service is at risk to pretty much the same kinds of attacks. If you pick a provider who controls everything about your service (they own the phone on your desk, you don’t get to configure anything yourself) then probably they make it so that everything uses hard-coded IP addresses and there is much less vulnerability to DDOS attacks on DNS services.
From this, my guess continues with a guess that if you were to try to migrate your services to any other company that provides more or less the same kinds of services that VOIP.MS provides, you will be migrating to a company that will be largely at risk to the same general kinds of attacks. The new company that you are considering switching to may have had just such a DDOS attack two months ago and you did not know it. Or they may be the company that is next in line for a DDOS attack two months from now and you just don’t happen to know it.
A further guess is that “security through obscurity” is probably not the way to go. By this I mean that picking a new VOIP service provider specifically because they are small and don’t have many customers, in the hopes that the ransom attackers will choose not to attack the company, might not be a good strategy. For one thing, if the attacker does finally get around to attacking the small fry, they probably would have even fewer resources to fight back or react. For another, a small-fry company might have other drawbacks just because they are small. Oh and there are quite a few companies that are small fry companies that are actually basically just resellers from bigger companies but they conceal it really well, and you only figure it out much later if ever. So they would get brought to their knees as soon as some attacker decides to attack the bigger upstream provider that the little company is reselling from.
I am pretty sure there are a few North American VOIP providers that offer the same types of unbundled services that VOIP.MS offers that are bigger than VOIP.MS, but I think there are not many of them. I know of one or two that I suspect are bigger. I have tried to use a couple of them and I have found them much harder to use from a technical point of view. Meanwhile, VOIP.MS is quite big and, I think, is run by very smart people. So I have no particular reason to think that migrating one’s business to a different service provider is going to somehow lead to a situation where smarter people are providing your service.
Our phones are running on the denver2.voip.ms server, and it seems to be running okay just now. But for example the chicago7.voip.ms and chicago3.voip.ms servers seem to be unresponsive. I am not sure why.
For now I am going to just sort of sit tight and not try changing VOIP telephone companies. I am hoping and trusting that the folks at VOIP.MS will figure out ways to deal with the attacks. Maybe moving their DNS hosting to different servers that somehow are more robust, maybe making the TTLs longer for a while. The have already migrated their www.voip.ms portal over to Cloudflare and that seems to be working better.
I welcome thoughts from people. Please post comments below.