Seamless migration from analog phone lines to SIP trunks?

Let’s suppose you have made a plan to migrate your office telephone system from analog phone lines to VOIP trunks.  Maybe you are doing this to reduce your monthly telephone bill to 85¢ as I blogged recently.  But regardless of why you are migrating, clearly you will want to carry out the migration in a way that minimizes the risk of disruption of incoming or outgoing telephone service.  In this article I describe a migration path that worked well for our firm, and along the way I explain a little bit more about SIP trunking and how it works.

A first thing to appreciate is that if you are migrating from analog phone lines to VOIP (SIP) trunks, you will be making use of local number portability or LNP.  LNP is the mechanism which allows a telephone customer to keep its telephone number despite a move from one telephone company (telco) to another.   Many countries now have LNP.  The US was one of the first countries to have LNP, and the discussion that follows assumes that the reader is in the US.  (Most of these comments would apply in lots of other countries, mutatis mutandis.)

When you decide to port a telephone number (or a block of telephone numbers) to a new telco, you do it by submitting a porting request to the new telco, sometimes called “the gaining carrier”.  You provide lots of information to the gaining carrier, typically including a copy of a recent telephone bill from the old telco (“the losing carrier”).  The gaining carrier sends an LSR (local service request) to the losing carrier.  If all goes well, the losing carrier provides an FOC (firm order confirmation) with an “FOC date”.  The FOC date is the date upon which the port is supposed to happen.

By now our firm has been through the LNP process about ten times for various telephone numbers (voice lines, fax lines, toll-free telephone numbers).  In our experience the port always happens on the FOC date.  Not earlier, not later.  It happens on that date.

There are at least three awkward things about FOC dates.

First, you don’t get to pick the FOC date.  You are told what the FOC date will be.  The only real control you have over the FOC date is that it will not happen until some time after you submit your porting request.  This means that if you want to make sure that your number port will not happen prior to some particular date, then you must have the self-control not to submit your porting request until that particular date.  The delay between (1) the date upon which you submit your porting request and (2) the date that is the FOC date is often in the range of 2-4 weeks.  One of our recent ports took two weeks but another one took only two business days.

Second, you may get very little advance warning about your FOC date.  One of our recent ports had an FOC date of June 29, but the FOC date was communicated to us only on the morning of June 29.  In that case our advance warning was about one hour.

Third, you don’t get to pick the time of day for the port to happen, and the actual time of day of the port might be any time on FOC day.  (Our most recent half a dozen ports all happened pretty consistently between about 10AM and noon on FOC day.)

The point of this article is to talk about what you can do to minimize the risk of disruption from a number port that is going to happen on a date that you do not get to pick, at a time of day that you do not get to pick, and that may happen with as little as just an hour of advance warning.

It will be helpful now to talk a little bit about how telephone calls work.

In a country where any particular telephone number might be handled by one telco on Monday and a different telco on Tuesday, how exactly does a telephone call succeed?  Here’s how telephone calls work in the US.  The calling party dials a telephone number, using their telco (“Telco 1”) for the outgoing telephone call.  Telco 1 does a lookup in a database called the NPAC (Number Portability Administration Center) database.  (Toll-free telephone numbers in the US rely upon a lookup in a similar database called RespOrg.)  The lookup retrieves information about which telco (Telco 2) handles that telephone number.  Telco 1 then passes the telephone call to Telco 2 which then completes the call.

This database lookup happens millions of times each day, once for each telephone call that somebody dials to a telephone number in the US.  It’s sort of amazing that this works so quickly and invisibly to users of the Public Switched Telephone Network (PSTN).  The database lookup usually injects much less than one second of delay in the calling process.

If you are doing a number port of a particular telephone number, then on FOC day, at some point during the day the NPAC database entry for that particular telephone number will change.  Prior to 10AM (for example) on FOC day the NPAC database will point the call to the losing carrier, and then starting at 10AM (for example) on FOC day the NPAC database will point the call to the gaining carrier.  Again, as mentioned above, you don’t get to pick the time of day that this is going to happen, and you don’t get to pick the actual day, and you may receive only an hour or two of advance warning when it is going to happen.

Having described the NPAC database and the LNP process, let’s talk about some real PBXs (private branch exchanges).

Figure 1 shows how our firm’s telephone system worked as recently as a few years ago.  Our PBX (call it “old PBX”) received analog telephone lines from the telco (in our case, Qwest).  Each analog telephone line was delivered to our PBX on a pair of copper wires (a “pair”).  In our case the distance from the Qwest central office to our PBX was about three miles.  Each of our telephone extensions was connected to our PBX by a pair of copper wires.  This was costing us around $55 per month per analog telephone line.  The old PBX connected to our telephone extensions (one of which is shown in Figure 1) by means of copper pairs

Figure 2 shows what we changed about four years ago.  The PBX stayed the same, and the telephone extensions stayed the same.  We ported our telephone numbers from Qwest to Comcast.  Comcast provided an ATA (analog telephone adapter).  This ATA connected to a coaxial cable which (through a series of amplifiers) extended about three miles to a Comcast facility.  The ATA provides dial tone, ringing signals, caller ID data and other standard telephone line signaling at a modular telephone jack.  In our case the ATA provided four such analog telephone lines.  Four phone cables, each about three feet in length, connected the modular jacks on the ATA to corresponding modular plugs on our PBX.

At this point it may be helpful to introduce a bit of terminology, namely FXS and FXO.  The modular jack on the ATA is called an FXS jack.  The modular jack on our PBX is called an FXO jack.  In practical terms an FXS jack is the sort of place where you could plug in an ordinary analog telephone and you could place and receive telephone calls.  In practical terms an FXO device is any device that, from the point of view of a telephone company, appears to be an ordinary analog telephone.  Each three-foot-long phone cable just mentioned was connecting an FXS jack to an FXO jack.  As a general matter if you see any FXO jack that is being used for something, it will be connected to an FXS jack.  As a general matter if you see any FXS jack that is being used for something, it will be connected to an FXO jack.

So in Figure 1, the telco provided four FXS jacks, and each FXS jack had a copper pair that was about three miles in length, connecting to a corresponding FXO jack on our PBX.  Each of these copper pairs was costing us about $55 per month.

In Figure 2, the Comcast ATA provided four FXS jacks, and each FXS jack had a copper pair that was about three feet in length, connecting to a corresponding FXO jack on our PBX.  Each of these copper pairs was costing us about $40 per month.  We had cut our monthly telephone bill by about fifteen dollars per copper pair.

So things stood until about a month ago when we decided to migrate from analog telephone lines to VOIP (SIP) telephone lines.  And the question was how to do the migration to minimize the risk of disruption for our incoming and outgoing telephone calls.  This brings us to Figure 3.

Figure 3 shows a new PBX.  The new PBX is a Grandstream UCM6104 PBX.  We selected it because it has four FXO ports.  (It also has two FXS ports but that was not important to the migration process described here.)  The new PBX has telephone extensions within our office, connected through our LAN (local area network) which is an ethernet LAN.  The new PBX also has telephone extensions located at various remote locations such as employees’ homes, connected through the Internet.

As will be appreciated, we were trying to migrate two things at about the same time.  We were migrating from an old PBX to a new PBX, and at about the same time we were migrating from analog telephone lines to VOIP lines.  Strictly speaking it would not have been necessary to link these migrations.

As a first example we could have kept using our old PBX and we could still have saved lots of money by migrating from analog to VOIP.  We could have done this by installing two Cisco SPA112 ATAs as I blogged here.   This would have provided four FXS ports which we would connect to the four FXO ports on our old PBX.

As a second example we could have migrated to our new PBX while retaining the old way (Comcast or Quest) of receiving our four dial tones.  In such a case we would have connected our four dial tones to the FXO ports on the new PBX.

But we made a plan to do two migrations at about the same time — migrating the PBX as well as migrating from analog to VOIP.  Which brings us to Figure 3.

What we did first was to power up the new PBX and deploy new SIP telephone extensions onto everybody’s desk.  So everybody had two phones on their desk — an old phone connected to the old PBX and a new phone connected to the new PBX.  We signed up for VOIP telephone service from a VOIP provider (we picked Voip.ms) which is VOIP Provider 1 in Figure 3.  We configured the outbound service with VOIP Provider 1 with the caller ID of our main telephone number (303-252-8800).  We configured the new PBX with a VOIP trunk that connects to VOIP Provider 1.

One important thing to learn about any Asterisk-based telephone system (such as this Grandstream system) is how to setup “outbound routes”.  An outbound route defines what happens when a user picks up a telephone extension and dials a telephone call.  Part of getting ready for phase 1 was to set up an outbound route so that an outbound telephone call would go out on the VOIP trunk to Voip.ms so that Voip.ms could complete the call.

The result was that you could pick up one of the new phones and dial an outgoing telephone call and the call would go out on the VOIP trunk through Voip.ms.  The called party would see our main telephone number on their caller ID.

Phase 1.  This permitted the first phase of our migration process.  Employees were instructed to use the old phones only for receiving telephone calls.  They were instructed to dial any outgoing telephone call on one of the new phones.

Phase 1 was important for several reasons.  First, it permitted us to reach an initial level of confidence that the new phones actually worked, and that the new PBX actually worked, and that the new telco (Voip.ms) actually worked.  It permitted us to reach a level of confidence that the VOIP trunk worked correctly.  It also permitted each user to arrive at some familiarity with the call quality.  Were the calls clear?  Was there background noise?  Did calls have annoying echos?  Did calls drop out or get cut off?  At all points during phase 1, if a particular employee were to encounter a problem with an outgoing call, the employee could if necessary hang up the new phone and place the same call on the old phone.  In doing so, the employee would use the old phone extension and the old PBX and the old telco, all of which were tried and true and familiar.

Things went well during phase 1.  Nobody found the need to give up and retry an outgoing call on an old phone.

During phase 1 we ordered up a DID (Direct inward dial) line from Voip.ms.   This was an arbitrary telephone number which we were going to use to test our ability to receive incoming calls at the new PBX.  This incurred a setup fee of 40¢ as well as a one-month service fee of 85¢, adding up to a cost of $1.25.  The $1.25 cost was well worth it.  This permitted us to gain experience with setting up what are called “inbound routes” in the PBX.  It is important, if you are using an Asterisk-based telephone system such as the Grandstream PBX, to learn how to set up inbound routes.  This test telephone number was part of this learning process.

During phase 1 we also set up telephone extension numbers and assigned them to people within the firm.  We set up a “ring group” which defines a group of particular telephone extensions, namely the employees whose job it is to answer incoming telephone calls.

During phase 1 we placed many, many test calls from cell phones to the test number.  This permitted our “ring group” employees to start learning how to answer a call, and how to transfer an incoming call to another person within the firm.

During phase 1 there were many more bits of progress.  For example, each employee configured his or her voice mail greetings for his or her voicemail mailbox on the new PBX.  People left voice mail messages for each other and gained experience with checking and clearing voice mail messages.

It will be appreciated that during phase 1 there were two phones on every desk and both PBXs were in service.  It is almost as if both Figures 2 and 3 were operating.  And to recap, incoming calls were coming in through the path of Figure 2 (Comcast to the old PBX to the old phone extensions).  Outgoing calls were going out on the path of Figure 3 (new phone extension to the new PBX to the VOIP trunk to Voip.ms).  In addition, many test calls were being placed to the new PBX on the path of Figure 3.

Phase 2.  The next important step was to move our analog telephone lines from the old PBX to the new PBX.  Recall that the Comcast ATA (“cable ATA” in Figures 2 and 3) has four FXS ports, and that as shown in Figure 2, there was a three-foot-long modular telephone cord connecting each FXS port to a corresponding FXO port on the old PBX.  For phase 2, we simply moved each of the four modular telephone cords from an FXO port on the old PBX to an FXO port on the new PBX.

For this to work, it was necessary to have already set up an “inbound route” on the new PBX so that the new PBX would know what to do with an incoming telephone call on any one of the FXO ports.  As I mentioned above, one of the important skills to develop when using an Asterisk-based system such as this Grandstream system is to be able to create such inbound routes.

One of the nice things about this move from phase 1 to phase 2 is that we got to pick when the move happened.  In our case we moved the phone cables from the old PBX to the new PBX in the evening after most of the employees had finished their work for the day.  We were then able to make test calls on our main telephone number 303-252-8800 to make sure the calls were being handled correctly in the new PBX.  If there had been some unexpected problem, we could have simply moved the four phone cables back to the old PBX and rescheduled phase 2 for some later date after figuring out how to solve the problem.

In our case the test calls worked fine.  Phase 2 had begun.

During phase 2, the situation was as shown in Figure 3.  Incoming calls to 303-252-8800 were coming in through Comcast and through the ATA and through modular phone cables to FXO ports on the new PBX.  The incoming call would go to the ring group and one of our employees in the ring group would answer the call and transfer it as needed to somebody else.

Outgoing calls would go out as described above.  A user at a telephone extension would dial an outgoing call and the new PBX would consult its outgoing route and would route the call through a VOIP trunk to Voip.ms.  The call would reach the called party, who would see the caller ID of 303-252-8800 on their phone.

Phase 2 continued for a few days, and we reached a level of confidence that incoming and outgoing phone calls were working as they should.  Call quality was excellent.

During phase 2, the old phones were still on employees’ desks and the old PBX was still powered on.  This was important because some people had lots of voice mail messages in their voice mail boxes on the old PBX.  Eventually each of our people had cleared out all of their voice mail messages on the old PBX.

It will be appreciated that each step carried out so far, including each step of phase 1 and each step of phase 2, was a step that happened at a time that we got to pick.  Each step was also capable of being “rolled back” if there had been some unexpected problem.  Throughout this process, incoming and outgoing phone service was uninterrupted.

Once we were at a level of comfort with the new PBX, it was time to get ready for porting the main telephone number 303-252-8800.  The main “getting ready” step was to configure an inbound route in the PBX so that it would know how to handle an incoming call on 303-252-8800 if the incoming call were to arrive on a VOIP trunk instead of arriving on one of the FXO ports.  It will be recalled that we had already gained some experience with inbound routes through the user of the test phone number.  This permitted us to be optimistic that we would successfully set up an inbound route that would work with respect to 303-252-8800 when the porting was complete.

At that point, we submitted the number porting request.  We sent detailed information about our Comcast telephone service (the soon-to-be “losing carrier”) to Voip.ms (the soon-to-be “gaining carrier”).  The gaining carrier sent an LSR to the losing carrier.  At that point we waited for the FOC to arrive.

As I say, we have seen some FOCs arrive a mere two business days after starting the port process.  But in this case, two weeks passed with no news about the porting process.

Finally at about 11AM on Thursday, June 29, we got an email from the gaining carrier telling us that our FOC date was, yes, June 29.  We dialed our main number 303-252-8800 and the call came through on one of the FXO ports of the new PBX.  (In other words, the call was still passing through Comcast.)  We kept dialing test calls to that number every few minutes, from cell phones.  Each call came through on one of the FXO ports.  At about noon things changed — a test call got a recorded message that the call could not be completed as dialed.  About two minutes later, the test call came through on the VOIP trunk.

At this point we were in phase 3.  In phase 3, all incoming and outgoing telephone calls were on the VOIP trunk.

It will be recalled that the NPAC database gets queried each time anybody dials a telephone call to any telephone number in the US.  What will be appreciated is that noon on June 29 is the time when the NPAC database got updated so that a call to the telephone number 303-252-8800 would get routed to Voip.ms instead of being routed to Comcast.  That is what moved us from phase 2 to phase 3.

There was almost no disruption of incoming calls due to the number porting process.  What made this possible was the use of a PBX having both FXO ports and VOIP trunk ports.  What made this possible was our having configured the PBX with incoming routes so as to handle correctly an incoming call on 303-252-8800 regardless of whether the call were to come in on an FXO port or on a VOIP trunk port.

We did not get to pick the day that the NPAC database would get updated.  We did not get to pick the time of day that the NPAC database would get updated.  We did not get much advance warning (only about an hour)  that the update was going to happen.  But the practical result was a lack of disruption for incoming calls, other than the aforementioned two-minute interval.

It will be appreciated that nothing about the number porting process presented any risk of disruption of outgoing phone service.  The outgoing phone calls had been going out on VOIP trunks for some days, with the caller ID of 303-252-8800 on the calls.  The update of the NPAC database did not affect this in any way.

An important step, now that the porting was complete, was to set up e911 service for outgoing telephone calls.  e911 service costs $1.50 per month.  It makes it so that if anyone in the office were to pick up an extension and dial 911, the call would reach the appropriate 911 service center and our office address would be transmitted to the 911 service center.  We set up the e911 service.

After a couple of days of stable telephone functions in phase 3, it was time to start winding things down with the old PBX.  We removed the old telephones from employees’ desks.  We removed the Comcast ATA and shipped it back to Comcast.  We removed the old PBX and made plans to sell the parts on eBay. We found a buyer, a company in Florida, for the old telephones and we found a buyer, a company in Georgia, for the old PBX.  The resulting situation is shown in Figure 4, which shows how things are now.

We just received our monthly bill from Comcast.  It includes continued monthly charges (about $160) for the four telephone numbers that were ported out on June 29.  We will have to contact Comcast to get the $160 recurring charge removed from our Comcast bill.  (We still use Comcast for Internet service.)

We just got our monthly bill from Voip.ms.  It includes the monthly fee for our telephone number 303-252-8800.  It is 85¢.