UCCX 8.5(1) SU3 database corruption during upgrade

When upgrading Unified Contact Center Express to version 8.5(1) SU3 it might happen that the database gets corrupted. The root cause is most likely because the switch version command has been initiated by the engineer while the upgrade was still processing (hey we’re network engineers, we loose our patience once in a while) . This might happen during other upgrades in UCCX 8 upgrades as well, but Cisco has now documented this. After this issue occurs one might think it is safe to use the switch version command once more to revert back to the backup partition. This however is extremely riskful because with each version switch a backup is made of the database. It may happen that during the switch version the good database is overwritten by the corrupt which is even worse. Starting UCCX 8.5(1) SU3 Cisco has changed the switch version command code, so that from this version on the database is first checked for corruption before copying it over.
Cisco has an excellent wiki post called Recovering from database corruption during switch version to 8.5(1) SU3 that explains what best to do if you find yourself in such a situation. If you’re not a complete masochist you might want to follow this guide if you’re ever in this situation.

Cisco IP Phone Reset to Factory Defaults

It tends to happen that a Cisco IP Phone can not succesfully upgrade to a new version, most likely due to versions being signed differently or a corrupted previous upgrade. The easiest way to fix this kind of issue is to perform a reset to factory defaults. To perform such a factory reset on a Cisco IP Phone (at least for 79xx series) there are two methods as described below. For both methods, make sure DHCP is correctly configured, network connection to the TFTP server is available and the correct firmware is loaded on the TFTP server. If not, the phone might become useless after the reset procedure!

The first method is the default factory reset, which I also like to call the soft reset. It will revert the phone back to factory default settings and will remove the current loaded firmware.

  1. Power off the phone
  2. Press and hold the # key
  3. Apply power to the phone while still holding the # key
  4. Wait some seconds untill the line LEDs start flashing one at a time in a loop, then release the # key
  5. Type the sequence 123456789*0#
  6. The phone will reset and start downloading the firmware

The second method is a more aggressive factory reset, which I also like to call the hard reset. It will remove all configuration and firmware. During this procedure the screen will become completely blank, but in the logs of the TFTP server you will notice it does download the bootloader and firmware. This procedure might take a long time, up to 20 minutes.

  1. Power off the phone
  2. Press and hold the # key
  3. Apply power to the phone while still holding the # key
  4. Wait some seconds untill the line LEDs start flashing one at a time in a loop, then release the # key
  5. Type the sequence 3491672850*#
  6. The phone will reset and start downloading the firmware, the screen will be completely blank for quite some time

CUBE call feature issues

Recently I had some bad experiences with call features and CUBE to connect with provider SIP trunks. The issue was that when a fair amount of calls were passing through, call features such as hold, transfer, group pickup etc. wouldn’t work correctly. We experienced a one-way audio stream while attempting those. If only a handful of calls were made, these call features went fine, except for the blind transfer which didn’t work at all. After troubleshooting a long time and having a TAC case open we didn’t get any wiser. At first we though it might have to do with the early offer or CUCM not passing mid-call SDP. Eventually I decided to upgrade the CUBEs and apparantly that solved all these issues.

Installed version: c2900-universalk9-mz.SPA.151-4.M.bin
Upgraded version: c2900-universalk9-mz.SPA.152-2.T.bin

I’ve asked TAC if these symptons could be matched to a known bug but no answer yet.
A quick search with google shows a lot of issues with the 15.1.4 version and SIP calls. If you experience inexplicable behavior, you might want to check the version.

Rightfax10 SIP T.38

Recently I had some trouble with faxes using Rightfax 10 and T.38 over SIP after migrating from E1 ISDN to provider SIP trunks. The setup is as follows:

Service provider SIP Trunk CUBE CUCM Rightfax

The service provider offered us G.711 A-law, RFC 2833 as DTMF and T.38 as fax protocol (with fallback support for G.711 passthrough) on the SIP trunk. The Rightfax server was already in place and configured for T.38 using SIP directly with the old ISDN gateways. In the new setup we went for a more centrally managed dialplan and wanted the SIP trunk with Rightfax directly on CUCM. Rightfax was managed by a third party, but to say the least, they were not really helpful to us. I asked them to comply to the service provider requirements and they told me that wouldn’t be a problem. Eventually it appears all they did was change the IP adresses for the new routes and that’s about it.

To start off, I must admit that I had absolutely zero experience with Rightfax at first. After I got management access to Rightfax, I found that it was using the Dialogic Brooktrout SR140 as a virtual fax board. It has support for SIP, T.38 and G.711 A-law. RFC 2833 however is not supported.

First issue: with RFC 2833 not being supported incoming faxes didn’t get through due to the mismatch on CUCM between the CUBE SIP trunk and the Rightfax SIP trunk. This is not that uncommon and can be easily solved using an MTP. The only MTP supporting RFC 2833 and OOB conversions on T.38 though are the hardware MTPs. I had some DSPs left in CUBE, so I configured them as MTP and added them in a seperate MRGL for the Rightfax SIP trunk. This still doesn’t do the trick as an incoming call would try to dynamically insert an MTP at the first CUBE SIP trunk and not at the Rightfax SIP trunk. The CUBE SIP trunk has an MRGL with the default CUCM MTPs, I didn’t want to change this because there are other calls requiring the regular MTPs (contact center express for instance). Therefor I checked the MTP required checkbox at the Rightfax SIP trunk so that an incoming call would actually make use of the hardware MTPs at the Rightfax SIP trunk side to deal with the mismatch. Solved

Second issue: outgoing faxes were still causing problems. Apparantly rightfax advertises both G.711 A-law as ยต-law. Initially I still believed that the third party had done everything they needed to do. Therefor I even experimented with transcoders. However after gaining management access on the Rightfax server, the real solution is to set the brooktrout configuration so that only A-law gets advertised. This can be done in Rightfax Enterprise Fax Manager. Stop the RightFax DocTransport service and right click it to open up the configuration.

rightfax-t38-01.jpg

 

Go to the settings of the SR140 and click on Configure Brooktrout.

rightfax-t38-02.jpg

 

Go to advanced mode

rightfax-t38-03.jpg

 

Go to SIP RTP paramenters and change the RTP codec list to “pcma” instead of “pcma pcmu”.

rightfax-t38-04.jpg

 

Third issue: some outgoing faxes went fine, some failed. Incoming faxes went fine all the time. It turned out that for certain outgoing faxes no T.38 switchover was initiated and therefor no T.38 was used and the fax failed. Why did some outgoing faxes work? Well for certain numbers the service provider was initiating the T.38 switchover, however for certain other numbers the service provider didn’t initiate it. Rightfax did not initiate any T.38 for outgoing faxes. I made the following changes to Rightfax.

First of all I enabled G.711 passthrough as fallback mechanism, since the service provider has passthrough as a fallback mechanism, Rightfax can be set to use this fallback as well.
Secondly I changed the Media Renegotiate Delay Outbound from -1, which means Rightfax does not initiate T.38 but waits for the other side to initiate it, to 2000 which means that Rightfax initiates T.38 switchover after 2s.

rightfax-t38-05.jpg

 

That’s about it. I only needed to save the configuration and press apply. Afterwards I started the DocTransport service again and from now on all faxes went fine.

For the sake of completeness my configuration of SIP trunks on CUCM and the dial-peer configuration on CUBE. The only thing to keep an eye on, besides the MTPs, is that Rightfax only supports UDP for SIP.

CUCM Rightfax Trunk configuration

rightfax-t38-06.jpg

 

 

rightfax-t38-07.jpg

CUCM Rightfax SIP Trunk Security Profile

rightfax-t38-08.jpg

 

 

CUCUM CUBE Trunk Configuration

rightfax-t38-09.jpg

 

CUCM CUBE SIP Trunk Security Profile

rightfax-t38-10.jpg

 

 

CUBE Dialpeer:

dial-peer voice 30001 voip
translation-profile outgoing SIPin
preference 1
destination-pattern [3-5]…
session protocol sipv2
session target ipv4:x.x.x.x
session transport udp
incoming called-number .
voice-class codec 200
dtmf-relay rtp-nte
fax-relay ecm disable
fax rate 14400
fax protocol t38 version 0 ls-redundancy 2 hs-redundancy 0 fallback pass-through g711alaw

no vad

ATA 186 Fax G.711 A-law passthrough

To configure the ATA 186 for fax G.711 A-law passthrough you need to browse to the web page of the ATA and change the following settings:

AudioMode
0x00150015
ConnectMode
0x90002400

These parameters are in hex and are obtained by changing bit values. More info and bit explanations can be found at http://www.cisco.com/en/US/docs/voice_ip_comm/cata/186_188/3_0/english/administration/guide/sccp/sccpaape.html

Register an IOS MTP

Just for my reference to quickly register an IOS MTP.

On the router:

voice-card 0
 dsp services dspfarm
!
dspfarm profile 1 mtp  
 codec g711alaw
 codec pass-through
 maximum sessions hardware 10
 maximum sessions software 10
 associate application SCCP
 no shut
!
sccp local GigabitEthernet0/0
sccp ccm x.x.x.x identifier 1 version 7.0 
sccp ccm y.y.y.y identifier 2 version 7.0 
sccp
sccp ccm group 1
 bind interface GigabitEthernet0/0
 associate ccm 1 priority 1
 associate ccm 2 priority 2
 associate profile 1 register MTP_TEST
 keepalive retries 5
 keepalive timeout 20
 switchover method immediate
 switchback method immediate
 switchback interval 5

On CUCM: go to Media Resources -> MTP and add a new one. Make sure that the name matches the name configured in the router, in this case MTP_TEST

ios-mtp