Avoid The Finger Pointing
2 Ways to prove the problem is Non-Avaya related
When installing or servicing customers, there will be times when you won’t be able to solve the problems on your own. This will lead to you having to engage other tiers, such as IT Engineers, Service Providers, 3rd Party Software Application Service Providers, and lastly, the equipment Manufacturer.
In this post, “Avoid The Finger Pointing – 2 Ways to Prove the Problem is Non-Avaya Related, I will walk you through simple ways to troubleshoot them. To help you deal with these type of unsolved problems, I have developed these two steps:
- 1.- Data Collection
- 2.- Vendor Meet
1.- Data Collection
In order to get to the bottom of the existing problem, you need to understand what’s broken.
First, you need to replicate the problem, verify the system configuration, make adjustments, and then test.
Replicate the problem – Have the customer demonstrate what the issues are. If this issue happens to be an intermittent problem, you need to test until it happens again. In the case of dropped calls, call quality, or dialing issues, you might have to have the system logging in traces.
Trace Logs – In the case of Avaya Aura, you can run MST traces in conjunction with Wireshark to help you understand the problem.
For IP Office systems, use SSA logging functionality to be able to save realtime data to a file, as well as a Monitor trace, and finally Wireshark.
What to record – Have the customer record the time and date of the problem, making it easier for you to go back to your logged files and compare the recorded data. Have the customer provide you with telephone number dialed or Caller ID of the calling number, time of the incident, and date of when it happened. You can be as detailed as gathering the person’s extension number, agent-ID, Skill, Department, etc.
System Configuration – After analyzing the traces, you might find some fixing to do. If these issues are related to dialing, trunking, network, manufacture, or service provider, follow these steps.
Dial-Related Issues – By looking at the traces, you should see the calls dropped, before hitting the Service Provider. This could be related to a COR, COS, ARS, or Route Pattern issue. For IP Office this could be something related to a User SC, User Rights, ARS, or System Short Codes.
- Look for the the Denial Events, Alarms, and Errors.
- SSA Logs displaying the dialed numbers to verify they are getting passed through the Provider
Trunking-Related Issues – When using DS1s, perform maintenance board tests, as well as loop-back test to verify that the hardware is working properly. Then, run tests between the CSU/DSU and the DS1 Card. Lastly, check the the T1 Smart Jack LEDs.
SIP Trunking – Ask for a network analysis test. This test will help you identify if the pipe is getting saturated.
H.323 Trunking – use Wireshark to log the RTP traffic.
Network-Related Issues – Check with the IT Department if there are any Network Policies in place, and Access lists enabled. Ask for a VLAN Trunk Queue Analysis, that displays counters and errors. Lastly, compare the TPC/UDP ports allowed and disallowed in the network.
Manufacturer-Related Issues – It is very important to followup with the Avaya Product Correction Notices (PCNs) for each product line, to keep you informed of known issues. As a good practice, escalate the issue with Avaya, if problem persist.
Service Provider-Related Issues – If the configuration is working as designed, and hardware is up to date with the software version and release, it is time to contact the service provider. Ask for a list of protocols, including DS1/SIP or Network Settings to compare them with your running configuration.
2.- Vendor Meet
Now that you have ruled out the Avaya system, it is your responsibility to schedule an in-person meeting with the Local Exchange Carrier or ISP.
What should you have them check?
For DS1 – have them check the Smart Jack and Fiber Links. If the T1 is extended, then have them verify the extension. They can run loop-back testing back to their Facilities to test L1.
Framing Errors and Slips – These are normally related to Clocking Sources. If using multiple T1s in the system, check that the correct Timing Source is configured for the correct DS1 Card.
Ascending/Descending – Configure your trunk group hunt-sequence to be the opposite from the Provider’s Configuration, e.g. if LEC configures to send calls in channel 1, then use Channel 23 as the first option for outgoing calls.
IAD Routers – Normally, these will share the voice and data traffic, converting Digital B-Channels to IP, using a payload of around 80K with G.711U/A causing a network to congest quicker, depending on the amount of calls made at one time, versus the size of the trunk. This can easily cause voice degradation, and dropped calls.
Have the provider look at Wireshark traces to see when the event happens. They can also change the payload size to 30k by applying G.729A to the conversion per call. This will increase the bandwidth utilized by the voice system.
For SIP/H.323 – Confirm that they have QoS configured in their network voice packet prioritization. Have them check their Policies, TCP/UPD ports. It is always a good idea to provide them with the range configured in the Avaya PBX.
Lastly, if all of the above is verified, have them run Wireshark traces in their IAD/Router to see if there is an abnormal activity.
I personally like to replace the parts in questions before getting anyone else involved, to ensure that the problem is not hardware-related.
Please note: I reserve the right to delete comments that are offensive or off-topic.