How to Troubleshoot QoS Issues in Enterprise Networks
A comprehensive guide to troubleshooting QoS issues in enterprise networks, covering systematic diagnosis approaches, common implementation gaps, and advanced diagnostic techniques for identifying and resolving QoS-related problems.
QoS troubleshooting in enterprise networks requires a systematic approach that goes beyond basic configuration verification. When voice calls drop, video conferences pixelate, or critical applications perform poorly despite seemingly correct QoS policies, the root cause often lies in implementation gaps, device limitations, or network design oversights that aren't immediately obvious.
Establishing a Troubleshooting Framework
Effective QoS troubleshooting starts with understanding the complete traffic path and policy inheritance. Begin by mapping the end-to-end flow, identifying every device that could potentially affect QoS treatment. This includes access switches, distribution layers, WAN edges, and any service provider hand-offs.
The most critical first step is verifying that QoS is actually enabled globally on each device. Many engineers assume QoS is active when policies are configured, but without mls qos on Catalyst switches or the equivalent platform commands, even perfect policies remain dormant.
Switch# show mls qos
QoS is enabled
QoS ip packet dscp rewrite is enabled
Switch# show mls qos interface gigabitethernet1/0/1
GigabitEthernet1/0/1
trust state: trust dscp
trust mode: trust dscp
trust enabled flag: ena
COS override: dis
default COS: 0Identifying Common QoS Implementation Gaps
The most frequent enterprise QoS issues stem from trust boundary violations and inconsistent marking policies. Access layer switches often trust all incoming DSCP markings by default, allowing end devices to potentially monopolize high-priority queues. Verify trust states on user-facing interfaces and implement proper trust boundaries at distribution layers where centralized policies can be enforced.
Another critical area is queue depth and scheduling verification. Default queue configurations rarely align with enterprise traffic patterns. Use show platform hardware fed switch active qos queue stats internal on newer Catalyst platforms to identify queue drops and buffer utilization patterns.
Switch# show platform hardware fed switch 1 qos queue stats internal
Pkts Bytes Drops
Port - Intf Queue Tx Tx Tx
----------------------------------------------------------
1/0/1 0 12458 1580416 0
1/0/1 1 0 0 0
1/0/1 2 5847 748416 124
1/0/1 3 89654 11477312 0Queue 2 drops in this output indicate congestion in what's typically the default data queue, suggesting inadequate bandwidth allocation or improper traffic classification.
WAN and Provider Integration Issues
Enterprise QoS troubleshooting becomes complex at WAN boundaries where internal policies must align with service provider offerings. Many providers remarking or stripping DSCP values at ingress points, effectively nullifying internal QoS efforts. Verify provider QoS support through interface statistics and consider implementing traffic contracts that specify marking preservation requirements.
For SD-WAN deployments, verify that underlay and overlay QoS policies are complementary rather than conflicting. SD-WAN devices often implement their own traffic shaping and prioritization, which can interfere with traditional QoS mechanisms if not properly coordinated.
Router# show policy-map interface tunnel100
Tunnel100
Service-policy output: SDWAN_QOS_POLICY
queue stats for all priority classes:
priority level 1
(total drops) 0
(bytes output) 15847264
Class-map: VOICE (match-any)
0 packets, 0 bytes
5 minute offered rate 0000 bps, drop rate 0000 bpsApplication-Specific Troubleshooting Techniques
Voice quality issues require specific attention to jitter, delay, and packet loss metrics. Use show call active voice brief and RTP statistics to correlate QoS policy effectiveness with actual voice quality measurements. Pay particular attention to interfaces showing consistent jitter above 30ms or packet loss exceeding 1%.
Video applications demand consistent bandwidth allocation and proper burst handling. Verify that video traffic classes have adequate committed information rates and that burst parameters accommodate video encoding variations. Many enterprises underestimate the bursty nature of modern video codecs, leading to periodic quality degradation despite average bandwidth availability.
Advanced Diagnostic Commands and Monitoring
Platform-specific commands provide deeper insight into QoS operation. On Nexus platforms, show queuing interface ethernet1/1 reveals per-queue statistics and threshold configurations. For ISR and ASR platforms, show platform hardware qfp active infrastructure bqs queue output default interface exposes queue depths and drop counters at the forwarding plane level.
Router# show platform hardware qfp active statistics drop
Global Drop Stats
----------------------------------------
Disabled 0
UnconfiguredInterface 0
Ipv4NoRoute 15
Ipv4NoAdj 0
QosDrops 847The QosDrops counter directly indicates traffic being dropped due to QoS policies, providing immediate confirmation that congestion management is active and potentially misconfigured.
Performance Validation and Continuous Monitoring
Effective QoS troubleshooting extends beyond fixing immediate issues to implementing ongoing monitoring that prevents future problems. Deploy network monitoring tools that can track queue utilization trends, application response times, and QoS policy effectiveness over time.
Establish baseline measurements for critical applications under normal and stressed conditions. This historical data proves invaluable when troubleshooting intermittent issues or validating the impact of configuration changes.
Consider implementing IP SLA probes that generate synthetic traffic matching your critical application profiles. These probes can provide early warning of QoS policy degradation before users experience service impact.
What's Next
Once you've mastered QoS troubleshooting fundamentals, the next logical progression involves implementing advanced QoS features like WRED congestion avoidance and traffic policing mechanisms. These topics build upon the diagnostic skills covered here and provide additional tools for managing network congestion in complex enterprise environments.
Tools and resources for this topic
- CCNP ENCOR 350-401 Official Cert Guide — The definitive ENCOR study resource by Brad Edgeworth. Covers enterprise infrastructure, virtualisation, and automation.