CODA-4582 - Open Issues for Investigation

[RogersDave]

*** This post was last edited May 2, 2017 ***

Good morning Community,

As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.

However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.

In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂

UDP Packet Loss

The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.

Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).

Bufferbloat

When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.

Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.

The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.

Update April 22: This problem seems resolved in firmware 2.0.10.27

5 GHz WiFi Low range for channels 36 to 48

Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.

Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.

Workaround: manually select higher channels (149-153-157-161)

Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27

Loss of OFDM Channel Lock

Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.

Update April 12: This problem is resolved in 2.0.10.26T2

List of connected device does not get fully populated

This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.

The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.

NAT Loopback not working for wired clients

When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.

Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)

LAN Counters not working

Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.

This problem has been reported to Hitron for investigation.

Unexpected modem reboot

Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.

Update April 12: This problem is resolved in 2.0.10.26T2

Missing SC-QAM Channels

After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.

Investigation continues with Hitron.

WiFi Survey

The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.

Guest Network

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless".  This has been reported in firmware 2.0.10.26T2.

Update April 22: This issue has been resolved in firmware 2.0.10.27

Update May 2: It seems this issue is not fully resolved and still experienced by some users

---

Future Planned Improvements

The following are items that we are working on in parallel of the above.

• Improvement in WiFi speeds
• Improvement in latency / bufferbloat

Dave

*Edited Labels*

[lethalsniper]

Still waiting to hear from @RogersDave why I keep on losing wireless 5g

[lethalsniper]

That surfboard modem was the best I ever had when it came to gaming all these new so call tech are horrible for gaming

[JohnBeaudin]

yup Gaming has been horrible with new modem series, I hope this get fixed been waiting years for that.. and aparently it won't be anytime soon 😞

[Datalink]

@Kdurantulla, here is the link for the chart that I forgot to add to my post:

http://mcsindex.com/

[Datalink]

@RyzenFX send a pm to @CommunityHelps and ask if your post from last night can be restored.  My guess is that it was nuked for some reason, but I doubt that there was a mod on duty at that time of the night. 

[RyzenFX]

@tonytoronto What you're describing could very well be the UDP packet loss issue. 

I'm just as tired as your are having to deal with this issue, and it would be great to have @RogersDave back so we can get more information about this issue. 

[Mythen]

@Datalink

Any ideas on how to deal with these frequent disconnects on my Pixel Xl.  Both my wife and I have one.  and it driving us both nuts.  Unwilling to buy a router.  

Any thoughts

Mythen

[JohnBeaudin]

Hopefully he returns next week, let's cross our fingers.. tomorrow is Monday.. we might have a chance.

[Datalink]

@Mythen are you able to disable IPV6 in the Pixels which would force them to use IPV4 only?

[RyzenFX]

@Datalink Just sent the PM to them. Anyways, the issue is purely UDP in nature, but I can't justify what is causing the issue. 

[Mythen]

@Datalink

Thats a very good question.  I will have to look around and see

[RyzenFX]

@Datalink The post was taken down by the spam filter, it's been approved and restored. Go to page 118 of this topic and scroll down until you can see my post again.

[dmbcanda]

Been having these speed issues for a week and have had a ticket open and no issue has been resolved.   Was told after 3 days to switch modem, which did not help and the ticket is still open and network team is investigating.   I just want the net to work:

Via Ethernet connected in Gateway, double black dot Coda-4582 on .24.

[SFShakeel]

I am facing serious issues with WIFI 5GHZ and sometimes with 2.4GHZ bands. 5Ghz is keep disconnecting in every minutes. I tried different setting but nothing change. I spoke with Tech support it was useless as well since they have no idea what regarding this issue. I have replaced this modem 3 times. First in Dec 2016 without any DOT on back, which I replaced with one DOT on back, now with two DOT's on back.  Modem with single DOT was working fine until it receved firmware 2.0.10.24

I am just wondering if anyone here is facing same problem and have any clue regarding this issue. I really wish I can use my own HITRON (small modem) with my personal router but can't as Rogers doesn't allow. I can't put two big router at the same time due to space issues.

[JohnBeaudin]

Any mods here knows when RogersDave will be back in the forums?

[User1445713]

Hitron wireless hardware / setup is troublesome. It works for some hardware and has issues with other. I had original CGN3 modem shutting off radio completely with my new Dell XPS laptop most of the time within first minute of coming on line. Zero issues with old laptop and phones. I switched the modem to CODA. With new modem I see occasional loss of communication on wireless and most of the the time it is within first minute of laptop connection. Quite often after restore from hibernation it takes a long time for the laptop to find and log back into Hitron 5G signal.

You can connect any number of routers or access points to Hitron modem as wireless back up or additional channels. Connect to old router (this cold be old Rogers cable modem as well) with computer first. Disable DHCP and preassign static IP from the Hitron range to router itself (where 192.168.1.1 is). Make it something like 192.168.1.250. Now you can plug it in using any of 4 switch ports between Rogers Hitron and old router. Do not use WAN port. This will give you additional 3 ports like a small Ethernet switch and additional WiFi signals. You can log into spare router using preassigned IP address.

[MegnaGaming]

---

@Datalink wrote:

@MegnaGaming you have occasional packet loss on the first hop to the router, which is a little unusual.  You shouldn't see any packet loss to the router.  I would suggest running an extended test to the router only.  Use the routers address as the target and set the ping interval to 1 second and let that run for a day.  What I would like to see is an extended run to the router without any packet loss.  If the packet loss is still evident, then its time to go hunting for the cause...

---

Hey @Datalink,

Here's what happened during the 24 hour test to just the router: https://i.gyazo.com/1650636f3ae3b9c7361bc7677e45a7eb.png

It should be noted I shut down any usual heavy load processes (for example, turned off steam to avoid game updates) and Sunday's are also my days off - so no uploading. As you can see there is some packet loss this morning some time.

I'll be starting the extended test directly to the CMTS and post the results once I have them.

EDIT: Looks like the CMTS doesn't accept direct ICMP requests. Gives 100% packet loss regardless of the interval. Will ping Rogers for 24 hours and see what it gives.

[RabidHamster]

Wondering if anyone can help me here, or if I just have to wait for Rogers to somehow magically sort things out on their end.  

I got 250u service installed about 2 months ago, and speeds were great until about 2 weeks ago.  

I was pretty consistently getting 330/22, but in the last 2 weeks both download and upload speeds have dropped a lot.  About 2 weeks ago, download speeds started to drop, and it's been dropping pretty much daily since then, today it's anywhere from 50-100 mbps.  The upload has dropped from 22ish to well below 10, like 2-4 usually, and often (right now) it's basically zero, like <1 mbps.

I have a CODA black dot modem, .24 firmware (just swapped for a new one today, same results).  

I have the modem bridged to an Asus router, but I've obviously tried the CODA alone, CODA bridged, wired / wireless, rebooting, factory reset, etc., all the same results.  All the cables are screwed in tightly, my computer / phone work fine (I build computers for a hobby, I know this isn't the problem).  

Rogers support was no help (got me to swap the modem, but same results), they said my line stats look fine, and from what I've seen in these forums, they are fine.  Anybody have any ideas?  Pretty sure I've tried everything and for whatever reason the service just isn't working properly, which is weird because it was fine until recently.

Any suggestions would be appreciated!

 EDIT:  The only thing I noticed change on the line stats was the downstream PLC power.  I think this started off around -4 (4?) when DOCSIS 3.1 hit me, and it's been dropping, -2, -1.7, etc., now down to -0.7 the last few days.  Could this be a problem?

Downstream Overview

Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Signal noise ratio (dB)
1	591000000	256QAM	0.000	7	37.356
2	567000000	256QAM	-0.100	3	37.636
3	573000000	256QAM	0.100	4	37.356
4	579000000	256QAM	0.000	5	37.356
5	585000000	256QAM	-0.300	6	37.636
6	561000000	256QAM	0.400	2	37.356
7	597000000	256QAM	-0.900	8	37.356
8	603000000	256QAM	-1.200	9	37.356
9	609000000	256QAM	-0.900	10	37.356
10	615000000	256QAM	-1.400	11	36.610
11	621000000	256QAM	-0.900	12	36.610
12	633000000	256QAM	-1.000	13	36.610
13	639000000	256QAM	-0.500	14	36.610
14	645000000	256QAM	-1.100	15	36.387
15	651000000	256QAM	-0.900	16	36.387
16	657000000	256QAM	-1.400	17	36.610
17	663000000	256QAM	-2.100	18	36.387
18	669000000	256QAM	-1.500	19	36.387
19	675000000	256QAM	-1.900	20	36.387
20	681000000	256QAM	-1.200	21	36.610
21	687000000	256QAM	-1.300	22	36.610
22	693000000	256QAM	-0.900	23	36.610
23	699000000	256QAM	-1.700	24	36.610
24	705000000	256QAM	-2.000	25	36.610
25	711000000	256QAM	-2.800	26	36.387
26	717000000	256QAM	-2.700	27	36.387
27	723000000	256QAM	-2.500	28	36.610
28	825000000	256QAM	-3.900	29	35.595
29	831000000	256QAM	-3.300	30	36.387
30	837000000	256QAM	-3.000	31	36.387
31	843000000	256QAM	-4.500	32	36.387
32	555000000	256QAM	0.400	1	37.356

OFDM Downstream Overview

Receiver	FFT type	Subcarr 0 Frequency(MHz)	PLC locked	NCP locked	MDC1 locked	PLC power(dBmv)
0	4K	275600000	YES	YES	YES	-0.700001
1	NA	NA	NO	NO	NO	NA

Upstream Overview

Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Bandwidth
1	23700000	ATDMA - 64QAM	37.500	2	6400000
2	38595785	ATDMA - 64QAM	41.000	3	3200000
3	30596000	ATDMA - 64QAM	37.500	1	6400000

OFDM/OFDMA Overview

Channel Index	State	lin Digital Att	Digital Att	BW (sc's*fft)	Report Power	Report Power1_6	FFT Size
0	DISABLED	0.5000	0.0000	0.0000	-inf	-1.0000	4K
1	DISABLED	0.5000	0.0000	0.0000	-inf	-1.0000	4K

[Kdurantulla]

Ok. I See.

So lets say I have my CODA router and the 2nd floor and then add another router(connected wired) to the main floor. Would that give me a better range?.

If that works, Would I be able to connect a switch to the basement from the 2nd router?.

So pretty much my connection would look like this wired. MODEM > ROUTER > SWITCH.

And the CODA would cover the 2nd floor wifi and router would cover the main floor wifi.

Does this make sense?.

[jjunge]

Hi all, my area got DOCSIS 3.1 this morning and ever since it switched over my CODA has bee Lin completely unstable. It reboots constantly, looses sync and sometimes loses 2/3 upstream channels until I reboot. Any idea?

[Double_K]

@RogersDave  Significant issues today with packet loss & loss of service;

12	03/07/2017 18:43:55	82000200	critical	No Ranging Response received - T3 time-out;CM-MAC=a8:4e:3f:xx:yy:zz;CMTS-MAC=00:17:10:xx:yy:zz;CM-QOS=1.1;CM-VER=3.1;
13	03/07/2017 18:43:55	82000300	critical	Ranging Request Retries exhausted;CM-MAC=a8:4e:3f:xx:yy:zz;CMTS-MAC=00:17:10:xx:yy:zz;CM-QOS=1.1;CM-VER=3.1;
14	03/07/2017 18:43:55	82000600	critical	Unicast Maintenance Ranging attempted - No response - Retries exhausted;CM-MAC=a8:4e:3f:xx:yy:zz;CMTS-MAC=00:17:10:xx:yy:zz;CM-QOS=1.1;CM-VER=3.1;
15	03/07/2017 18:44:21	82000200	critical	No Ranging Response received - T3 time-out;CM-MAC=a8:4e:3f:xx:yy:zz;CMTS-MAC=00:17:10:xx:yy:zz;CM-QOS=1.1;CM-VER=3.1;
16	03/07/2017 18:44:22	82000300	critical	Ranging Request Retries exhausted;CM-MAC=a8:4e:3f:xx:yy:zz;CMTS-MAC=00:17:10:xx:yy:zz;CM-QOS=1.1;CM-VER=3.1;
17	03/07/2017 18:44:22	82000600	critical	Unicast Maintenance Ranging attempted - No response - Retries exhausted;CM-MAC=a8:4e:3f:xx:yy:zz;CMTS-MAC=00:17:10:xx:yy:zz;CM-QOS=1.1;CM-VER=3.1;
18	03/07/2017 18:44:47	82000400	critical	Received Response to Broadcast Maintenance Request, But no Unicast Maintenance opportunities received - T4 time out;CM-MAC=a8:4e:3f:xx:yy:zz;CMTS-MAC=00:17:10:xx:yy:zz;CM-QOS=1.1;CM-VER=3.1;
19	03/07/2017 18:46:02	82000200	critical	No Ranging Response received - T3 time-out;CM-MAC=a8:4e:3f:xx:yy:zz;CMTS-MAC=00:17:10:xx:yy:zz;CM-QOS=1.1;CM-VER=3.1;
20	03/07/2017 18:46:43	90000000	warning	        MIMO Event MIMO: Stored MIMO=1 post cfg file MIMO=-1;CM-MAC=a8:4e:3f:xx:yy:zz;CMTS-MAC=00:17:10:xx:yy:zz;CM-QOS=1.1;CM-VER=3.1;

Experiencing significant ICMP latency & loss

Mar 7 15:49:48	dpinger		WAN_DHCP 99.230.80.1: Clear latency 38634us stddev 92694us loss 5%
Mar 7 15:49:18	dpinger		WAN_DHCP 99.230.80.1: Alarm latency 33299us stddev 67410us loss 6%
Mar 7 15:49:14	dpinger		WAN_DHCP 99.230.80.1: Clear latency 32907us stddev 66562us loss 5%
Mar 7 15:48:56	dpinger		WAN_DHCP 99.230.80.1: Alarm latency 26093us stddev 20114us loss 6%
Mar 7 13:48:01	dpinger		WAN_DHCP 99.230.80.1: Clear latency 17751us stddev 27425us loss 0%
Mar 7 13:47:51	dpinger		WAN_Rogers_IPv6 2001:4860:4860::8888: Clear latency 33715us stddev 6782us loss 5%
Mar 7 13:44:13	dpinger		WAN_Rogers_IPv6 2001:4860:4860::8888: Alarm latency 33871us stddev 8442us loss 21%
Mar 7 13:44:05	dpinger		WAN_DHCP 99.230.80.1: Alarm latency 12728us stddev 5006us loss 6%
Mar 7 04:38:44	dpinger		WAN_Rogers_IPv6 2001:4860:4860::8888: Clear latency 29128us stddev 2286us loss 5%
Mar 7 04:37:34	dpinger		WAN_Rogers_IPv6 2001:4860:4860::8888: Alarm latency 32699us stddev 2386us loss 21%
Mar 7 04:30:20	dpinger		WAN_Rogers_IPv6 2001:4860:4860::8888: Clear latency 34023us stddev 3550us loss 5%
Mar 7 04:29:14	dpinger		WAN_Rogers_IPv6 2001:4860:4860::8888: Alarm latency 28918us stddev 1804us loss 22%
Mar 7 04:19:34	dpinger		WAN_Rogers_IPv6 2001:4860:4860::8888: Clear latency 30095us stddev 2874us loss 5%
Mar 7 04:18:26	dpinger		WAN_Rogers_IPv6 2001:4860:4860::8888: Alarm latency 29159us stddev 2172us loss 21%
Mar 7 04:14:37	dpinger		WAN_Rogers_IPv6 2001:4860:4860::8888: Clear latency 28918us stddev 1870us loss 5%
Mar 7 04:13:28	dpinger		WAN_Rogers_IPv6 2001:4860:4860::8888: Alarm latency 28730us stddev 1747us loss 22%

Also experiencing UDP latency & packet loss on VoIP RTP packets;  Max Jitter 739 Lost Packets 109 Min MOS 2.6867

CODA-4582 Original Launch hardware, version 2.0.10.24