CODA-4582 - Open Issues for Investigation

[RogersDave]

*** This post was last edited May 2, 2017 ***

Good morning Community,

As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.

However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.

In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂

UDP Packet Loss

The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.

Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).

Bufferbloat

When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.

Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.

The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.

Update April 22: This problem seems resolved in firmware 2.0.10.27

5 GHz WiFi Low range for channels 36 to 48

Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.

Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.

Workaround: manually select higher channels (149-153-157-161)

Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27

Loss of OFDM Channel Lock

Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.

Update April 12: This problem is resolved in 2.0.10.26T2

List of connected device does not get fully populated

This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.

The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.

NAT Loopback not working for wired clients

When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.

Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)

LAN Counters not working

Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.

This problem has been reported to Hitron for investigation.

Unexpected modem reboot

Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.

Update April 12: This problem is resolved in 2.0.10.26T2

Missing SC-QAM Channels

After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.

Investigation continues with Hitron.

WiFi Survey

The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.

Guest Network

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless".  This has been reported in firmware 2.0.10.26T2.

Update April 22: This issue has been resolved in firmware 2.0.10.27

Update May 2: It seems this issue is not fully resolved and still experienced by some users

---

Future Planned Improvements

The following are items that we are working on in parallel of the above.

• Improvement in WiFi speeds
• Improvement in latency / bufferbloat

Dave

*Edited Labels*

[traghipp]

Thanks for the response. Yes, I have made sure I've selected the same server and I get the same differences in speed.

[nigelncwong]

Hi,

I am on 2.0.10.23 firmware.  Modem constantly needing to be rebooted to maintain connection.  This is especially when downloading heavily.

Is this modem being recalled?  How can I tell?

[Alex4161]

@traghipp

Try running a speedtest from this site to see how it fairs:

http://speedtest-tor1.digitalocean.com/

[timfolkins]

I have been using that one.

800+ after a reboot of the modem

15-25 before the reboot after only 13 hours.

Interestingly enough - it's only the download that suffers. The upload remains mostly normal. (edit)

This only happened after .23 --- .19 was fine for me.

I was better off on the 250 plan at this rate.

is @RogersDave out there to address this particular issue? Thanks in advance.

[RyzenFX]

It could take a while to get Response from Dave given the recent events of many modems being bricked. Rogers and Hitron engineers are probably working day and night out to determine the root cause of the modems bricking.

[gp-se]

---

@StephKitty wrote:

@RogersDave I'm on firmware 2.0.10.19 and lately i've noticed my speeds deteriorating throughout the day.  For example, i've gone from downloads of 800+Mbps and uploads of 30Mbps to about 30Mbps down and 5+Mbps up in the span of two days.

The only solution is to reboot the modem to get back the speeds.  What's the deal here?  I've spoken to a Rogers rep, and was told that all CODA-4582 modems (original and black dot versions) have been recalled.  This is the first i'm hearing of this.  Also was told that I can't just trade in my modem because there is no availability due to demand and the recall.  The only solution she could provide me was to have the "Rogers Engineers" push a firmware update to 2.0.10.23 which would fix the issues of the recall. I thought we were still testing 2.0.10.20??  Very confused. 

I have the original CODA-4582 non-black dot version on a Rogers Gigabit connection.  

---

can you login to the modem and post the Signal levels under the DOCSIS WAN page. There should be 32 channels down and the signal levels between -10 and +10, with 0 being Optimal.

Also the latest Beta firmware is .23, you can try having that pushed to your modem and see if it helps. Next there isn't a recall, however a small percentage of original modems had an issue with interference causing issues, the new "black dot" modems have extra shielding to solve that issue. But not all modems are affected, the easiest way to see if you are affected is to make sure you have 32 downstream channels on the DOCSIS WAN page.

[lethalsniper]

@RogersDave can I go back on .19 firmware ever since on .23 nothing but issues devices loosing connection websites just keep on loading buffering, also getting a error message on Xbox one about dns need to Reboot modem so please 23 is not working for me as of right now thanks.

[lethalsniper]

Getting lots of T3 timouts , MDD timeout ,and SYNC Timing Synchronization failure - Loss of Sync. @Datalink @RogersDave

[StephKitty]

@gp-se I'm not sure how to read it but here it is:

Downstream Overview

Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Signal noise ratio (dB)
1	609000000	256QAM	5.700	10	38.605
11	615000000	256QAM	6.000	11	38.605
12	621000000	256QAM	6.300	12	38.605
13	633000000	256QAM	5.600	13	38.605
14	639000000	256QAM	5.500	14	38.605
15	645000000	256QAM	5.500	15	38.605
16	651000000	256QAM	5.600	16	38.605
17	657000000	256QAM	5.400	17	38.605
18	663000000	256QAM	5.400	18	38.605
19	669000000	256QAM	5.300	19	38.605
20	675000000	256QAM	5.600	20	38.605
21	681000000	256QAM	5.500	21	38.605
22	687000000	256QAM	5.300	22	38.605
23	693000000	256QAM	5.200	23	37.636
24	699000000	256QAM	4.900	24	37.636
25	705000000	256QAM	4.400	25	37.636
26	711000000	256QAM	4.000	26	37.636
27	717000000	256QAM	3.900	27	37.636
28	723000000	256QAM	3.200	28	37.636
29	825000000	256QAM	6.700	29	38.983
30	831000000	256QAM	6.400	30	37.636
31	837000000	256QAM	6.000	31	37.356

OFDM Downstream Overview

Receiver	FFT type	Subcarr 0 Frequency(MHz)	PLC locked	NCP locked	MDC1 locked	PLC power(dBmv)
0	NA	NA	NO	NO	NO	NA
1	4K	275600000	YES	YES	YES	5.699997

Upstream Overview

Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Bandwidth
1	38596000	ATDMA - 16QAM	33.250	3	3200000
2	30596000	ATDMA - 16QAM	32.250	1	6400000
3	23700000	ATDMA - 16QAM	32.000	2	6400000

OFDM/OFDMA Overview

Channel Index	State	lin Digital Att	Digital Att	BW (sc's*fft)	Report Power	Report Power1_6	FFT Size
0	DISABLED	0.5000	0.0000	0.0000	-inf	-1.0000	4K
1	DISABLED	0.5000	0.0000	0.0000	-inf	-1.0000	4K

[lethalsniper]

Correction it was firmware .20 before .23 @RogersDave please read post above and let me know thanks.

[DaveFro]

---

@StephKitty wrote:

@gp-se I'm not sure how to read it but here it is:

Downstream Overview

Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Signal noise ratio (dB)
1	609000000	256QAM	5.700	10	38.605
11	615000000	256QAM	6.000	11	38.605
12	621000000	256QAM	6.300	12	38.605
13	633000000	256QAM	5.600	13	38.605
14	639000000	256QAM	5.500	14	38.605
15	645000000	256QAM	5.500	15	38.605
16	651000000	256QAM	5.600	16	38.605
17	657000000	256QAM	5.400	17	38.605
18	663000000	256QAM	5.400	18	38.605
19	669000000	256QAM	5.300	19	38.605
20	675000000	256QAM	5.600	20	38.605
21	681000000	256QAM	5.500	21	38.605
22	687000000	256QAM	5.300	22	38.605
23	693000000	256QAM	5.200	23	37.636
24	699000000	256QAM	4.900	24	37.636
25	705000000	256QAM	4.400	25	37.636
26	711000000	256QAM	4.000	26	37.636
27	717000000	256QAM	3.900	27	37.636
28	723000000	256QAM	3.200	28	37.636
29	825000000	256QAM	6.700	29	38.983
30	831000000	256QAM	6.400	30	37.636
31	837000000	256QAM	6.000	31	37.356

OFDM Downstream Overview

Receiver	FFT type	Subcarr 0 Frequency(MHz)	PLC locked	NCP locked	MDC1 locked	PLC power(dBmv)
0	NA	NA	NO	NO	NO	NA
1	4K	275600000	YES	YES	YES	5.699997

Upstream Overview

Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Bandwidth
1	38596000	ATDMA - 16QAM	33.250	3	3200000
2	30596000	ATDMA - 16QAM	32.250	1	6400000
3	23700000	ATDMA - 16QAM	32.000	2	6400000

OFDM/OFDMA Overview

Channel Index	State	lin Digital Att	Digital Att	BW (sc's*fft)	Report Power	Report Power1_6	FFT Size
0	DISABLED	0.5000	0.0000	0.0000	-inf	-1.0000	4K
1	DISABLED	0.5000	0.0000	0.0000	-inf	-1.0000	4K

---

Oh wow, you are missing most of your downstream channels!

I believe this is due to the 2.4 ghz interference issue on the "non-black-dot" units. The black dot units have extra shielding to prevent the 2.4 ghz signals from messing with the modem and causing it to drop channels. Without all your channels you will have terrible speed. I'll bet that when you first reboot, and your speed is good, you have all your channels 1-32, but after time and interference they start dropping. Check your channels when you first reboot, and then once the speeds slows down to compare.

Your signal levels however seem great, and you have docsis 3.1 active in your area.

For now you could try disabling 2.4 ghz radio on the modem, or if you are in bridge mode, you could try moving your wireless router way further away from your modem.

I'm sure you will be told to exchange your modem for the black dot version if you talk to tech support.

[markopas]

I am still on the .19 firmware and went in to go look at my status to see if there have been any f/w updates (none). I did notice the same issue as mentioned by StephKitty where I am now missing channels 2 - 10 (might even be more than that). But everything still works as it has in the past with no changes, I am still getting about 900-950 down and 30 up. Wireless is still ok, I mean 5GHz can always use some tweaking (which I tried over the weekend playing with the channels and bandwidth).

[bfanek]

@RogersDave

I would like to return this faulty CODA modem back.

I'm still having connection dropping issues every few hours, i have to reboot the modem 2-4 times DAILY.

This is unacceptable, and this modem is clearly faulty. It's essential that my connection *never* drops because i'm connecting remotely to my computer at all times.

I do not want to wait until you guys fix all these issues with this modem (if ever).

What are my options of using other modems? Could you please provide a list of supported modems?

I do not need wifi features as I'm bridging it with my own wifi router that works perfectly.

Thanks!

[RyzenFX]

If you plan to return the faulty modem, your options are the CGN3ACSMR or the CGNM-3552. If you're on Gigabit you should get the 3552 as it supports speeds up to 1.2Gbps. The CGN3ASCMR only supports 960Mbps  However, given that many people have bricked modems, and are constantly returning to stores to have them replaced, there's a good chance that the modems are out of stock. Before returning the modem, I suggest calling your local Rogers store to see if they have stock of the modems.

[Datalink]

@bfanek if that modem doesn't have a black dot near the power cord, then it might make sense to swap it for one that does, as that black dot indicates that the Docsis tuner has additional shielding installed to prevent interference from nearby 2.4 Ghz wifi transmitters. 

1.  Is that one of the original CODA-4582s without the black dot?

2.  If so, try the following experiment if the modem and router (with 2.4 Ghz operating) are close together: 

     a.  separate the two by several feet so that the router can't influence the modem; or

     b.  shut down the 2.4 Ghz on the router

In both cases, reboot the modem after either one of those actions.

Are you connecting to the computer via VPN?  Just curious at this point as I know the CGNM-3552 would crash and reboot after two to three hours of continuous VPN usage.  That might be driven entirely upon the type of VPN and protocol used.  I don't know if the same applies to this modem as I haven't had the opportunity to look for that scenario.

Depending on your internet plan, you would be eligible for another modem model.  The local Rogers store could tell you what model goes with what plan.  The one problem in this case is whether or not you're on a gigabit plan.  I don't know if there are restrictions in place for that plan to use the CODA-4582 only, or if it will allow the CGNM-3552.  Maybe one of the mods can answer that question.

The only problem with swapping modems at the moment is borking of several hundred/thousand (?) modems over the weekend due to some process that Rogers was running.  As a result, modems are in high demand and short supply, at least temporarily.  So, before you head to a store to swap modems, call first to see if any are in stock. 

[StephKitty]

@gp-se Thanks for the help.  I didn't realize that I was missing so many of these channels.  A Rogers tech came over about an hour ago to swap out the modem and make sure my signals were okay.

I now have the Black Dot version of the modem from Rogers which seems to have fixed the issue so far.  One thing I did notice though is that my firmware is only 2.0.10.13 and has not been updated to the latest firmware it should be on which i believe is 2.0.10.19.  Not a huge deal I guess since it's working so far, i'll just wait for it to be pushed to the latest firmware eventually.  

[StephKitty]

---

@Datalink wrote:

The only problem with swapping modems at the moment is borking of several hundred/thousand (?) modems over the weekend due to some process that Rogers was running. 

---

What do you mean by Borking?  Like they messed up several hundred/thousand modems over the weekend?  

Would this possibly be because of the "IPV6" fix they had planned?

[Datalink]

Don't know.  Whatever was running, it looks like it definitely didn't go as planned.

[arrago]

you think we will get native ipv6 again soon?

[StephKitty]

---

@arrago wrote:

you think we will get native ipv6 again soon?

---

@arrago From what i've heard, looks like something messed up the native IPV6 in firmware 2.0.10.13 so they've disabled it for the time being.   The problem HAS been fixed in firmware 2.0.10.19 but since people are still getting the upgraded modem at firmware 2.0.10.13, they decided to wait. Once all CODA-4582 modems have been updated to the latest confirmed firmware, they can finally enable native IPV6 on our routers.

When that will be, who knows.  Last update I saw from @RogersDave was about a month ago, and it was noted on the very first post that they were working on something within a couple of weeks.  

[bfanek]

@Datalink

Thank you kindly for your reply.

I'm using the black dot version, and 2.4Ghz is already shut down as i'm using it in Bridge mode.

I'm not connected to a VPN, and on the 1gbit plan. I will definitely swap it with one of those modems if available and hopefully have these issues resolved.

cheers