CODA-4582 - Open Issues for Investigation

[RogersDave]

*** This post was last edited May 2, 2017 ***

Good morning Community,

As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.

However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.

In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂

UDP Packet Loss

The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.

Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).

Bufferbloat

When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.

Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.

The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.

Update April 22: This problem seems resolved in firmware 2.0.10.27

5 GHz WiFi Low range for channels 36 to 48

Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.

Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.

Workaround: manually select higher channels (149-153-157-161)

Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27

Loss of OFDM Channel Lock

Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.

Update April 12: This problem is resolved in 2.0.10.26T2

List of connected device does not get fully populated

This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.

The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.

NAT Loopback not working for wired clients

When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.

Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)

LAN Counters not working

Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.

This problem has been reported to Hitron for investigation.

Unexpected modem reboot

Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.

Update April 12: This problem is resolved in 2.0.10.26T2

Missing SC-QAM Channels

After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.

Investigation continues with Hitron.

WiFi Survey

The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.

Guest Network

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless".  This has been reported in firmware 2.0.10.26T2.

Update April 22: This issue has been resolved in firmware 2.0.10.27

Update May 2: It seems this issue is not fully resolved and still experienced by some users

---

Future Planned Improvements

The following are items that we are working on in parallel of the above.

• Improvement in WiFi speeds
• Improvement in latency / bufferbloat

Dave

*Edited Labels*

[PBreau]

Rogers CODA-4582 Router TimeZone Settings Atlantic time is missing

I am trying to change my timezones on my router (CODA-4582) as I want to setup time restriction. when I go into the settings of the router and time zone there is no Atlantic Time Canada... Anyone ever encounter this ? 

Cheers

[Lurker]

---

@PBreau wrote:
Rogers CODA-4582 Router TimeZone Settings Atlantic time is missing

I am trying to change my timezones on my router (CODA-4582) as I want to setup time restriction. when I go into the settings of the router and time zone there is no Atlantic Time Canada... Anyone ever encounter this ? 

Cheers

---

Atlantic Time on the Hitron is listed as:

UTC-4:00 Bolivia, Venezuela, Atlantic Time

[PBreau]

Thanks for the reply it worked! Cheers

[sewilson]

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless".  I have firmware 2.0.10.28T2. 

How can I get this resolved?

[Notchy]

When connecting to the Guest Network, I get an error message displayed "only allow DHCP client to use this wireless".  I have firmware 2.0.10.28T2. 

Is there a work around to have my guest have access to my wifi?

[gp-se]

---

@Notchy wrote:

When connecting to the Guest Network, I get an error message displayed "only allow DHCP client to use this wireless".  I have firmware 2.0.10.28T2. 

Is there a work around to have my guest have access to my wifi?

---

@Notchy @sewilson

Where are you guys getting this error message? On the Laptop, on a mobile phone?

What Operating System are the devices using? We need more info to properly diagnose this. 

[sewilson]

I have been getting this error on all of the following devices.

• Laptop - Windows 10 Home (version 1703)
• Tablet - Android (Version 7.0)
• iPod Touch - iOS (Version 10.3.3)

[Notchy]

@sewilson 

I get the error on the following 

- Laptop (windows 10)

- Desktop PC (Windows 10)

- iphone 5s (latest iOS)

- ipad ( this was my guest's device)

- Android phone

[RyzenFX]

@Notchy @sewilson 

That's because firmware 2.0.10.28T2 doesn't include the fix for the GuestSSID not working properly. The latest trial firmware (2.0.10.31) contains the fix, but if I were you I would wait until it is released network-wide to avoid any extra issues with the trial firmware.

[sewilson]

Thanks for the reply. It seems this issue has been around for a few months now with the current network-wide firmware.

Do you know any ballpark time frame when firmware 2.0.10.31 is planned to be released network-wide? Are they nearing the end of the testing cycle?

[wes162]

I have a new Coda-4582 after swapping out an earlier one, and before that a CGN3.  The new box worked well for a week or two, but gradually i found that it was losing DL speed which would recover after a reboot.  UL speeds are unaffected.  Reboots to restore DL speeds were at first required only a few days apart, then daily, now it is hourly.  The problem occurs even with everything disconnected and wifi disabled.  Using only an old laptop plugged into the box's LAN port with a cat-5 cable and you get the same problem.  On the DOCSIS WAN status my signal strength is within spec on all 32 channels, but the OFDM Downstream Overview will show receiver 1 locked at 4K with power of approx. 2 dbMv (receiver 0 is NA) and after an hour or so, when speeds die off, the receiver 1 drops to NA like receiver 0.  If I swap out this box, I expect to see the same thing from the next one, since this is exactly the same story with the previous one.  Firmware is 2.0.10.28T2 which I am told is most recent.

Very frustrated here.  Front line support knows nothing about OFDM.  Its above their pay grade.  Finally convinced on to send out a "senior" tech on Sunday, but I doubt he'll understand this either because those guys are normally cable installers and not network engineers and clearly this is a complex issue since it has been ongoing for months.

[wes162]

I should clarify that the issue described in my recent post would appear to be the condition identified in the original thread, pasted below, but occurring with a more recent firmware version than 26T2...

Loss of OFDM Channel Lock

Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.

Update April 12: This problem is resolved in 2.0.10.26T2

[wes162]

I posted earlier about a loss of OFDM channel lock on Firmware 2.0.10.28T2

I discovered something further about this issue.  After a reboot, the box will authenticate, establish an OFDM channel lock (which i now suspect is what causes it displays the light blue download light), and run smoothly for a while.   After a short while, it loses its OFDM channel lock, and downloads run super slowly (but the box still displays the light blue download light).  Instead of rebooting again, I just disconnected it from the cable network, waited a few minutes, then reconnected it.  The box registers itself with Rogers again at that point, but now it only shows the dark blue download light, and runs without either OFDM receiver locked.    DL speeds are much higher than before the cable was pulled and reconnected and it appears to last, at least longer than after a reboot (so far).  It seems to me this suggests a problem with the modem on the Rogers side being unable to adapt to a customer modem losing its OFDM lock.  Perhaps the Rogers side modem is continuing to try to multiplex when the customer side modem lost its lock, after it registered with the network having locked multiplex.  Not a fix really, just a possible explanation?

[RogersDave]

@wes162,

The issue with loss of OFDM locks affects a very small number of modems but it is something we are investigating.

We may have a fix that I'd like to try on your modem.

Would you be able to send me a private message @RogersDave including your modem MAC address and serial number? I will run some checks and deploy the fix to you to see if it helps your situation.

--Dave

[Alex4161]

@RogersDave

One of my PC's seems to have issues getting an IP Address from the CODA modem.  From the logs, it looks like there were 2 firmware updates sent to my modem in the same day.

I was just wondering if this is a coincidence that I am having this issue after the firmware update.  Has anyone else noticed something weird getting connected to the CODA modem? 

Thanks

[8dmk]

Dave, 

I am am late to the party and wanted some updated advice on a problem I am having with 

Xbox One and EA NHL18.

It seems this is common, but I can connect to Multiplayer on EASHL but not on HUT .

i have Gigabit service with the white modem.

 Thanks 

[wes162]

@RogersDave, one additional observation I should probably mention is that I had similar symptoms with the previous Coda, i.e. High DL speeds after reboot which degraded later on.  I cannot be certain it was a loss of OFDM channel lock on that earlier modem because I was unaware of this and wasn't looking for it at the time, but it sure seems likely it was the same issue.  The support line had me swap it out and the problem went away with the new one.  Additionally, this box seemed to work flawlessly for a couple weeks before the issue arose.  In fact, it only started after I returned from a vacation and powered up the box which had been unplugged while I was away.  Given that two coda's now appear to have the same problem at my location it likely isn't limited to a small number of them. It may have something to do with conditions on my line or with the CMTS at the other end.

[lethalsniper]

U need to disable ipv6

[techninja]

openvpn results in 92% download speed reduction (40 Mbps vs 500 Mbps) 

Today is the first day I've had Rogers service and noticed that any of my devices connected via openvpn would drop in speed down to a max of 40 Mbps after a few minutes of usage. I'm on a CODA-4582 and without any VPN I tested getting downloads up to ~60 MB/s which is expected. 

Once connected to any openvpn server, the ones I use for work, personal ones on dedicated servers (1 Gbps uplink), speed tests show around 400 Mbps download which is to be expected with VPN overhead + encryption. Gradually download speeds dwindle to a maximum of 5 MB/s. These same servers on connected to from other networks do not have any extreme slow downs and the likelihood of multiple VPN servers at different data centres experiencing the same capacity issue would be a highly unlikely coincidence. Earlier in the day these servers were being utilized from a Teksavvy connection, with only your typical slowdown from VPN overhead.

This occurs regardless of device (desktop, laptop, phone), wired or wireless with any OS (Linux, macOS, Windows, Android) as well.

The same issue occurs when the modem is in bridged or residential gateway mode. Is there a problem with the CODA-4582 handling openvpn traffic? Is Rogers throttling openvpn connections? 

[nl2]

@techninja

Thanks for the sanity check. I am experiencing the same openvpn connection slowdown symptoms that you have described.

[techninja]

I was able to get one server to higher speeds by disabling the MTU=1500 option, which leads me to believe there is some kind of packet processing bug. Can you try disabling that option if you can and see if it makes any difference? However the problem seems to still occur intermittently.