CODA-4582 - Open Issues for Investigation

[RogersDave]

*** This post was last edited May 2, 2017 ***

Good morning Community,

As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.

However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.

In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂

UDP Packet Loss

The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.

Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).

Bufferbloat

When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.

Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.

The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.

Update April 22: This problem seems resolved in firmware 2.0.10.27

5 GHz WiFi Low range for channels 36 to 48

Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.

Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.

Workaround: manually select higher channels (149-153-157-161)

Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27

Loss of OFDM Channel Lock

Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.

Update April 12: This problem is resolved in 2.0.10.26T2

List of connected device does not get fully populated

This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.

The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.

NAT Loopback not working for wired clients

When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.

Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)

LAN Counters not working

Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.

This problem has been reported to Hitron for investigation.

Unexpected modem reboot

Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.

Update April 12: This problem is resolved in 2.0.10.26T2

Missing SC-QAM Channels

After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.

Investigation continues with Hitron.

WiFi Survey

The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.

Guest Network

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless".  This has been reported in firmware 2.0.10.26T2.

Update April 22: This issue has been resolved in firmware 2.0.10.27

Update May 2: It seems this issue is not fully resolved and still experienced by some users

---

Future Planned Improvements

The following are items that we are working on in parallel of the above.

• Improvement in WiFi speeds
• Improvement in latency / bufferbloat

Dave

*Edited Labels*

[lethalsniper]

Yea I find rogers and there modems dont work well with ea games especially NHL 17 eashl, lag and slow it got worse now with the coda modem .. wasn't rogers and ea supposed to have a fix by now?

[Datalink]

Something isn't making sense.  I think the vast majority of gamers with the 4582 are seeing a reduction in latency.  I know that Dave indicated much earlier that your CMTS required attention, but, I don't know if anything has been done yet.  In terms of latency with EA, I saw a comment, by @RogersKevin I think, indicating that they were looking at peering directly with EA.  Again, this takes time.  I've found the latency to the CMTS has dropped with DOCSIS 3.1 running, and that's before I test to a target anywhere, so that latency reduction affects everything.  My guess would be that any latency to the CMTS might be a big part in what you're seeing, but that's speculation at this point. 

'

[lethalsniper]

No not yet. From what I know they will start fixing this issue by August , by splitting the node in building and replacing some wires from what I know. And I will do some pingplotter test tonight

[Telek]

Hi @RogersDave,

So I've been playing around with DFS and it appears that one of the two ranges does not work.

Setting the range to 100-104-108-112 does indeed put the frequency there. Some of my devices refuse to see it, but that's a reception problem. My Nexus 6P gets 126.1/51.6. However, using my AC3200U at 149-153-157-161 I get 217.7/52.4 on the same device.  Laptops show similar speed difference.

The problem is that setting the range on the CODA to 116-120-124-128 actually activates it in the high range at 149-153-157-161.

[RogersDave]

---

@Telek wrote:

 The problem is that setting the range on the CODA to 116-120-124-128 actually activates it in the high range at 149-153-157-161.

---

This *may* be normal operation. Because these are DFS bands, if the modem detects radar transmission, it has to move away from the band.

I will run a few tests to see if I can replicate including in an RF chamber.

Dave

[RyzenFX]

@RogersDave I was reading the notes for 2.0.10.30 and it mentioned that it includes support for band steering. Can band steering be enabled through modem the interface, or can it only be enabled through your end?

[Triple_Helix]

@RogersDave is there any truth to the rumors of another new modem to replace the CODA-4582?

Just wondering!

TIA,

Daniel 

[RogersDave]

---

@RyzenFX wrote:

@RogersDave I was reading the notes for 2.0.10.30 and it mentioned that it includes support for band steering. Can band steering be enabled through modem the interface, or can it only be enabled through your end?

---

In 2.0.10.30, we have moved the buttons for band-steering and DFS in the customer interface. These will show up under a new "Advanced" tab in the wireless menu. They are both disabled by default but now customers will have the option to enable them.

---

@Triple_Helix wrote:

@RogersDave is there any truth to the rumors of another new modem to replace the CODA-4582?

Just wondering!

TIA,

Daniel 

---

We did announce that we would be launching a new gateway in 2017 in our financial report. That's all I can tell.

[dallasdinatake]

Software Version 2.0.10.30T1

[Kkksss]

Hello,

Recently my router was updated to the 2.0.10.29 firmware, and it seemed to have returned some of the problems i had before i had the 2.0.10.27 firmware. Noticeably higher ping times and latency in games like League of Legends, Overwatch, Team Fortress 2, etc. As well as noticeable packet loss. So for reference Overwatch on 2.0.10.27 getting 30-35ms, and 2.0.10.29 getting 50-60ms. And as well on league 25ms -> 40-50ms respectively. Wondering if that is an issue for some other users of this router who are gaming.

[jgamm]

ya my lags back

[MegnaGaming]

Yeah, I posted this under another thread here: http://communityforums.rogers.com/t5/Internet/Rogers-Hitron-CODA-4582-Hardware/m-p/398576/highlight/...

Basically noticed the same thing 😞

[MegnaGaming]

Update on this @RogersDave - I'll be doing a factory reset tonight to hopefully revert back to .27 - the latency issue continues to be a problem and it's significant enough that I need to revert. Hoping the next beta patch fixes this issue as it impacts how I make a living 😄

[shak007]

I am still having the issue with this. I picked up the modem just last week. Is there a resolution?

Software Version 2.0.10.27

Guest Network 

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless". 

[m-red]

Nest Equipment

I have been experiencing issues with my Nest Thermostat, Nest Protect and Nest Cam with intermittent connection to my Coda 4582 router since early May. Everything was working great before that. After spending 3-4 hours trying to resolve this problem with Nest and Rogers support and experimentation with various router settings, has not solved the the issue.

I can only suspect that the new firmware is the culprit.

I would appreciate it Rogers could add this to the list of issues to investigate.

[Mythen]

https://www.google.com/get/videoqualityreport/?v=76mN0Zl5mEg

Anyone tell me what this means.  Getting major burring watching youtube this evening

Mythen

[Max24]

I've been having ongoing speed issues since June 23rd when my modem stopped being able to lock all 32 downstream channels with "RCS Partial Service" being listed in the modem logs. The first Rogers rep had opened a ticket with the maintenance team to diagnose the issue. The 2nd rep I spoke with said that I required a tech visit, he came by and tested all the signals coming into the house as well as the signal going to the modem and said the signals were perfect and the modem needed to be replaced.

After replacing the modem the "RCS Partial Service" error went away but I am only receiving 400-500Mbps Download and 3Mbps upload with a ping of 5-25ms using various speedtest.net Toronto servers. I phoned back in after replacing the modem and the 3rd rep said that the signal looks perfect and that he said a  network ticket needed to be created to diagnose this issue since everyone else is the area is not experiencing this issue.

Modem DOCSIS WAN Stats:

IP Address   xxxxxx
Subnet Mask    255.255.252.0
Gateway IP Address    7.1.200.1
DHCP Lease Time    😧 07 H: 00 M: 00 S: 00
Downstream Overview
Port ID    Frequency (MHz)    Modulation    Signal strength (dBmV)    Channel ID    Signal noise ratio (dB)
1    591000000    256QAM    1.600    7    38.605
2    855000000    256QAM    -2.200    3    36.610
3    861000000    256QAM    -3.400    4    35.780
4    579000000    256QAM    1.700    5    38.983
5    585000000    256QAM    1.300    6    38.605
6    849000000    256QAM    -1.500    2    36.610
7    597000000    256QAM    1.100    8    38.605
8    603000000    256QAM    1.100    9    37.356
9    609000000    256QAM    1.400    10    38.605
10    615000000    256QAM    1.300    11    38.605
11    621000000    256QAM    1.400    12    37.636
12    633000000    256QAM    1.900    13    37.636
13    639000000    256QAM    2.300    14    38.605
14    645000000    256QAM    2.500    15    37.636
15    651000000    256QAM    2.700    16    37.636
16    657000000    256QAM    3.000    17    38.605
17    663000000    256QAM    3.300    18    38.605
18    669000000    256QAM    3.400    19    38.605
19    675000000    256QAM    3.600    20    38.605
20    681000000    256QAM    3.300    21    37.636
21    687000000    256QAM    3.400    22    38.605
22    693000000    256QAM    3.300    23    38.605
23    699000000    256QAM    3.400    24    37.636
24    705000000    256QAM    3.400    25    38.605
25    711000000    256QAM    3.400    26    38.983
26    717000000    256QAM    3.200    27    38.983
27    723000000    256QAM    2.900    28    38.605
28    825000000    256QAM    0.700    29    37.356
29    831000000    256QAM    0.500    30    37.356
30    837000000    256QAM    0.000    31    37.356
31    843000000    256QAM    -0.600    32    36.610
32    303000000    256QAM    3.700    1    38.983
OFDM Downstream Overview
Receiver    FFT type    Subcarr 0 Frequency(MHz)    PLC locked    NCP locked    MDC1 locked    PLC power(dBmv)
0    4K    275600000    YES    YES    YES    3.799999
1    NA    NA    NO    NO    NO    NA
Upstream Overview
Port ID    Frequency (MHz)    Modulation    Signal strength (dBmV)    Channel ID    Bandwidth
1    38595824    ATDMA - 64QAM    39.000    3    3200000
2    30596000    ATDMA - 64QAM    35.250    1    6400000
3    23700000    ATDMA - 64QAM    34.500    2    6400000
OFDM/OFDMA Overview
Channel Index    State    lin Digital Att    Digital Att    BW (sc's*fft)    Report Power     Report Power1_6    FFT Size
0    DISABLED    0.5000    0.0000    0.0000    -inf    -1.0000    4K
1    DISABLED    0.5000    0.0000    0.0000    -inf    -1.0000    4K

DOCSIS Event:

1    01/01/1970 00:02:06    84000500    critical    SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=xxxxxx0;CMTS-MAC=00:17:10:00:ec:6f;CM-QOS=1.1;CM-VER=3.1;
2    01/01/1970 00:01:10    82000200    critical    No Ranging Response received - T3 time-out;CM-MAC=xxxxx0;CMTS-MAC=xxxxa;CM-QOS=1.1;CM-VER=3.1;
3    07/03/2017 20:24:03    90000000    warning    MIMO Event MIMO: Stored MIMO=-1 post cfg file MIMO=-1;CM-MAC=xxxx0;CMTS-MAC=xxxx;CM-QOS=1.1;CM-VER=3.1;
4    07/03/2017 20:24:13    69010200    notice    SW Download INIT - Via Config file bac108000106ac202e04ac10
5    07/03/2017 20:25:03    69011200    notice    SW download Successful - Via Config file
6    07/03/2017 20:33:30    90000000    warning    MIMO Event MIMO: Stored MIMO=-1 post cfg file MIMO=-1;CM-MAC=xxx;CMTS-MAC=xxx2a;CM-QOS=1.1;CM-VER=3.1;
7    07/03/2017 20:55:04    82000200    critical    No Ranging Response received - T3 time-out;CM-MACxxxxx0;CMTS-MAC=0xxxx;CM-QOS=1.1;CM-VER=3.1;
8    07/03/2017 20:58:43    68010300    error    DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=xxxx0;CMTS-MAC=xxxx;CM-QOS=1.1;CM-VER=3.1;
9    07/03/2017 21:56:05    82000200    critical    No Ranging Response received - T3 time-out;CM-MACxxxxxx;CMTS-MAC=xxx;CM-QOS=1.1;CM-VER=3.1;

Speedtests:

Teksavvy Server

Rogers Server

Beanfield Metroconnect

DSLReports

Ping is also very unstable while gaming, modem is in bridge mode on Gigabit plan.

[Alex4161]

@Max24

I had issues with modems and speed in the past and here are some suggestions I can pass along:

Verify your speed profile:

Verify that you are on the correct speed profile.  You can send a private message to  @CommunityHelps and they can confirm that you are on the proper profile.  No amount of speed tests will give you the right speeds if you are not setup correctly.  Please keep in mind that in the early stages of the Gigabit deployment, many areas were just provisioned to 500 Mbps speeds. When the area was confirmed for Gigabit speeds, most accounts were automatically bumped up to 1 Gbps but some customers profiles were not upgraded due to various factors.

Try These Speed Test Sites:

Try running your speed tests from the following servers:

Rogers:  http://speedcheck.rogers.com/en.html

Digital Ocean:  http://speedtest-tor1.digitalocean.com/

Make sure you are testing with a WIRED CAT 6 Cable

I am not sure if you are using a wired Cat 6 cable or wifi as this makes a HUGE difference.  Most AC based wifi adapters would max out around 550 Mbps in real world tests so your current speeds are not too far from this value.

Check your network card settings:

This article which deals with MS SQL Server Optimization has several suggestions that one can use to tweak the settings on their network card:  https://msdn.microsoft.com/en-us/library/cc558565(v=bts.10).aspx

The points that I would look at adjusting are the following.  Please note that you should be comfortable making these changes and some of the options may not be present for your network card.

Power Option – Configure the network adapter driver to prevent power management functionality from turning off the network adapter to save power. This functionality may be useful for client computers but should seldom, if ever, be used on a BizTalk Server or SQL Server computer.

Fixed Speed/Duplex (do not use AUTO) - It is very important that the network speed, duplex, and flow control parameters are set to correspond to the settings on the switch to which they are connected. This will mitigate the occurrence of periodic “auto-synchronization” which may temporarily take connections off-line.

Max Coalesce Buffers - Map registers are system resources used to convert physical addresses to virtual addresses for network adapters that support bus mastering. Coalesce buffers are available to the network driver if the driver runs out of map registers. Set this value as high as possible for maximum performance. On servers with limited physical memory, this may have a negative impact as coalesce buffers consume system memory. On most systems however, the maximum setting can be applied without significantly reducing available memory.

Max Transmit/Send Descriptors and Send Buffers - This setting specifies how many transmit control buffers the driver allocates for use by the network interface. This directly reflects the number of outstanding packets the driver can have in its “send” queue. Set this value as high as possible for maximum performance. On servers with limited physical memory, this may have a negative impact as send buffers consume system memory. On most systems however, the maximum setting can be applied without significantly reducing available memory.

Max Receive Buffers - This setting specifies the amount of memory buffer used by the network interface driver when copying data to the protocol memory. It is normally set by default to a relatively low value. Set this value as high as possible for maximum performance. On servers with limited physical memory, this may have a negative impact as receive buffers consume system memory. On most systems however, the maximum setting can be applied without significantly reducing available memory.

All offload options ON - In almost all cases performance is improved when enabling network interface offload features. Some network adapters provide separate parameters to enable or disable offloading for send and receive traffic. Offloading tasks from the CPU to the network adapter can help lower CPU usage on the server which will improve overall system performance. The Microsoft TCP/IP transport can offload one or more of the following tasks to a network adapter that has the appropriate capabilities:

Wake On LAN disabled (unless being used) – Configure the network adapter driver to disable wake-on lan functionality. This functionality may be useful for client computers but should seldom if ever be used on a BizTalk Server or SQL Server computer.

Registry Tweaks

If you are comfortable with registry tweaks, TCP optimizer 4 can be helpful to get you to squeeze the maximum performance from your network card.  Again, this is only recommended to those who know what they are getting into and is not something I can suggest to the avg. user.

https://www.speedguide.net/downloads.php

As an aside, I had my 9 year old desktop PC running an Q9300 CPU and a Gigbit network adapter not getting past 300 Mbps in speed tests.  After performing the above suggestions, I can get close to 900 Mbps.  

Hope this helps

Thanks

[Telek]

Great post, but one small clarification:

@Alex4161 wrote:

Make sure you are testing with a WIRED CAT 6 Cable

---

Being on WiFi absolutely will kill your performance, good point, but the CAT 6 isn't necessary -- CAT 5e works perfectly fine for gigabit.

It's strange that you had an old computer that only got 300mbit, pretty much any computer should, out of the box, be able to get gigabit speeds (or close to it) on their gigabit port. I have a laptop from 2005 which, OOTB Windows 7, gets the full ~930mbit on speedtest (when my internet connection permits) and 120MB/sec sustained copy between computers (from SSD to SSD). It was true back in Windows XP days that tweaks were needed, especially for poor drivers or cards, but these days OOTB everything should be fine.

One easy way to test, if you have multiple devices, is to just copy a file from one computer to another. If you get near gigabit speeds there, you can skip all of that tweaking as the problem lies elsewhere. You can also run the speed tests on the other computer and compare.

I had an issue before on my new laptop where there is built-in QoS software which auto-detected my connection speed at 250mbit, and thus limited my internet speed to the same. It was very frustrating for a while as I was SURE that Rogers had me on the wrong profile (since I could computer-to-computer copy files at gigabit speeds), only to find from another computer that the internet speeds were full. Took me a while to find the QoS software setting.

[Max24]

@Alex4161

Thanks for the detailed response. I've sent a message to @CommunityHelps to verify that I am on the correct speed profile. I should mention that this issue only recently started happening and before June 23rd I was able to get max speed consistently throughout the day averaging: ~900-950Mbps Download and 40-53Mbps Upload speeds. So it sounds to me that they've decided to throttle my connection since no other customers in the area have reported any speed issues or anything related.

The Digital Ocean results were 643.62Mbps Download and 4.07Mbps Upload using the TOR1 server. I don't use flash player anymore so I've skipped the rogers speed test. As for your network optimization tweaks I've already implemented them when I upgraded my computer in January but thank you.

[MegnaGaming]

Just another update @RogersDave: It appears that reverting back to .27 has had no impact on improving my latency issue. I'll continue to troubleshoot on my end but it seems this isn't related 😞

Feel free to toss me back onto the .29 if you're able.