cancel
Showing results for 
Search instead for 
Did you mean: 

CODA-4582 - Open Issues for Investigation

RogersDave
Retired Support
Retired Support

*** This post was last edited May 2, 2017 ***

 

Good morning Community,

 

As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.

 

However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.

 

In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂

 

UDP Packet Loss

The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.

 

Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).

 

 

Bufferbloat

When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.

 

Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.

 

The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.

 

Update April 22: This problem seems resolved in firmware 2.0.10.27

 

 

5 GHz WiFi Low range for channels 36 to 48

Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.

 

Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.

 

Workaround: manually select higher channels (149-153-157-161)

 

Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27

 

 

Loss of OFDM Channel Lock

Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.

 

Update April 12: This problem is resolved in 2.0.10.26T2

 

 

List of connected device does not get fully populated

This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.

 

The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.

 

 

NAT Loopback not working for wired clients

When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.

 

Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)

 

 

LAN Counters not working

Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.

 

This problem has been reported to Hitron for investigation.

 

 

Unexpected modem reboot

Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.

 

Update April 12: This problem is resolved in 2.0.10.26T2

 

 

Missing SC-QAM Channels

After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.

 

Investigation continues with Hitron.

 

 

WiFi Survey

The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.

 

 

Guest Network

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless".  This has been reported in firmware 2.0.10.26T2.

 

Update April 22: This issue has been resolved in firmware 2.0.10.27

Update May 2: It seems this issue is not fully resolved and still experienced by some users


 

Future Planned Improvements

The following are items that we are working on in parallel of the above.

  • Improvement in WiFi speeds
  • Improvement in latency / bufferbloat

 

 

Dave

 

*Edited Labels*

2,620 REPLIES 2,620

Re: CODA-4582 - Open Issues for Investigation

NBomb
I Plan to Stick Around

Yupppppp. Multiple to no effect.

This seems to be somewhat intermittent, although without a pattern. Internet was fine this morning for example, although parents were complaining of TV issues earlier.

 

(sorry, didn't see the second paragraph) I'm in bridge mode, no wifi enabled.

(FYI, they're getting certain channels either not displaying audio/video, or being heavily artifacted/garbled, and in general the respone and channel load being slower, so it's not just my modem, apparently)

Re: CODA-4582 - Open Issues for Investigation

Datalink
Resident Expert
Resident Expert

Ok, in that case call tech support and advise the CSR that your tv service is suffering from pixelation and audio issues.  That should result in a faster response in terms of getting a tech out to your home.  No doubt that there is some issue that also affects the internet modem as well. 

 

Edit:  what router model are you running?  There is an issue with D-Link routers that is causing major issues.  You could try an experiment and that is to connect a pc directly to the modem with the modem in Bridge mode.  Restart the pc so that it picks up the correct IP address and then run a check at ipv6-test.com just to ensure that IPV4 is up and running correctly.  After that run a speedtest at www.speedtest.net with the Toronto or Montreal Rogers servers. Only point to remember is that the pc has to rely on its own firewall while its connected to the modem.  That check will hopefully determine if the problem is modem related only or if there is an issue with connecting to the router.



Re: CODA-4582 - Open Issues for Investigation

NBomb
I Plan to Stick Around

Router is an RT-N66U. I'm not concerned about it, as I'm not expecting to get maxed out speed atm, but 1mbit isn't reasonable.

Re: CODA-4582 - Open Issues for Investigation

Datalink
Resident Expert
Resident Expert

Yup, agree with that.  I wouldn't expect any issues with the 66U.  Were you seeing higher speeds before, in the range that the 66U can handle?

 

Edit:  I would get a tech out to your home to check the cabling and connectors.



Re: CODA-4582 - Open Issues for Investigation

NBomb
I Plan to Stick Around

Oh yeah. Speed tests were all over the place between 350-800 mbit, but download using a good torrent was solid at around ~38MB/s before CPU max. Now, that is but a dream~

Re: CODA-4582 - Open Issues for Investigation

Datalink
Resident Expert
Resident Expert

Ok, get tech support to chase after it and then go from there when the tech has completed his or her work.



Re: CODA-4582 - Open Issues for Investigation

Jeffj
I Plan to Stick Around

So i spoke to support about my issue of losing channel 13 over about a days time and the speeds going to a crawl, and then latency and packetloss happening. 

Had them do a signal check, and check the CMTS and my neighbours. CMTS and neighbours looked fine they said i was dropping almost all the packets they were sending my modem and it appeared to be a bad modem  or a damaged line.

 

Rebooted the modem and they tested again and they said it looks perfect. Support was at a loss to say what it is but said i could exchange it. Seeing as it is looking like a firmware quark since others on .23 are having the issue. I'll hang in there and wate for our Hero @RogersDave to have a look when he actually gets a spare moment. Hear he has been working non stop on other things. 

As a note i am not on DOCSIS 3.1 in my area, I will probably be the exact last Rogers customer to get it since my town is tiny lol

Re: CODA-4582 - Open Issues for Investigation

Datalink
Resident Expert
Resident Expert

Considering that you rebooted the modem and it behaved as it should, that kind of points to a memory leak of some kind from some firmware task.  Keep an eye on the performance and whether or not a reboot gets the modem back into operation again.  Is the modem running in Gateway or Bridge mode, and, do you run a VPN for extended periods of time at all?



Re: CODA-4582 - Open Issues for Investigation

Jeffj
I Plan to Stick Around

@Datalink It's in bridged mode, And this has been an ongoing liek clock work issue for the past about week. Reboot always fixes it and then in about a days time its back to being ase useless as a wet paper bag of hammers.

 

I do run VPN connections for a good period of time for work. I use an open VPN connection and SSH tunnels/session back to work, and sometimes from work to home when i need to test something externally or grab files from my home machines. 

 

I've also been using my OpenVPN sessions in TCP mode because UDP has been well unreliable for the past weeks.

 

 

EDIT: I just had a thought, the first day i noticed this issue was In the morning. I had fallen asleep and left my OVPN session open all night. and the internet was crying uncle. 

Re: CODA-4582 - Open Issues for Investigation

Datalink
Resident Expert
Resident Expert

The modem has issues with UDP transfer, no doubt about that.  Running the CGNM-3552 in Bridge mode with a VPN running and running a ping test, I ended up with ping timeouts that increased in frequency until the modem crashed and restarted.  I suspect that the 4582 may have inherited some issues with VPNs that date back much further to previous modem models and their firmware.  I doubt that its been properly identified.  I wonder what would happen if you weren't using a VPN.  I haven't had any issue so far with my 4582 running in Bridge mode, but, I know that there's an issue with UDP. 

 

Edit:  Keep an eye on your VPN usage and see if there is any correlation to the poor modem performance.



Re: CODA-4582 - Open Issues for Investigation

Jeffj
I Plan to Stick Around

@Datalink wrote:

The modem has issues with UDP transfer, no doubt about that.  Running the CGNM-3552 in Bridge mode with a VPN running and running a ping test, I ended up with ping timeouts that increased in frequency until the modem crashed and restarted.  I suspect that the 4582 may have inherited some issues with VPNs that date back much further to previous modem models and their firmware.  I doubt that its been properly identified.  I wonder what would happen if you weren't using a VPN.  I haven't had any issue so far with my 4582 running in Bridge mode, but, I know that there's an issue with UDP. 


I noticed those issues with VPN and ping timeouts, I reported that to dave and sent him some ping plotsand logs, he noted it and I know he was looking into it. When he isnt so busy, I'll ask him if he ever found what was up with that. As I know exactly what your talking about in that aspect i have personally seen it as well.

 

UDP also as you know we spoke and ran some tests and produced pretty like results on that area of issue.  I can probably lay off the OPEN vpn for a bit and just use ssh and see how that works out. But ultimatly I'd hope this is resolved as VPN was this terrible of an issue for me before .23

Re: CODA-4582 - Open Issues for Investigation

crazyshot
I Plan to Stick Around

Hello,

 

Does anyone here get latency spikes playing League of Legends? I contacted Rogers about this but my modem is in spec as well as there are no issues in the area. I keep getting spikes upwards to 80ms and at times 100+. My friends also on Rogers never spike at all so I don't seem to understand why I am getting it. I do play on ethernet. I even tried forwarding ports but it seems like it made the situation worse. I miss playing around 28ms everyday but I have no clue what changed. 

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a Senior Contributor

@crazyshot

It's called Hiltron, especially since the CGN3 series , since then we suffer from the same issues with lag and udp, the coda 4582 have the same UDP problems as well.. so for now we can only wait.

Re: CODA-4582 - Open Issues for Investigation

Yea I'm on the coda and yea I getting the same lag spikes you are talking about and I play on the Xbox one hopefully there will be a firmware fix

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a Senior Contributor

I wonder if it can really be fixed..

Re: CODA-4582 - Open Issues for Investigation

@crazyshot what modem do you have, as seen by the product sticker at the back of the modem?  It should be one of CGN3xxxx, CGNM-3552 or the new CODA-4582 (large white modem).



Re: CODA-4582 - Open Issues for Investigation

crazyshot
I Plan to Stick Around

Is there an ETA for this firmware update?

Re: CODA-4582 - Open Issues for Investigation

crazyshot
I Plan to Stick Around
Hi, I use the CODA-4582.

Re: CODA-4582 - Open Issues for Investigation

Ok, for the record, this modem doesn't have the same latency issues that the other Hitron modems have, at least not for IPV4 ICMP and TCP/IP.  UDP is an open question as I don't have access to a UDP server for testing.  There is UDP packet loss with this modem that can be observed in a UDP transfer test.  UDP is used for DNS lookup, gaming and dedicated data transfer types so its possible that you are experiencing both UDP latency and packet loss.  Thats just an educated guess at this point.  There are firmware updates rolling out on a continuous basis for this modem.  I think we're up to the fourth or fifth update since its introduction in December, and there are more on the way.  I don't know when Intel and Hitron will be targeting UDP issues, but its on the list.  Maybe when @RogersDave isn't so busy, he can update the forum on the plan to tackle this specific item.

 

What you could do in the mean time is download and setup pingplotter or mutliping to keep an eye on the modem to CMTS path, looking for any instances of packet loss and looking at the latency that occurs during the day between the modem and CMTS.  That alone might explain some of what you are seeing.  You don't have to  buy a licence for either as they will run in evaluation mode for 14(?) days.  Pingplotter then kicks down to a more limited freebie mode, not sure about multiping. 

 

Here's a post that indicates how to set that up:

 

http://communityforums.rogers.com/t5/forums/forumtopicpage/board-id/Getting_connected/message-id/355...

 



Re: CODA-4582 - Open Issues for Investigation

RyzenFX
I'm a Reliable Contributor
@crazyshot I'm experiencing this too- but more on the packet loss side. In League, I'm losing 5-8 packets/min. This translates to 50% packet loss when hitting the server. @RogersDave told me that they have to report to Hitron for further testing. Also, Dave has opened a ticket to Intel about this b/c the issue was reproducible on the Intel Puma 7 base modem. We should see him address this issue next week when he is available.

Re: CODA-4582 - Open Issues for Investigation

crazyshot
I Plan to Stick Around

Okay, that sounds great! Keep me posted