01-05-2017 11:03 AM - edited 05-02-2017 07:09 AM
*** This post was last edited May 2, 2017 ***
Good morning Community,
As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.
However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.
In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂
UDP Packet Loss
The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.
Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).
Bufferbloat
When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.
Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.
The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.
Update April 22: This problem seems resolved in firmware 2.0.10.27
5 GHz WiFi Low range for channels 36 to 48
Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.
Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.
Workaround: manually select higher channels (149-153-157-161)
Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27
Loss of OFDM Channel Lock
Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.
Update April 12: This problem is resolved in 2.0.10.26T2
List of connected device does not get fully populated
This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.
The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.
NAT Loopback not working for wired clients
When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.
Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)
LAN Counters not working
Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.
This problem has been reported to Hitron for investigation.
Unexpected modem reboot
Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.
Update April 12: This problem is resolved in 2.0.10.26T2
Missing SC-QAM Channels
After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.
Investigation continues with Hitron.
WiFi Survey
The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.
Guest Network
When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless". This has been reported in firmware 2.0.10.26T2.
Update April 22: This issue has been resolved in firmware 2.0.10.27
Update May 2: It seems this issue is not fully resolved and still experienced by some users
Future Planned Improvements
The following are items that we are working on in parallel of the above.
Dave
*Edited Labels*
02-02-2017 10:13 PM - edited 02-02-2017 10:16 PM
Yupppppp. Multiple to no effect.
This seems to be somewhat intermittent, although without a pattern. Internet was fine this morning for example, although parents were complaining of TV issues earlier.
(sorry, didn't see the second paragraph) I'm in bridge mode, no wifi enabled.
(FYI, they're getting certain channels either not displaying audio/video, or being heavily artifacted/garbled, and in general the respone and channel load being slower, so it's not just my modem, apparently)
02-02-2017 10:17 PM - edited 02-02-2017 10:21 PM
Ok, in that case call tech support and advise the CSR that your tv service is suffering from pixelation and audio issues. That should result in a faster response in terms of getting a tech out to your home. No doubt that there is some issue that also affects the internet modem as well.
Edit: what router model are you running? There is an issue with D-Link routers that is causing major issues. You could try an experiment and that is to connect a pc directly to the modem with the modem in Bridge mode. Restart the pc so that it picks up the correct IP address and then run a check at ipv6-test.com just to ensure that IPV4 is up and running correctly. After that run a speedtest at www.speedtest.net with the Toronto or Montreal Rogers servers. Only point to remember is that the pc has to rely on its own firewall while its connected to the modem. That check will hopefully determine if the problem is modem related only or if there is an issue with connecting to the router.
02-02-2017 10:30 PM
Router is an RT-N66U. I'm not concerned about it, as I'm not expecting to get maxed out speed atm, but 1mbit isn't reasonable.
02-02-2017 10:35 PM - edited 02-02-2017 10:36 PM
02-02-2017 10:37 PM
Oh yeah. Speed tests were all over the place between 350-800 mbit, but download using a good torrent was solid at around ~38MB/s before CPU max. Now, that is but a dream~
02-02-2017 10:42 PM
02-02-2017 11:33 PM
So i spoke to support about my issue of losing channel 13 over about a days time and the speeds going to a crawl, and then latency and packetloss happening.
Had them do a signal check, and check the CMTS and my neighbours. CMTS and neighbours looked fine they said i was dropping almost all the packets they were sending my modem and it appeared to be a bad modem or a damaged line.
Rebooted the modem and they tested again and they said it looks perfect. Support was at a loss to say what it is but said i could exchange it. Seeing as it is looking like a firmware quark since others on .23 are having the issue. I'll hang in there and wate for our Hero @RogersDave to have a look when he actually gets a spare moment. Hear he has been working non stop on other things.
As a note i am not on DOCSIS 3.1 in my area, I will probably be the exact last Rogers customer to get it since my town is tiny lol
02-02-2017 11:46 PM - edited 02-02-2017 11:47 PM
Considering that you rebooted the modem and it behaved as it should, that kind of points to a memory leak of some kind from some firmware task. Keep an eye on the performance and whether or not a reboot gets the modem back into operation again. Is the modem running in Gateway or Bridge mode, and, do you run a VPN for extended periods of time at all?
02-02-2017 11:51 PM - edited 02-02-2017 11:55 PM
@Datalink It's in bridged mode, And this has been an ongoing liek clock work issue for the past about week. Reboot always fixes it and then in about a days time its back to being ase useless as a wet paper bag of hammers.
I do run VPN connections for a good period of time for work. I use an open VPN connection and SSH tunnels/session back to work, and sometimes from work to home when i need to test something externally or grab files from my home machines.
I've also been using my OpenVPN sessions in TCP mode because UDP has been well unreliable for the past weeks.
EDIT: I just had a thought, the first day i noticed this issue was In the morning. I had fallen asleep and left my OVPN session open all night. and the internet was crying uncle.
02-02-2017 11:58 PM - edited 02-03-2017 12:00 AM
The modem has issues with UDP transfer, no doubt about that. Running the CGNM-3552 in Bridge mode with a VPN running and running a ping test, I ended up with ping timeouts that increased in frequency until the modem crashed and restarted. I suspect that the 4582 may have inherited some issues with VPNs that date back much further to previous modem models and their firmware. I doubt that its been properly identified. I wonder what would happen if you weren't using a VPN. I haven't had any issue so far with my 4582 running in Bridge mode, but, I know that there's an issue with UDP.
Edit: Keep an eye on your VPN usage and see if there is any correlation to the poor modem performance.
02-03-2017 12:02 AM
@Datalink wrote:The modem has issues with UDP transfer, no doubt about that. Running the CGNM-3552 in Bridge mode with a VPN running and running a ping test, I ended up with ping timeouts that increased in frequency until the modem crashed and restarted. I suspect that the 4582 may have inherited some issues with VPNs that date back much further to previous modem models and their firmware. I doubt that its been properly identified. I wonder what would happen if you weren't using a VPN. I haven't had any issue so far with my 4582 running in Bridge mode, but, I know that there's an issue with UDP.
I noticed those issues with VPN and ping timeouts, I reported that to dave and sent him some ping plotsand logs, he noted it and I know he was looking into it. When he isnt so busy, I'll ask him if he ever found what was up with that. As I know exactly what your talking about in that aspect i have personally seen it as well.
UDP also as you know we spoke and ran some tests and produced pretty like results on that area of issue. I can probably lay off the OPEN vpn for a bit and just use ssh and see how that works out. But ultimatly I'd hope this is resolved as VPN was this terrible of an issue for me before .23
02-03-2017 11:14 AM
Hello,
Does anyone here get latency spikes playing League of Legends? I contacted Rogers about this but my modem is in spec as well as there are no issues in the area. I keep getting spikes upwards to 80ms and at times 100+. My friends also on Rogers never spike at all so I don't seem to understand why I am getting it. I do play on ethernet. I even tried forwarding ports but it seems like it made the situation worse. I miss playing around 28ms everyday but I have no clue what changed.
02-03-2017 11:36 AM
It's called Hiltron, especially since the CGN3 series , since then we suffer from the same issues with lag and udp, the coda 4582 have the same UDP problems as well.. so for now we can only wait.
02-03-2017 11:43 AM
02-03-2017 12:18 PM
I wonder if it can really be fixed..
02-03-2017 12:20 PM
@crazyshot what modem do you have, as seen by the product sticker at the back of the modem? It should be one of CGN3xxxx, CGNM-3552 or the new CODA-4582 (large white modem).
02-03-2017 12:21 PM
Is there an ETA for this firmware update?
02-03-2017 12:21 PM
02-03-2017 12:22 PM - edited 02-03-2017 12:39 PM
Ok, for the record, this modem doesn't have the same latency issues that the other Hitron modems have, at least not for IPV4 ICMP and TCP/IP. UDP is an open question as I don't have access to a UDP server for testing. There is UDP packet loss with this modem that can be observed in a UDP transfer test. UDP is used for DNS lookup, gaming and dedicated data transfer types so its possible that you are experiencing both UDP latency and packet loss. Thats just an educated guess at this point. There are firmware updates rolling out on a continuous basis for this modem. I think we're up to the fourth or fifth update since its introduction in December, and there are more on the way. I don't know when Intel and Hitron will be targeting UDP issues, but its on the list. Maybe when @RogersDave isn't so busy, he can update the forum on the plan to tackle this specific item.
What you could do in the mean time is download and setup pingplotter or mutliping to keep an eye on the modem to CMTS path, looking for any instances of packet loss and looking at the latency that occurs during the day between the modem and CMTS. That alone might explain some of what you are seeing. You don't have to buy a licence for either as they will run in evaluation mode for 14(?) days. Pingplotter then kicks down to a more limited freebie mode, not sure about multiping.
Here's a post that indicates how to set that up:
02-03-2017 12:34 PM
02-03-2017 12:37 PM
Okay, that sounds great! Keep me posted