CODA-4582 - Open Issues for Investigation

[RogersDave]

*** This post was last edited May 2, 2017 ***

Good morning Community,

As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.

However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.

In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂

UDP Packet Loss

The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.

Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).

Bufferbloat

When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.

Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.

The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.

Update April 22: This problem seems resolved in firmware 2.0.10.27

5 GHz WiFi Low range for channels 36 to 48

Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.

Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.

Workaround: manually select higher channels (149-153-157-161)

Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27

Loss of OFDM Channel Lock

Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.

Update April 12: This problem is resolved in 2.0.10.26T2

List of connected device does not get fully populated

This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.

The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.

NAT Loopback not working for wired clients

When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.

Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)

LAN Counters not working

Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.

This problem has been reported to Hitron for investigation.

Unexpected modem reboot

Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.

Update April 12: This problem is resolved in 2.0.10.26T2

Missing SC-QAM Channels

After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.

Investigation continues with Hitron.

WiFi Survey

The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.

Guest Network

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless".  This has been reported in firmware 2.0.10.26T2.

Update April 22: This issue has been resolved in firmware 2.0.10.27

Update May 2: It seems this issue is not fully resolved and still experienced by some users

---

Future Planned Improvements

The following are items that we are working on in parallel of the above.

• Improvement in WiFi speeds
• Improvement in latency / bufferbloat

Dave

*Edited Labels*

[prateeck7]

@RogersDavePlease update me to .24 as the tech swapped the modem in my house. I PM'd you with the MAC address. Thank you.

[JohnBeaudin]

@RogersDave

I tested the .24 firmware ( Latency buildup over time)

Joined a server with 62 ms, was stable for a few minutes, then it started to slowly build over time reached 74 ms at the end of the game, so still doing the same behavior unfortunately.

[traghipp]

Rogers tech support told me that I've been upgraded to docsis 3.1 as of today but in my DOCSWAN overview, under OFDM/OFDMA Overview, both channel 0 and 1 are disabled. Wouldn't one of these be enabled if I was on docsis 3.1?

Thank you

[Jeffj]

---

@JohnBeaudin wrote:

@RogersDave

I tested the .24 firmware ( Latency buildup over time)

Joined a server with 62 ms, was stable for a few minutes, then it started to slowly build over time reached 74 ms at the end of the game, so still doing the same behavior unfortunately.

---

62ms to 74ms isnt exactly a build up. It could have more to do with the servers load on the other end. Pings in an online session can be affected by others who have joined the session as well.  

The build up that I was experiencing and have not isnce was it get to the point where id be consistently stuck in 100+ pings with spikes to 400-500ms when to the CMTS.

[Hybrid_Noodle]

Yes @traghipp one of those channels should be showin as enabled. You may need to do a hard reset of the modem for it to pick up the 3.1 change. The other possibility is that the Rogers tech had incorrect information.

[JohnBeaudin]

Yes I am aware it's not a mega build up.

But my neighbor on bell can join the same server and his ping will stay a constant 28 ms for the whole hour until match end, but meand my friends on rogers our ping will fluctuate and experience a slow build up ( nothing huge) but usually 10-15 ms more ms at the end of the match vs when the match start.

Please note it's 5vs5 so no more players join, it can stay 62 ms for a few rounds and then over time it start to increase a bit until it usually reach 72-74 ms ( That's on Chicago or New york location of course)

[VivienM]

---

@JohnBeaudin wrote:

Yes I am aware it's not a mega build up.

But my neighbor on bell can join the same server and his ping will stay a constant 28 ms for the whole hour until match end, but meand my friends on rogers our ping will fluctuate and experience a slow build up ( nothing huge) but usually 10-15 ms more ms at the end of the match vs when the match start.

Please note it's 5vs5 so no more players join, it can stay 62 ms for a few rounds and then over time it start to increase a bit until it usually reach 72-74 ms ( That's on Chicago or New york location of course)

---

Am I correctly remembering from another post that you are in NB?

[JohnBeaudin]

@VivienM

Yes I am in North-East NB

[VivienM]

---

@JohnBeaudin wrote:

@VivienM

Yes I am in North-East NB

---

And that means that, as RogersDave said to you earlier, distance/physics is going against you.

Unless things have changed (and the absence of reverse DNS on Rogers' routers makes it hard to tell), the Rogers IP network is very very Toronto-centric. To reach, say, NYC, your packets could easily end up doing NB -> Toronto -> Chicago -> NYC. (or NB -> Toronto -> Ashburn -> NYC... or even NB -> Toronto -> NYC) That's going to add a lot of latency compared to another provider that might do NB -> Montreal -> NYC.

Plus, you're in NE NB - it wouldn't surprise me if the fiber to your headend ran from Moncton, so if your traffic is going down to Moncton before heading north-west to Toronto, that's another little bit of latency added there.

Note, too, that looking at layer 3 stuff in traceroutes is a good start but is not really enough here because traceroutes won't show you the physical routing of the underlying fiber.

I don't know if there are any fiber routes directly from NB into the eastern U.S.; if there are, and another provider used those, then again, latency would be further reduced...

[JohnBeaudin]

I am already running tests with Dave about the route and we're working on somehting to improve the route.

[Telek]

... If @RogersDave doesn't get employee of the year, Rogers should be disbanded.

[SickBeast]

I just want to thank @RogersDave for his ongoing support of these new modems and the gigabit service in general.

Also, I have a question.  I'm thinking about running my Coda modem in bridge mode.  I know for certain that my router is affected by the bug in the previous firmware where bridge mode wouldn't work.  It actually bricked my old modem when I tried it.  I noticed that this issue has been resolved since the .23 firmware.  Do you guys think it's safe for me to put my modem into bridge mode now?  Are there any advantages?  Right now all I have done is disabled wifi on the Coda modem and I basically daisy chained my D-link router.  What do you guys think?  I really don't want to brick another modem.  Plus I hear that Rogers is having issues swapping modems right now.

[Jeffj]

Run it in bridge mode. Unless you went out of your way to configure that router as a switch with no routing and dhcp serving. your double natting and that will cause issues and speed loss.

On the flip side. What dlink router do you have? as it might be too slow to handle being the gateway and achieving proper speeds over the coda. It's important to verify that you have HARDWARE NAT capabilities on any router you use if you are on the 250u package or up. 

---

@SickBeast wrote:

@I just want to thank @RogersDave for his ongoing support of these new modems and the gigabit service in general.

Also, I have a question.  I'm thinking about running my Coda modem in bridge mode.  I know for certain that my router is affected by the bug in the previous firmware where bridge mode wouldn't work.  It actually bricked my old modem when I tried it.  I noticed that this issue has been resolved since the .23 firmware.  Do you guys think it's safe for me to put my modem into bridge mode now?  Are there any advantages?  Right now all I have done is disabled wifi on the Coda modem and I basically daisy chained my D-link router.  What do you guys think?  I really don't want to brick another modem.  Plus I hear that Rogers is having issues swapping modems right now.

---

[SickBeast]

---

@Jeffj wrote:

Run it in bridge mode. Unless you went out of your way to configure that router as a switch with no routing and dhcp serving. your double natting and that will cause issues and speed loss.

On the flip side. What dlink router do you have? as it might be too slow to handle being the gateway and achieving proper speeds over the coda. It's important to verify that you have HARDWARE NAT capabilities on any router you use if you are on the 250u package or up. 

---

@SickBeast wrote:

@I just want to thank @RogersDave for his ongoing support of these new modems and the gigabit service in general.

Also, I have a question.  I'm thinking about running my Coda modem in bridge mode.  I know for certain that my router is affected by the bug in the previous firmware where bridge mode wouldn't work.  It actually bricked my old modem when I tried it.  I noticed that this issue has been resolved since the .23 firmware.  Do you guys think it's safe for me to put my modem into bridge mode now?  Are there any advantages?  Right now all I have done is disabled wifi on the Coda modem and I basically daisy chained my D-link router.  What do you guys think?  I really don't want to brick another modem.  Plus I hear that Rogers is having issues swapping modems right now.

---

---

Thanks!  You were right!  My speeds are better now.  Plus I didn't brick my modem.  I really appreciate your help and advice.

My router is a D-link 868L.  It has been tested to achieve 900+mbps in terms of WAN to LAN routing performance.  And this has proven to be true in my testing.

[Hwaiting]

---

@Jeffj wrote:

---

@RogersDave wrote:

---

@gp-se wrote:

@RogersDave got .24 a few hours ago, network is SLOW, so I checked my DOCSIS Logs: 

---

@gp-se, I just ran a remote test on your modem and can clearly see signs of interference at 591 MHz and possibly at 597 MHz.

For the geeks out there, this is really close to a quarter wavelength of WiFi channel 1 (2401-2423 MHz).

My best recommendation is to swap this modem whenever you have a chance.

Dave

---

Just to explain to the non geek a little further and to geek out a bit my self lol

Cable operates between 100MHz to 1000MHZ  area. so much ofthat range is with in a quarter to half wave legnth of the 2.4ghz spectrum.  The fact that most consumer routers sue quarter wave, quarter wave di pole or half wave antennas due to the form factor they need to achieve. this probably has a signifigant factor in the interference as well. 

Also the antenna placement on these access points is not optimal, again due totheformfactor they have to achieve so they actually generate some noise in the spectrum them selves, but typically this doesnt matter because it can still cover the average household so no one notices or cares untill something like interfering with your CM happens. 

Back a few yeats ago, i did an experiment and shielded a TPlink WR-1040D antennas from them selves so they dont cross talk, and essentially doubled the range of the unit, but it changed it from a omni directional unit to a directional due to the shielding. I was also able to reduce the transmit power by almost 60% and still gain better range and thus reducing noise signifigantly. 

Wireless is a cruel mistress sometimes. 

---

So in the case where the modem is experiencing interference, is it experiencing the interference from it's own radio antennas?  And if we are using it in bridge mode with our own routers, does separation from the modem and router make much of a difference if the waves are omnidirectional?  Is there a minimal distance we should have them separated by?  Thanks.

[Jeffj]

---

@Hwaiting wrote:

---

@Jeffj wrote:

---

@RogersDave wrote:

---

@gp-se wrote:

@RogersDave got .24 a few hours ago, network is SLOW, so I checked my DOCSIS Logs: 

---

@gp-se, I just ran a remote test on your modem and can clearly see signs of interference at 591 MHz and possibly at 597 MHz.

For the geeks out there, this is really close to a quarter wavelength of WiFi channel 1 (2401-2423 MHz).

My best recommendation is to swap this modem whenever you have a chance.

Dave

---

Just to explain to the non geek a little further and to geek out a bit my self lol

Cable operates between 100MHz to 1000MHZ  area. so much ofthat range is with in a quarter to half wave legnth of the 2.4ghz spectrum.  The fact that most consumer routers sue quarter wave, quarter wave di pole or half wave antennas due to the form factor they need to achieve. this probably has a signifigant factor in the interference as well. 

Also the antenna placement on these access points is not optimal, again due totheformfactor they have to achieve so they actually generate some noise in the spectrum them selves, but typically this doesnt matter because it can still cover the average household so no one notices or cares untill something like interfering with your CM happens. 

Back a few yeats ago, i did an experiment and shielded a TPlink WR-1040D antennas from them selves so they dont cross talk, and essentially doubled the range of the unit, but it changed it from a omni directional unit to a directional due to the shielding. I was also able to reduce the transmit power by almost 60% and still gain better range and thus reducing noise signifigantly. 

Wireless is a cruel mistress sometimes. 

---

So in the case where the modem is experiencing interference, is it experiencing the interference from it's own radio antennas?  And if we are using it in bridge mode with our own routers, does separation from the modem and router make much of a difference if the waves are omnidirectional?  Is there a minimal distance we should have them separated by?  Thanks.

---

As long at the modem is the black dot with the proper shielding distance *shouldnt* matter and it shouldnt interfear with its self. Otherwise Rogers recommended having your wireless 2.4 turnedo ff on the Coda if its not a black dot and have your router 6 feet minimum away. 

[traghipp]

What's strange is that the color of my download indicator on the coda has changed color to light purple. However, the ofdm/ofdma overview shows both channels disabled. I performed a hard reset as was recommended but the channels are still disabled.

Could this be another bug?

[rjmaxim]

---

@Jeffj wrote:

---

@Hwaiting wrote:

---

@Jeffj wrote:

---

@RogersDave wrote:

---

@gp-se wrote:

@RogersDave got .24 a few hours ago, network is SLOW, so I checked my DOCSIS Logs: 

---

@gp-se, I just ran a remote test on your modem and can clearly see signs of interference at 591 MHz and possibly at 597 MHz.

For the geeks out there, this is really close to a quarter wavelength of WiFi channel 1 (2401-2423 MHz).

My best recommendation is to swap this modem whenever you have a chance.

Dave

---

Just to explain to the non geek a little further and to geek out a bit my self lol

Cable operates between 100MHz to 1000MHZ  area. so much ofthat range is with in a quarter to half wave legnth of the 2.4ghz spectrum.  The fact that most consumer routers sue quarter wave, quarter wave di pole or half wave antennas due to the form factor they need to achieve. this probably has a signifigant factor in the interference as well. 

Also the antenna placement on these access points is not optimal, again due totheformfactor they have to achieve so they actually generate some noise in the spectrum them selves, but typically this doesnt matter because it can still cover the average household so no one notices or cares untill something like interfering with your CM happens. 

Back a few yeats ago, i did an experiment and shielded a TPlink WR-1040D antennas from them selves so they dont cross talk, and essentially doubled the range of the unit, but it changed it from a omni directional unit to a directional due to the shielding. I was also able to reduce the transmit power by almost 60% and still gain better range and thus reducing noise signifigantly. 

Wireless is a cruel mistress sometimes. 

---

So in the case where the modem is experiencing interference, is it experiencing the interference from it's own radio antennas?  And if we are using it in bridge mode with our own routers, does separation from the modem and router make much of a difference if the waves are omnidirectional?  Is there a minimal distance we should have them separated by?  Thanks.

---

As long at the modem is the black dot with the proper shielding distance *shouldnt* matter and it shouldnt interfear with its self. Otherwise Rogers recommended having your wireless 2.4 turnedo ff on the Coda if its not a black dot and have your router 6 feet minimum away. 

---

Just my 2cts of ignorance...isn't the interference only when you are running Coda in Gateway and also attach a router, when both units are broadcasting WiFi frequencies.....I cannot see how the Coda in Bridge mode can be affected by the WiFi frequencies of a router to slow it down. I am not a DataCom guru and would like to understand. Cheers...     

[VivienM]

---

@rjmaxim wrote:

---

Just my 2cts of ignorance...isn't the interference only when you are running Coda in Gateway and also attach a router, when both units are broadcasting WiFi frequencies.....I cannot see how the Coda in Bridge mode can be affected by the WiFi frequencies of a router to slow it down. I am not a DataCom guru and would like to understand. Cheers...     

---

No, this is much lower level radio interference. Let me try to explain in a fairly non-technical way (I don't have the engineering background for a technical explanation)

Fundamentally, a cable network is essentially a collection of radio waves at different frequencies up to 860MHz packaged onto a piece of coax. Other things outside the cable network (e.g. TV broadcasters, cellular providers, etc.) also broadcast radio waves at the same frequency. One thing that's important with cable networks is that you shouldn't leave any coax sitting there not connected to anything because that piece of coax will act as an antenna for the radio waves around it - this is why, for example, you shouldn't have a 3-way splitter if you only have 2 devices, or why you shouldn't have an outlet connected if there's no device there.

The problem with the non-black dot modems (and I am not fundamentally sure that the black dot modems completely fully fix this) is that signals from 'outside' the coax are able to interfere with the signals on the coax before they reach the cable tuner. Those outside signals can come from... anywhere, i.e. including any neighbours' wifi, your own separate router in bridge mode, etc... and some part of the CODA between the coax cable and the tuner effectively acts as an antenna for them.

[c0d3kernel]

I just got the modem yesterday. Initially I though it was my pfsense firewall going all wonky on me. But after investigating, appears that the new CODA 4582 modem keeps on dropping the connection for which the remedy is to reboot the modem.

I suppose I'll also need to have the modem upgraded to the latest firmware. Anybody see improvements with the new firmware?

[HKPolice1]

I'm 90% sure I've figured out a big part of the speed degredation problem:

The CODA-4582 modem is overheating

Even with firmware .24, speedtests during OFF PEAK hours would vary around 7-800Mbps, but would shoot back to over 900Mbps after a fresh reboot before degrading again shortly after. The results were never consistent when degraded, back to back tests could vary by as much as 50Mbps even though I was using the Rogers server on speedtest.net.

Ever since day one, I've felt a lot of heat radiating off the top of the modem. Yes I removed all the plastic wrap before plugging it in. I never gave much thought about the heat at first but considering how inconsistent the performance is, I wanted to experiment.

I placed a standard 120mm case fan on the top of the unit blowing down and shrouded the upper half of the modem on the sides to force the air to exhaust out the bottom half. Did a fresh reboot and now I'm getting consistent ~930Mbps downstream speedtests back to back during off peak hours. I've never been able to get such consistent speedtest results before and I've had the modem since December. On peak hours from ~9pm to 1am are still varying from 600-800Mbps since my node is oversaturated.