cancel
Showing results for 
Search instead for 
Did you mean: 

CODA-4582 - Open Issues for Investigation

RogersDave
Retired Support
Retired Support

*** This post was last edited May 2, 2017 ***

 

Good morning Community,

 

As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.

 

However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.

 

In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂

 

UDP Packet Loss

The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.

 

Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).

 

 

Bufferbloat

When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.

 

Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.

 

The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.

 

Update April 22: This problem seems resolved in firmware 2.0.10.27

 

 

5 GHz WiFi Low range for channels 36 to 48

Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.

 

Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.

 

Workaround: manually select higher channels (149-153-157-161)

 

Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27

 

 

Loss of OFDM Channel Lock

Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.

 

Update April 12: This problem is resolved in 2.0.10.26T2

 

 

List of connected device does not get fully populated

This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.

 

The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.

 

 

NAT Loopback not working for wired clients

When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.

 

Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)

 

 

LAN Counters not working

Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.

 

This problem has been reported to Hitron for investigation.

 

 

Unexpected modem reboot

Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.

 

Update April 12: This problem is resolved in 2.0.10.26T2

 

 

Missing SC-QAM Channels

After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.

 

Investigation continues with Hitron.

 

 

WiFi Survey

The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.

 

 

Guest Network

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless".  This has been reported in firmware 2.0.10.26T2.

 

Update April 22: This issue has been resolved in firmware 2.0.10.27

Update May 2: It seems this issue is not fully resolved and still experienced by some users


 

Future Planned Improvements

The following are items that we are working on in parallel of the above.

  • Improvement in WiFi speeds
  • Improvement in latency / bufferbloat

 

 

Dave

 

*Edited Labels*

2,620 REPLIES 2,620

Re: CODA-4582 - Open Issues for Investigation

RyzenFX
I'm a reliable contributor

No problem. I'm thinking that the cause for the bufferbloat issue is because of the Speedboost feature and the CMTS. The bufferbloat could be the exact reason why I'm dropping packets.

Re: CODA-4582 - Open Issues for Investigation

Somewhere in all of these threads is a discussion on Bufferbloat as calculated by the DSLReports speedtest, and it has to do with the initial speeds versus the speeds as seen during the remainder of the test.  The result is that the slower speeds as seen beyond the initial rise is considered as bufferbloat by the test and results in an inaccurate calculation.  I think in that discussion Dave pointed out that you have to set the speed for the service that you're on, which should result in a more accurate calculation.  Fwiw, my personal opinion is that the DSLReports servers are, for the most part, too far away to result in a useful indication of the last mile capability, that is to say, between the CMTS and the modem, especially for gigabit service.  There are only two local servers, Toronto and Montreal, every other server is fairly distant, and the server speeds in many cases are low, compared to the Rogers, Beanfield and Fibrenoire servers in Toronto and Montreal.  I found that testing somewhere between midnight to 8:00 am is the only time where you might see a useful result.  Outside of those times, you might as well not bother, and that includes both speedtests and the mythical "bufferbloat".  

 

Also fwiw, I see more random packet loss with the CODA-4582 than I have with other modems which I believe is a little unusual.  That is seen during pingtests to the CMTS. 

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a senior contributor

@gp-se

 

I think we are still several years away from FTTH.. ( 5years+ if not more.. ) 

 

With D3.1 full duplex I don't see any reason to start rolling FTTH any sooner.

Re: CODA-4582 - Open Issues for Investigation

tonytoronto
I plan to stick around

  I can't add much to what actual problem with 4582 and UDP might be, but i can say, since we switched to 3552 i have had zero issues with browsing and playing games. 

 

 

  Like RyzenFX has noted, the speed test on the 3552 are usually steady, on the CODA was always spikes, would hit same speeds but very inconsistent. Bufferbloat was always fail on the CODA, still some on the 3552 but much better.

 

 

Re: CODA-4582 - Open Issues for Investigation


@RyzenFX wrote:

@4kgigabitguy @RogersDave @Datalink I just switched my CGNM-3552 back to the CODA-4582. I ran some test games of LoL, however it's packet loss is nowhere near the packet loss I averaged with the 3552; it's worse. I averaged  6.7 packets lost/min. However, I do have a suspicion that the CMTS is not communicating with the modem properly (i.e CMTS is not giving the modem enough upstream signal strength, which needs to be adjusted). 

 

Upstream Signal Strength:

Upstream Overview

Port IDFrequency (MHz)ModulationSignal strength (dBmV)Channel IDBandwidth
130596000ATDMA - 64QAM29.25016400000
238595785ATDMA - 64QAM33.25033200000
323700000ATDMA - 64QAM29.25026400000

 

 


The uplink signal strengh on the CODA is measured in a completely different way compared to previous devices and this is due to a change in the DOCSIS specification. The power level now takes into account the bandwidth of the channel.

 

For a 6.4 MHz channel, you signal level will be about 6 dB lower than what would be reported by a D3.0 modem. For a 3.2 MHz channel, the power level will be about 3 dB lower than a D3.0 modem.

 

In the case above, this means that if you were to connect a D3.0 modem in the exact same conditions, the power levels would be 35, 36, 35 wich is perfectly fine. The modem is also instructed by the CMTS how to regulate power level so that the signal arrives at the CMTS with a value close to 0. I checked your modem and all three uplink channels are received at the CMTS at 1 dB which is perfect.

 

Dave

Re: CODA-4582 - Open Issues for Investigation


@RyzenFX wrote:

 

@RogersDave @Datalink What do you think of this? I think it makes sense, bufferbloat can be due to the speedboost feature and CMTS, but I'm really curious what both of your opinions are.

 


It is possible that is has an impact with some applications. If the application estimates the available speed and then uses that as a baseline for the rest of the connection, there could be negative impact.

 

There is however positive impact to speedboost when browsing as it makes pages a lot more responsive.

 

I will however "downgrade" my test package and redo some of the tests.

 

Dave

Re: CODA-4582 - Open Issues for Investigation

RogersDave
Retired Support
Retired Support

This morning, I pushed firware 2.0.10.26T2 to all trial participants.

 

If I missed your modem, it is because I don't have your latest information. Send me a PM with your new MAC address and I'll update my records accordingly.

 

As stated previously, we have update on boot enabled on the CODA-4582 which means that the configuration file that your modem downloads at boot include additional information where to download the production firmware (2.0.10.24). When I manually upgrade your modem, I set a flag to instruct your modem to ignore that configuration statement but a factory reset will clear that flag and your modem will be downgraded to production code.

 

If that happens, send me a PM and I'll repush the test firmware to your modem.

 

Dave

Re: CODA-4582 - Open Issues for Investigation

Triple_Helix
I plan to stick around

@RogersDave wrote:

This morning, I pushed firware 2.0.10.26T2 to all trial participants.

 

If I missed your modem, it is because I don't have your latest information. Send me a PM with your new MAC address and I'll update my records accordingly.

 

As stated previously, we have update on boot enabled on the CODA-4582 which means that the configuration file that your modem downloads at boot include additional information where to download the production firmware (2.0.10.24). When I manually upgrade your modem, I set a flag to instruct your modem to ignore that configuration statement but a factory reset will clear that flag and your modem will be downgraded to production code.

 

If that happens, send me a PM and I'll repush the test firmware to your modem.

 

Dave


Thanks Dave I got it!

 

Software Version2.0.10.26T2

 

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a senior contributor

@RogersDave

 

Are we making progress on the UPD issue?

Re: CODA-4582 - Open Issues for Investigation

RyzenFX
I'm a reliable contributor
I believe he's testing for bufferbloat that affects the modem on Internet Tiers below Ignite Gigabit which contain speedboost.

I think that the bufferbloat is way more than it should be on the CODA. I made a suspicion that the bufferbloat is causing the UDP issue. I'm pretty sure it affects upload aswell, which could be the reason why my upload speed is not as consistent when on the 3552.

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a senior contributor

@RogersDave

 

Can we a do a test Dave, can you turn off Speedboost on my modem? so I can test and see if I get better results in gaming?

Re: CODA-4582 - Open Issues for Investigation

Adva
I'm here a lot

People that are experiencing issues with gigabit, do you guys happen to be on FTTH by any chance or coax cable?

I am on 250/20 (FTTH) and have been solid. I fear if I change my package to gigabit i could experience issues.

Re: CODA-4582 - Open Issues for Investigation

Jeffj
I plan to stick around

@RyzenFX wrote:
I believe he's testing for bufferbloat that affects the modem on Internet Tiers below Ignite Gigabit which contain speedboost.

I think that the bufferbloat is way more than it should be on the CODA. I made a suspicion that the bufferbloat is causing the UDP issue. I'm pretty sure it affects upload aswell, which could be the reason why my upload speed is not as consistent when on the 3552.

As far as I'm aware Upload is not speedboosted at all only downstream.

Re: CODA-4582 - Open Issues for Investigation

Telek
I plan to stick around

@Adva wrote:

People that are experiencing issues with gigabit, do you guys happen to be on FTTH by any chance or coax cable?

I am on 250/20 (FTTH) and have been solid. I fear if I change my package to gigabit i could experience issues.


Smiley Surprised you have FTTH?! Nice! In that case you're probably on a totally different system then we are, so gigabit shouldn't have any problems at all, no?

 

What modem do you have?  Is it the strange thing where the fibre is converted to coax in your house, then you just have a normal modem on that?

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a senior contributor

You're the first one on FTTH to speak to these forums, we are all on COAX.. 

 

What kind of modem do you have? 

Re: CODA-4582 - Open Issues for Investigation


@JohnBeaudin wrote:

@RogersDave

 

Can we a do a test Dave, can you turn off Speedboost on my modem? so I can test and see if I get better results in gaming?


It is not something easy to do in production. The Speedboost parameters is tied to your Tier of service.

 

In order to disable Speedboost on a specific tier, we have to build a new configuration file, deploy in production and assign your modem to it.

 

I downgraded my test modem to 150u service and I will be running some test. In parallel, I downgraded the modem from one of the Hitron engineer to 150u and tasked him to play League of Legends for the rest of the day 🙂

 

Hopefully, he'll see some results and will be able to collect the appropriate logs.

 

Dave

 

Re: CODA-4582 - Open Issues for Investigation

RyzenFX
I'm a reliable contributor
Rogers doesn't use any separate modem for gigabit connections. Their FTTH is RFoG, so the fibre connection is converted to coax in the home and the modem that is used is the same as the ones we are provided.

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a senior contributor

ok Thanks @RogersDave !

 

 

Re: CODA-4582 - Open Issues for Investigation

Adva
I'm here a lot

That's right. Rogers Fibre comes to my basement, then they have this converter I call it which turns the wire into a regular cable wire which is connected to my modem. I am on the old 250U package with the old advance wifi router/modem. (I do have access to the other competitor fibre as well, but I can get a better promo with Rogers)

 

I am just warry to signing up for gigtabit package and experience speed and other issues which folks have described here.

 

Re: CODA-4582 - Open Issues for Investigation

Mythen
I plan to stick around
@RogersDave

I got to say that's a pretty rough job to have. Playing league of legends all day. Lol

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a senior contributor

Well you can make him play CSGO as well

 

make him join official servers

 

make sure he runs net_Graph 3 in console so he can monitor the ping and packet loss.