cancel
Showing results for 
Search instead for 
Did you mean: 

CODA-4582 - Open Issues for Investigation

RogersDave
Retired Support
Retired Support

*** This post was last edited May 2, 2017 ***

 

Good morning Community,

 

As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.

 

However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.

 

In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂

 

UDP Packet Loss

The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.

 

Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).

 

 

Bufferbloat

When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.

 

Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.

 

The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.

 

Update April 22: This problem seems resolved in firmware 2.0.10.27

 

 

5 GHz WiFi Low range for channels 36 to 48

Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.

 

Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.

 

Workaround: manually select higher channels (149-153-157-161)

 

Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27

 

 

Loss of OFDM Channel Lock

Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.

 

Update April 12: This problem is resolved in 2.0.10.26T2

 

 

List of connected device does not get fully populated

This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.

 

The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.

 

 

NAT Loopback not working for wired clients

When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.

 

Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)

 

 

LAN Counters not working

Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.

 

This problem has been reported to Hitron for investigation.

 

 

Unexpected modem reboot

Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.

 

Update April 12: This problem is resolved in 2.0.10.26T2

 

 

Missing SC-QAM Channels

After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.

 

Investigation continues with Hitron.

 

 

WiFi Survey

The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.

 

 

Guest Network

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless".  This has been reported in firmware 2.0.10.26T2.

 

Update April 22: This issue has been resolved in firmware 2.0.10.27

Update May 2: It seems this issue is not fully resolved and still experienced by some users


 

Future Planned Improvements

The following are items that we are working on in parallel of the above.

  • Improvement in WiFi speeds
  • Improvement in latency / bufferbloat

 

 

Dave

 

*Edited Labels*

2,620 REPLIES 2,620

Re: CODA-4582 - Open Issues for Investigation

Telek
I plan to stick around

Haha I was so mad when my modem rebooted today, I was all worried that the problems had returned. Didn't even notice that it was .24 ! Phew.

So, I went 16 hours with no speed problems and no reboots. That's longer than usual!

Re: CODA-4582 - Open Issues for Investigation

@RogersDave thanks, I know there is a ticket for the area congestion when I called in Rogers and spoke to tech 2 he told me I wasn't the only one having this issue and said they where working on it ,, do you know what's the progress on this @RogersDave

Re: CODA-4582 - Open Issues for Investigation

jjunge
I plan to stick around
@RogersDave - my modem was fine on .19 and on the last few minutes went back to .24 and the minute it did my wired connections are gone again. This is crazy... am I really the only one having this problem? I may need to go off the beta program because I definitely can't have my Ethernet connections not working and can't keep getting upgraded to a non functional version. Can you put me back to the latest production firmware and remove me from the program for now please?

Re: CODA-4582 - Open Issues for Investigation

@RogersDave, I wouldn't mind trying out the new version on .23 as well
Already on the trial program with the first version pf .23 for over a week.

Re: CODA-4582 - Open Issues for Investigation

Telek
I plan to stick around

Holy #@($* batman.  On the latest 0.24 about 1h45m after the reboot.

 

I'm a happy camper at this moment.  I need to figure out which server this hit, and keep hitting it again for the tests.

 

Re: CODA-4582 - Open Issues for Investigation

rjmaxim
I plan to stick around

@Telek wrote:

Holy #@($* batman.  On the latest 0.24 about 1h45m after the reboot.

 

I'm a happy camper at this moment.  I need to figure out which server this hit, and keep hitting it again for the tests.

 


Did you do a Factory Reset after .24? Modem just rebooted and now on .24 Without having done a reset, speeds still same as.19 Will try factory reset and check again.

Re: CODA-4582 - Open Issues for Investigation

Telek
I plan to stick around

@rjmaxim wrote:

Did you do a Factory Reset after .24? Modem just rebooted and now on .24 Without having done a reset, speeds still same as.19 Will try factory reset and check again.


 I did not. Didn't touch it, actually. I've been doing speed tests all day, haven't seen that high yet, HOWEVER I am not sure that speedtest.net / Ookla is actually setup to reliably handle gigabit speed tests. I might have just gotten lucky on this one.

Re: CODA-4582 - Open Issues for Investigation

RyzenFX
I'm a reliable contributor

Woke up this morning to be pleasantly  suprised with DOCSIS 3.1 Smiley Happy 

 

I noticed that my ping went down by 3ms, and interestingly enough, the UDP Packet Loss I experienced on DOCSIS 3.0 is gone aswell. I confirmed this by playing a couple games of League of Legends- all games ranged from (0-2 packets lost/min).  From what I'm currently observing, I think UDP packet loss is only experienced on DOCSIS 3.0. @RogersDave does DOCSIS 3.1 handle UDP packets differently from DOCSIS 3.0?

 

I'm suspecting that Active Queue Management (DOCSIS 3.1) is the driving force behind this. 

Re: CODA-4582 - Open Issues for Investigation

timfolkins
I plan to stick around

.24 received and things seem to be up and running here. Speedtest resulted in 850+ up and 40+ down. Signal levels are primarily +- 1.0 and no 3.1 in my area as of yet (south Mississauga). I will continue to watch for any abnormalities.

 

I am bridged with wired and wireless connections (to my router).

 

Thanks @RogersDave@

Re: CODA-4582 - Open Issues for Investigation

rjmaxim
I plan to stick around

@Telek wrote:

@rjmaxim wrote:

Did you do a Factory Reset after .24? Modem just rebooted and now on .24 Without having done a reset, speeds still same as.19 Will try factory reset and check again.


 I did not. Didn't touch it, actually. I've been doing speed tests all day, haven't seen that high yet, HOWEVER I am not sure that speedtest.net / Ookla is actually setup to reliably handle gigabit speed tests. I might have just gotten lucky on this one.


I did Full Factory reset and you can see the speeds below. Also included DSLreports speed test usingn 12 streams.

OOKLA Speed Tests / Date - TimeBell/TorontoShaw/WinpegRogers/Tor
    
Feb 6 - 11:15   DOCSIS 3.0 (2.0.10.13)269/45-13329/36-31310/47-8
Feb 6 - 16:4091/28-1192/32-3088/41-5
Feb 7 - 07:40259/45-6350/32-31340/48-8
Feb 7 - 11:30279/45-9350/37-30398/37-5
Feb 7 - 16:45268/35-14334/29-29333/22-10
Feb 8 - 08:10296/47-6347/38-30381/48-8
Feb 8 - 11:30      293/46-10346/37-30438/48-8
Feb 8 - 16:05   DOCSIS 3.1 (2.0.10.19)89/17-1091/19-3291/26-4
Feb 8 - 20:1089/38-1191/34-3090/38-4
Feb 9 - 08:15318/46-7358/34-29328/46-4
Feb 9 - 11:3091/47-691/37-3090/41-7
Feb 9 - 16:4592/41-1792/32-3092/31-10
Feb 9 - 21:3090/20-2089/26-3282/30-9
Feb 10 - 10:00273/46-6352/34-29374/41-7
Feb 10 - 16:30247/19-17334/19-30323/28-8
Feb 11 - 08:15290/47-9346/43-2991/46-4
Feb 12 - 12:10265/30-9345/26-30354/22-5
Feb 12 - 16:45267/30-11329/28-30362/18-4
Feb 14 - 10:00327/19-6269/20-31371/16-5
Feb 14 - 16:45308/10-7296/13-31352/14-8
Feb 14 - 20:20267/8-14256/9-30364/9-9
Feb 15 - 11:00313/43-14366/42-30268/44-3
Feb 15 - 16:45177/24-20177/21-27372/27-7
Feb 15 - 22:30160/15-13287/7-32299/13-7
Feb 16 - 18:00   DOCSIS 3.1 (2.0.10.24)127/31-16199/29-31

261/33-8

 

DSL.jpg

Re: CODA-4582 - Open Issues for Investigation

gp-se
I'm an advisor

@RogersDave got .24 a few hours ago, network is SLOW, so I checked my DOCSIS Logs:

 

1 02/16/2017 18:07:11 82000200 critical No Ranging Response received - T3 time-out;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
2 02/16/2017 18:11:17 84000500 critical SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
3 02/16/2017 18:11:17 82000200 critical No Ranging Response received - T3 time-out;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
4 02/16/2017 18:14:52 84000500 critical SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
5 02/16/2017 18:14:52 82000200 critical No Ranging Response received - T3 time-out;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
6 02/16/2017 18:16:58 84000500 critical SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
7 02/16/2017 18:16:59 82000200 critical No Ranging Response received - T3 time-out;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
8 02/16/2017 18:17:29 84000500 critical SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
9 02/16/2017 18:17:31 82000200 critical No Ranging Response received - T3 time-out;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
10 02/16/2017 18:25:49 84000500 critical SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
11 02/16/2017 18:25:51 82000200 critical No Ranging Response received - T3 time-out;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
12 02/16/2017 18:26:39 84000500 critical SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
13 02/16/2017 18:26:41 82000200 critical No Ranging Response received - T3 time-out;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
14 02/16/2017 18:26:57 84000500 critical SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
15 02/16/2017 18:26:58 82000200 critical No Ranging Response received - T3 time-out;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
16 02/16/2017 18:31:40 84000500 critical SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
17 02/16/2017 18:31:42 82000200 critical No Ranging Response received - T3 time-out;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
18 02/16/2017 18:34:45 84000500 critical SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
19 02/16/2017 18:34:46 82000200 critical No Ranging Response received - T3 time-out;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;
20 02/16/2017 18:34:55 84000500 critical SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=00:17:10:90:d2:15;CM-QOS=1.1;CM-VER=3.1;

 

Is my Modem suffering from interference, or is this software/firmware related?

 

I haven't factroy reset yet, incase you wanted to check things out.

 

*EDIT* I disabled 2.4Ghz and everything is running full speed, no more sync errors. I had this exact same issue when I first got this modem on .13 

 

Okay I did more testing, I enabled 2.4Ghz again, however this is what I noticed. If I use Channel 1, I get SYNC errors right away and speed is like 5mbps max. I then changed the channel to 11, no more sync errors, and full speed is back!

 

So it looks like disabling 2.4Ghz isn't needed, just change the channel to 6 or 11. 

 

 

 

Re: CODA-4582 - Open Issues for Investigation

tonytoronto
I plan to stick around

  Got .24 update, so far so good. Download speeds are 330/22 (ignite 250). Did some ping tests and pings are good. Time to turn on 2.4 wi-fi and see what happens.  Fingers crossed.

  Thanks Dave!

Re: CODA-4582 - Open Issues for Investigation

fedge
I plan to stick around

On 2.0.10.19 now...

 

Untitled-1.jpg

 

 

Tests from 7:11pm eastern. Yeah, I know this is primetime.

Re: CODA-4582 - Open Issues for Investigation


@gp-se wrote:

@RogersDave got .24 a few hours ago, network is SLOW, so I checked my DOCSIS Logs: 


@gp-se, I just ran a remote test on your modem and can clearly see signs of interference at 591 MHz and possibly at 597 MHz.

 

For the geeks out there, this is really close to a quarter wavelength of WiFi channel 1 (2401-2423 MHz).

 

My best recommendation is to swap this modem whenever you have a chance.

 

Dave

Re: CODA-4582 - Open Issues for Investigation


@fedge wrote:

On 2.0.10.19 now...

 


I missed you in the update today as I still have your CGNM-3552 modem listed. If I missed anybody else, send me a private message with your new information. I'll update the list tomorrow and perform a second wave of upgrades.

 

Dave

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a senior contributor

@RyzenFX

 

This is huge!

 

I have a few more questions for you if you dont mind..

 

what was your ping playing LoL on D3.0 vs D3.1 did it went down? and how much?

 

and if you play other games, are you getting the same results?

Re: CODA-4582 - Open Issues for Investigation

Triple_Helix
I plan to stick around

I got the new Firmware .24

 

2.0.10.24 (February 16, 2017) *

  • Tentative Fix: Slowly degrading speed over time
  • Tentative Fix: Latency build up over time

 

Image and video hosting by TinyPic

 

All looks and feels good!

 

Downstream Overview
Port IDFrequency (MHz)ModulationSignal strength (dBmV)Channel IDSignal noise ratio (dB)
1591000000256QAM5.9003138.983
2363000000256QAM2.7001040.366
3369000000256QAM3.1001140.366
4375000000256QAM2.6001238.983
5381000000256QAM1.6001338.983
6387000000256QAM1.9001438.983
7393000000256QAM2.5001538.983
8399000000256QAM3.5001638.983
9405000000256QAM3.7001740.366
10411000000256QAM3.8001840.366
11417000000256QAM4.3001940.366
12423000000256QAM4.8002040.366
13429000000256QAM4.5002138.983
14435000000256QAM3.9002240.366
15441000000256QAM4.0002338.983
16447000000256QAM4.0002440.366
17555000000256QAM5.9002540.366
18561000000256QAM6.6002640.946
19567000000256QAM6.5002740.366
20573000000256QAM6.4002840.366
21579000000256QAM6.4002940.366
22585000000256QAM6.3003040.366
23357000000256QAM2.500938.983
24597000000256QAM5.6003238.605
25603000000256QAM6.4003338.983
26609000000256QAM7.1003440.366
27615000000256QAM7.3003540.366
28621000000256QAM7.6003638.983
29633000000256QAM7.4003738.983
30639000000256QAM6.6003838.983
31645000000256QAM5.9003938.605
32651000000256QAM5.3004038.605
OFDM Downstream Overview
ReceiverFFT typeSubcarr 0 Frequency(MHz)PLC lockedNCP lockedMDC1 lockedPLC power(dBmv)
0NANANONONONA
1NANANONONONA
Upstream Overview
Port IDFrequency (MHz)ModulationSignal strength (dBmV)Channel IDBandwidth
130596000ATDMA - 64QAM40.75016400000
238596000ATDMA - 64QAM43.00033200000
323700000ATDMA - 64QAM39.75026400000
OFDM/OFDMA Overview
Channel IndexStatelin Digital AttDigital AttBW (sc's*fft)Report PowerReport Power1_6FFT Size
0DISABLED0.50000.00000.0000-inf-1.00004K
1DISABLED0.50000.00000.0000-inf-1.00004K

Re: CODA-4582 - Open Issues for Investigation

prateeck7
I plan to stick around

Hello @RogersDave,

 

I hope you never leave Rogers. People like you give me some hope that corporate greed can't prevail all the time.

 

Thank you for all your effort.

Re: CODA-4582 - Open Issues for Investigation

Telek
I plan to stick around

@RogersDave wrote:

@gp-se, I just ran a remote test on your modem and can clearly see signs of interference at 591 MHz and possibly at 597 MHz.

 

For the geeks out there, this is really close to a quarter wavelength of WiFi channel 1 (2401-2423 MHz).

 

My best recommendation is to swap this modem whenever you have a chance.

 

Dave


That's interesting. Is the implication that there's insufficient shielding? Or is it that the shielding is actually doing the reverse, and causing resonance inside the shielded area?  What is it that you look at to determine this?

Re: CODA-4582 - Open Issues for Investigation

RyzenFX
I'm a reliable contributor

@JohnBeaudin wrote:

@RyzenFX

 

This is huge!

 

I have a few more questions for you if you dont mind..

 

what was your ping playing LoL on D3.0 vs D3.1 did it went down? and how much?

 

and if you play other games, are you getting the same results?


@JohnBeaudin

 

I'm just as excited as you are! Smiley Happy

 

Regarding my ping on LoL, because I was affected by the UDP packet loss issue on D3.0 my ping was somewhere between 28ms-65ms. NOW, with D3.1 I'm experiencing a constant 24-27ms- a great improvement! 

 

I also played a game of Battlefield 1 tonight and averaged a ping of 24ms. Everything seemed extremly fluid, and I experienced no issues at all. 

Re: CODA-4582 - Open Issues for Investigation


@Telek wrote:
That's interesting. Is the implication that there's insufficient shielding? Or is it that the shielding is actually doing the reverse, and causing resonance inside the shielded area?  What is it that you look at to determine this?

Based on my experience, this happens if the DOCSIS tuner is not sufficiently shielded (or the shield is not perfectly aligned on the tuner).

 

I look at the error rate (QAM codeword errors) on each of the individual SC-QAM channels.

 

Dave