cancel
Showing results for 
Search instead for 
Did you mean: 

CODA-4582 - Open Issues for Investigation

RogersDave
Retired Support
Retired Support

*** This post was last edited May 2, 2017 ***

 

Good morning Community,

 

As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.

 

However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.

 

In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂

 

UDP Packet Loss

The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.

 

Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).

 

 

Bufferbloat

When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.

 

Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.

 

The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.

 

Update April 22: This problem seems resolved in firmware 2.0.10.27

 

 

5 GHz WiFi Low range for channels 36 to 48

Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.

 

Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.

 

Workaround: manually select higher channels (149-153-157-161)

 

Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27

 

 

Loss of OFDM Channel Lock

Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.

 

Update April 12: This problem is resolved in 2.0.10.26T2

 

 

List of connected device does not get fully populated

This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.

 

The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.

 

 

NAT Loopback not working for wired clients

When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.

 

Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)

 

 

LAN Counters not working

Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.

 

This problem has been reported to Hitron for investigation.

 

 

Unexpected modem reboot

Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.

 

Update April 12: This problem is resolved in 2.0.10.26T2

 

 

Missing SC-QAM Channels

After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.

 

Investigation continues with Hitron.

 

 

WiFi Survey

The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.

 

 

Guest Network

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless".  This has been reported in firmware 2.0.10.26T2.

 

Update April 22: This issue has been resolved in firmware 2.0.10.27

Update May 2: It seems this issue is not fully resolved and still experienced by some users


 

Future Planned Improvements

The following are items that we are working on in parallel of the above.

  • Improvement in WiFi speeds
  • Improvement in latency / bufferbloat

 

 

Dave

 

*Edited Labels*

2,620 REPLIES 2,620

Re: CODA-4582 - Open Issues for Investigation

Telek
I plan to stick around

I also have the new beta firmware, and my speeds are the same as they were before - about 210/53.  Yes, I'm on the gigabit package and have seen a high of 924/53.

 

Ping has never been an issue, it's the degradation over time that requires reboots twice a day that's been killing me.

 

Here's hoping that this fixes that problem at least.

Re: CODA-4582 - Open Issues for Investigation

Bob35789
I've been here awhile

@RogersDave I Have done a restart my speeds are even slightly worse. 50v/8^

Re: CODA-4582 - Open Issues for Investigation

Bond007
I plan to stick around
Hi Dave,

Please include me in the updated .23 firmware.

Thanks,

Re: CODA-4582 - Open Issues for Investigation

jjunge
I plan to stick around
With this revised .23 firmware I now have a new problem. Nothing hardwired with Ethernet is connecting anymore. My VoIP modem and Xbox - as if they are not plugged in at all. Wireless is fine - help! Anyone else?

Re: CODA-4582 - Open Issues for Investigation

Jeffj
I plan to stick around

I am fine here, in bridge mode.

Re: CODA-4582 - Open Issues for Investigation

Personal opinion, I would run a factory reset.  That's a last resort option if a modem restart doesn't resolve the problem.

Re: CODA-4582 - Open Issues for Investigation

jjunge
I plan to stick around
I've done several restarts and a factory reset already - still nothing

Re: CODA-4582 - Open Issues for Investigation

gp-se
I'm an advisor

I had a similiar problem a few days ago on the original beta .23 firmware. I did this method of factory reset to get my LAN devices to get an ip address:

-With the Modem Powered on, Press and HOLD the reset button on the back of the modem for 20 seconds.

-let the modem boot up, and wait about 2-3 minutes.

-Then press and HOLD the reset button again for 20 seconds.

-Once the modem boots up, run setup using a wired PC. 

 

For me when I went to "Connected Devices" under basic setup I noticed one of my wired devices was getting two ip address's for whatever reason. The only way to fix this was to follow the steps I did above. Also before the factory reset make sure any devices you have under "DHCP Reservation" are all deleted. Even though you're about to do a factory reset, it still doesn't clear out properly, and this caused my issues. 

 

Re: CODA-4582 - Open Issues for Investigation

toolcubed
I'm a senior contributor
@RogersDave, sign me up for the revised .23 firmware! As long as you can continue to push to additional testers 🙂

Re: CODA-4582 - Open Issues for Investigation

QueJoder
I plan to stick around

To RogersDave -Network Architect:

 

To RogersDave, the Network Architect:

(in charge of Re: CODA-4582 - Open Issues for Investigation)

 

Hello, This is the reported Status of my White Modem with black Dot received from Rogers Store on Friday 17 February 2017 (worse performance that the replaced Black 3552). Inconsistent speeds On Average 650/700 D/L and 20/25 Upload via Speedtest.net by OKRA on the Rogers Toronto Server. Also 2/3 Internet signal disconnection per day. Note the last disconnection and my manual reconnection happened only 5 hours ago.

 

This is my 3rd modem since signing for Gigabit Internet at the end of November 2016 and after visits from 5 different technicians and levels of authority.(including a couple of “senior” technicians with their own Individual Business cards including their names on them.

 

In response to my opened ticket on 15Feb2017 regarding the loss of internet connection and the lousy download and upload internet speeds, today I have received grom Rogers Communications the following email:

Important Message

 

 

Hello,

This is an update about your case number C120xxxxxxxxxx.

We heard from you on 2/15/2017 about a problem you were having with your Rogers Internet Service.

We're sorry to say there's been a delay in fixing this problem. Thank you for your patience as our technical teams work on this.

We will contact you as soon as there's an update available.

Thanks for choosing Rogers.

 

 

 

 

 

 

Note In the immediate block below it is curious how the Port ID numbers and the Channel ID numbers 1 and 11 are reversed

Channels 1 to 31 show noise ratios (db) over 40 except Channel 32:(38.983)

Port ID

Frequency (MHz)

Modulation

Signal strength (dBmV)

Channel ID

Signal noise ratio (dB)

1

615000000

256QAM

4.100

11

40.366

11 555000000 256QAM 3.900 1 40.366

 

32 843000000 256QAM 4.200 32 38.983

 

 

Downstream Overview

Port ID

Frequency (MHz)

Modulation

Signal strength (dBmV)

Channel ID

Signal noise ratio (dB)

1

615000000

256QAM

4.100

11

40.946

2

561000000

256QAM

3.900

2

40.366

3

567000000

256QAM

4.300

3

40.366

4

573000000

256QAM

4.300

4

40.366

5

579000000

256QAM

4.100

5

40.366

6

585000000

256QAM

4.200

6

40.366

7

591000000

256QAM

4.400

7

40.366

8

597000000

256QAM

4.200

8

40.946

9

603000000

256QAM

4.300

9

40.366

10

609000000

256QAM

4.200

10

40.366

11

555000000

256QAM

4.000

1

40.366

12

621000000

256QAM

4.400

12

40.366

13

633000000

256QAM

4.900

13

40.366

14

639000000

256QAM

4.800

14

40.946

15

645000000

256QAM

4.900

15

40.366

16

651000000

256QAM

4.700

16

40.366

17

657000000

256QAM

4.900

17

40.366

18

663000000

256QAM

4.900

18

40.946

19

669000000

256QAM

4.800

19

40.366

20

675000000

256QAM

5.100

20

40.946

21

681000000

256QAM

4.600

21

40.946

22

687000000

256QAM

4.600

22

40.946

23

693000000

256QAM

4.600

23

40.366

24

699000000

256QAM

4.900

24

40.946

25

705000000

256QAM

4.600

25

40.946

26

711000000

256QAM

4.900

26

40.366

27

717000000

256QAM

5.100

27

40.366

28

723000000

256QAM

4.900

28

40.366

29

825000000

256QAM

4.100

29

40.366

30

831000000

256QAM

3.800

30

40.366

31

837000000

256QAM

4.200

31

40.366

32

843000000

256QAM

4.200

32

38.983

 

 

 

OFDM Downstream Overview

 

Receiver

FFT type

Subcarr 0 Frequency(MHz)

PLC locked

NCP locked

MDC1 locked

PLC power(dBmv)

0

NA

NA

NO

NO

NO

NA

1

4K

275600000

YES

YES

YES

4.500000

Upstream Overview

Port ID

Frequency (MHz)

Modulation

Signal strength (dBmV)

Channel ID

Bandwidth

1

23700000

ATDMA - 64QAM

31.500

2

6400000

2

38596000

ATDMA - 64QAM

34.500

3

3200000

3

30596000

ATDMA - 64QAM

31.000

1

6400000

OFDM/OFDMA Overview

  

Channel Index

State

lin Digital Att

Digital Att

BW (sc's*fft)

Report Power

Report Power1_6

FFT Size

0

DISABLED

0.5000

0.0000

0.0000

-inf

-1.0000

4K

1

DISABLED

0.5000

0.0000

0.0000

-inf

-1.0000

4K

DOCSIS Provisioning

This menu displays the connectivity status of the modem and its boot state

HW init

Success

Find Downstream

Success

Ranging

Success

DHCP

Success

Time of Day

Success

Download CM Config File

Success

Registration

Success

EAE status

Disable

BPI status

AUTH:authorized, TEK:operational

 

 

Mister David:

In summary and according to your knowledge of the situation with the Gigabit service delivery issue, what should be expected in the next week or so and when will all this mess be soived definitively and Gigabit downloads like speeds and 50Mb/s will be offered by Rogers? to us the paying customers

 

Thank you kindly

 

** removed personnal information **

Re: CODA-4582 - Open Issues for Investigation

@QueJoder I've forwarded your entire post to Dave.  Please delete the entire post, except perhaps for the last paragraph if that is what you would like to do.  The entire post contains the modem MAC address, serial number etc, which shouldn't be left in an open forum.  Dave can either respond to your post or message you via private message when he has the time.  As he has indicated previously, he is working with the Rogers, Hitron, Intel and Casa Systems engineers to determine the cause of the poor performance issue observed with DOCSIS 3.1 and develop the firmware changes necessary to correct the problem.  As a result, he does not have the time to address individual issues as he would like to, so, as a result, we will have to be patient while the combined staff of Rogers, the chipset developers (Intel), manufacturer (Hitron), and CMTS manufacturer (Casa Systems) work out the details to correct the problem.  

Re: CODA-4582 - Open Issues for Investigation

QueJoder
I plan to stick around
Sorry about the mention of s Device address.
Did not see it in the midsta of that plethora of data. Secondly, i never signed into rogers community as QUODER" so i dont know where did that come from
I did send the text though
Is it ok to include my email so i can get any eventual answers?
And lastly I have no idea as to how to delete a post as i need no edit or delete link.

Thank you for passing my post to Dave, the Network Architect

Re: CODA-4582 - Open Issues for Investigation

fedge
I plan to stick around

Tested 4:00 am Thursday, February 16. On version 2.0.10.13.

 

On Rogers' own speed test service, the results were 805dn, 33up & 8ms latency.

 

What speed *should* I be getting? What speed test is most accurate?

Re: CODA-4582 - Open Issues for Investigation

pacerdjl
I plan to stick around

@RogersDave OK, I have pushed the CODA-4582 pretty hard for most of the night. Lots of upload and download across WAN (2.4G, as well as wired), and across LAN devices. No reboots required (so far, after 13+ hours)

  • the speeds up and down have not retreated. Still about 330/21 on 250/20 service
  • bufferbloat is not good (wasn't great before)
  • latency and "jitter" have been pretty good, staying within 12ms and 6ms respectively on a wired connection (although both have increased slightly since first speed test)

Will intitiate a download stream for the rest of the morning, then will check again by mid-afternoon - will update my findings then.

 

After the Feb. 15, 16:56 revised .23 firmware push, the only DOCSIS Event in the log was the following "critical" event, 9.5 hours later, at 02:28:55:

 

No Ranging Response received - T3 time-out;CM-MAC=xx:xx:xx:xx:xx:xx;CMTS-MAC=xx:xx:xx:xx:xx:xx;CM-QOS=1.1;CM-VER=3.1;

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a senior contributor

@RogersDave

 

Do you have enough feed back on teh firmware to roll it out to all participants, I heard it lowered ping for testers, so I can't wait to try it out.

Re: CODA-4582 - Open Issues for Investigation

RogersDave
Retired Support
Retired Support

This test firmware has been running for over 12 hours on a few modems. I am trying to provide some feedback on the effectiveness and make a decision by the end of the day to extend to program to a larger audience. Any feedback is appreciated...

 

 


@JohnBeaudin wrote:

Ping lower?? OMG I WANT IN lol


 @JohnBeaudin, I know you want lower pings but in your case, the biggest problem is simply physical distance and the laws of physics apply....

 

There are core network improvements in the work this year and next year which may help you a bit but no silver bullet on the modem.

 


@jjunge wrote:
With this revised .23 firmware I now have a new problem. Nothing hardwired with Ethernet is connecting anymore. My VoIP modem and Xbox - as if they are not plugged in at all. Wireless is fine - help! Anyone else?

 @jjunge, I tried looking at your modem and see 1 ethernet port up and was able to ping from the modem GUI a device which I believe is hardwired. Was there any change last night?

 


@Bob35789 wrote:

@RogersDave I Have done a restart my speeds are even slightly worse. 50v/8^


 @Bob35789, I tried investigating your modem but you are running in bridge mode and I therefore have no visibility or ability to run performance tests. From what I see everything looks fine on my side.

 

Dave

Re: CODA-4582 - Open Issues for Investigation

Jeffj
I plan to stick around

@RogersDave

 

So far so good for me here. Did a bunch of Streaming with Beam.pro last night. Their new FTL protocol allows for latency free streaming and my connection did not let me down. I was able to maintain a 60fps 1080p Stream with latencts sub second to my viewers.

 

Spent the rest of the evening streaming media from streaming services untill i fell asleep, woke up this morning to the internet STILL WORKING as expected. *so happy*

 

We may have something nice here.

http://www.speedtest.net/result/6057838885.png

Re: CODA-4582 - Open Issues for Investigation

jjunge
I plan to stick around
@RogersDave - neither hardwired device was able to get an ip - port 1 is connected to a switch which is connected to an Xbox and a computer. Port two was connected to VoIP but I disconnected it so that my calls come to my cell. I'm at work right now but will test agin when I get home

Re: CODA-4582 - Open Issues for Investigation

Telek
I plan to stick around

I'm not a gamer, so can't comment on gaming.

 

Pings to normal servers haven't changed at all, they're still the same.

 

Gaming server pings when they're in the 50+ms range means that it's entirely on their end, considering that normal servers can respond in 4-15ms.

 

I don't see modem firmware doing much about that.

Re: CODA-4582 - Open Issues for Investigation

Telek
I plan to stick around

@Jeffj wrote:

 

So far so good for me here. Did a bunch of Streaming with Beam.pro last night. Their new FTL protocol allows for latency free streaming and my connection did not let me down. I was able to maintain a 60fps 1080p Stream with latencts sub second to my viewers.


What did you see before?

 

That protocol, IIRC, seems to be anywhere between 8-12mbit. As long as your connection is up, there shouldn't have ever been any issues?  At least what I've been seing before the upload was never affected, it was the download that dropped out.

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a senior contributor

Modem firmware can improve the latency in many ways, it will be best if Dave or @Datalink explain it to you.