cancel
Showing results for 
Search instead for 
Did you mean: 

CODA-4582 - Open Issues for Investigation

RogersDave
Retired Support
Retired Support

*** This post was last edited May 2, 2017 ***

 

Good morning Community,

 

As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.

 

However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.

 

In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂

 

UDP Packet Loss

The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.

 

Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).

 

 

Bufferbloat

When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.

 

Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.

 

The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.

 

Update April 22: This problem seems resolved in firmware 2.0.10.27

 

 

5 GHz WiFi Low range for channels 36 to 48

Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.

 

Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.

 

Workaround: manually select higher channels (149-153-157-161)

 

Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27

 

 

Loss of OFDM Channel Lock

Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.

 

Update April 12: This problem is resolved in 2.0.10.26T2

 

 

List of connected device does not get fully populated

This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.

 

The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.

 

 

NAT Loopback not working for wired clients

When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.

 

Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)

 

 

LAN Counters not working

Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.

 

This problem has been reported to Hitron for investigation.

 

 

Unexpected modem reboot

Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.

 

Update April 12: This problem is resolved in 2.0.10.26T2

 

 

Missing SC-QAM Channels

After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.

 

Investigation continues with Hitron.

 

 

WiFi Survey

The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.

 

 

Guest Network

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless".  This has been reported in firmware 2.0.10.26T2.

 

Update April 22: This issue has been resolved in firmware 2.0.10.27

Update May 2: It seems this issue is not fully resolved and still experienced by some users


 

Future Planned Improvements

The following are items that we are working on in parallel of the above.

  • Improvement in WiFi speeds
  • Improvement in latency / bufferbloat

 

 

Dave

 

*Edited Labels*

2,620 REPLIES 2,620

Re: CODA-4582 - Open Issues for Investigation

Jeffj
I plan to stick around

@JohnBeaudin wrote:

For some reason ARK is the game where I get the lowest ping and by far out of any games I play.. I can get as low as 29 ping.

 

Dota2: 65 ms

CSGO: 70-75 MS

Overwatch: 45-50 ms

ARK: 29-31 ms

 

all on US-East servers..


I'm about to see if my ping is better i typically get 55-75 to my Private server.  Its the being booted from party chat that was killing me. it would happen all night multiple times, and that was due to latency, parties drop like flies when they get a few seconds of latency/dropped packets, and (possibly UDP?) 

Re: CODA-4582 - Open Issues for Investigation

RyzenFX
I'm a reliable contributor

For anybody who is able to do this on their CODA-4582 and has 10 or more devices connected to the modem and is affected by the UDP packet loss issue:

 

1. Try testing for the issue when you have 10 or more devices connected to it. 

2. Try testing for the issue when you have only 1 device connected to it.

 

If you can, please post the results of both tests including ping, packet loss, and state responsiveness of the game. I'm on to something and this should give me a good idea what's happening.

 

Re: CODA-4582 - Open Issues for Investigation

Here is some food for thought on this.  For anyone running a pc based game, download and install Wireshark. Then, when you are in a game fire up Wireshark and run a short data capture while you're in game, actually doing some activity within the game.  The more activity on your part while engaging another external party, the better.  The goal is to cause/generate UDP packet transfer in both directions.  You probably only need somewhere between 2 to 5 min to capture enough data. Then stop the capture and save the file.  You don't want to run long captures just yet as the file size can grow pretty fast.  I'm sure there is a way to filter data and save that to a file but I havent' looked at Wireshark closely enough to determine exactly how to do that.  I would suggest starting with small files for now and work our way up if necessary.  

 

You don't necessarily have to be a Wireshark guru to read the data, although, it does help, and I'm not a Wireshark guru by any means.  But, you should be able to identify inbound and outbound UDP packets from the display and determine if any of the inbound packets indicate packet corruption, fragmentation or anything else.  

 

You could do a screen capture and send that to Dave via pm as it will contain your IP address and various IP addresses that your pc is communicating with.  Possibly Dave will request the file via pm, don't know at this point.

 

It would be interesting to see what turns up on the data capture.   

 

https://www.wireshark.org/

 

Edit:  Don't post a Wireshark screen capture to this or any other public forum due to the type and amount of IP addresses contained in the capture.  Or, in other words, for your own online security, keep your own data secure. 

 

Re: CODA-4582 - Open Issues for Investigation

tonytoronto
I plan to stick around

  You welcome.

   Kids are finally happy and wife too, no more complaning about internet not working.

   I like using my ipad while in the threadmill, usually watching Netflix or from somethig on the server, with CODA was getting disconnects 2-3 times every hour, so freaking annoying, plus trying to "fix" while walking was pain in the b***.  no more disconnects with 3552.

   Spent little time last night checking pings and such, to some servers might have gone up couple ms, but nothing that will make diference surfing or playing games. Theres couple little things with this modem too, but no biggie, all things i can leave with for now.


@lethalsniper wrote:

@tonytoronto@ so i picked up the  CGNM-3552 modem and me and brother both played online using the xbox one, and so far so good game feels more responsive and alot less laggy now dont know why but could be the UDP like you said Thanks @tonytoronto was having bad experience and very frustrated when on the coda modem.

@hopefully @RogersDave can rollout a udp fix for the coda modem because its  useless for gamers and there are a lot of us out there.


 

Re: CODA-4582 - Open Issues for Investigation

Jeffj
I plan to stick around

@Datalink wrote:

Here is some food for thought on this.  For anyone running a pc based game, download and install Wireshark. Then, when you are in a game fire up Wireshark and run a short data capture while you're in game, actually doing some activity within the game.  The more activity on your part while engaging another external party, the better.  The goal is to cause/generate UDP packet transfer in both directions.  You probably only need somewhere between 2 to 5 min to capture enough data. Then stop the capture and save the file.  You don't want to run long captures just yet as the file size can grow pretty fast.  I'm sure there is a way to filter data and save that to a file but I havent' looked at Wireshark closely enough to determine exactly how to do that.  I would suggest starting with small files for now and work our way up if necessary.  

 

You don't necessarily have to be a Wireshark guru to read the data, although, it does help, and I'm not a Wireshark guru by any means.  But, you should be able to identify inbound and outbound UDP packets from the display and determine if any of the inbound packets indicate packet corruption, fragmentation or anything else.  

 

You could do a screen capture and send that to Dave via pm as it will contain your IP address and various IP addresses that your pc is communicating with.  Possibly Dave will request the file via pm, don't know at this point.

 

It would be interesting to see what turns up on the data capture.   

 

https://www.wireshark.org/

 

Edit:  Don't post a Wireshark screen capture to this or any other public forum due to the type and amount of IP addresses contained in the capture.  Or, in other words, for your own online security, keep your own data secure. 

 


The only problem with this is, unles you have a managed switch with a monitoring port you're only going to see one side of the traffic, even in permiscuous mode. An alternitive is to man in the middle the connection but then you have to log the connection one way at a time so it isnt exactly matching in and out transactions, and it also causes you to only duplex at 10 base t when doing so.

 

So unless an end user has the equipment to do it, the data collected might not be very useful to @RogersDave

 

 

Re: CODA-4582 - Open Issues for Investigation

Datalink
Resident Expert
Resident Expert

Yup, absolutely agree, passive tap to log both inbound and outbound would be ideal.  Too bad this never occurs under ideal conditions 😞  You're correct in that it would only show the effects on one side of the traffic which is the inbound traffic.  But, its better than nothing and it would serve as a check to determine if there are issues with inbound UDP traffic for gaming or any other application.  Ideally, you would monitor both sides of the modem for both inbound and outbound traffic to see the before and after effects that are imposed by the modem.  But, that requires a much higher level of equipment and ability to match the packet frames.  Seems to me that the chipset manufacturers should be the ones doing this on a regular basis in order to ensure that the modem operates as it should.  It shouldn't be left to the tail end of the manufacturing and distribution chain to come up with methods to accomplish this. Personal opinion of course 😞

Re: CODA-4582 - Open Issues for Investigation

Jeffj
I plan to stick around

@Datalink wrote:

Yup, absolutely agree, passive tap to log both inbound and outbound would be ideal.  Too bad this never occurs under ideal conditions 😞  You're correct in that it would only show the effects on one side of the traffic which is the inbound traffic.  But, its better than nothing and it would serve as a check to determine if there are issues with inbound UDP traffic for gaming or any other application.  Ideally, you would monitor both sides of the modem for both inbound and outbound traffic to see the before and after effects that are imposed by the modem.  But, that requires a much higher level of equipment and ability to match the packet frames.  Seems to me that the chipset manufactures should be the ones doing this on a regular basis in order to ensure that the modem operates as it should.  It shouldn't be left to the tail end of the manufacturing and distribution chain to come up with methods to accomplish this. Personal opinion of course 😞


I agree with you, this should be Hitron soing tests on Reported gaming infrastructure and other reported service issues with UDP.

 

I have no problems letting Dave set up some sort of monitor on my Wan connection on his end but i dont even know if thats possible for him to track my I/O in that aspect.

One thing ive noticed pinging download.xbox.com which hits their CDN, is the thing flip flops back and forth between IPV6 and IPV4 and whn it does i get a latency spike.  I wonder if this is also salting the wound so to speak on the issues with gaming CDN's and UDP.

Re: CODA-4582 - Open Issues for Investigation

Datalink
Resident Expert
Resident Expert

Wonder why it doesn't operate solely on IPV4 or IPV6 instead of switching back and forth?  You should pm Dave with the details and logs if you have any that show that.

Re: CODA-4582 - Open Issues for Investigation

Jeffj
I plan to stick around

Trying to ponder on a way to send this data to @RogersDave in some sort of meaningful format as my trial of PinPlotter has expired so it only shows real time data and ive just happened to catch it in the ipv4 ipv6 flip flop act a few times today. 

Re: CODA-4582 - Open Issues for Investigation

Datalink
Resident Expert
Resident Expert

You can see red arrows on the bottom? And if you hover the mouse over the arrow it displays the IP address change?

 

I've seen that over the last couple of days, running a ping test to google.ca where the IP address for google changes, and then changes back.  The first change results in a slightly higher ping time and remains that way until it changes back.  This is all with IPV4 as I have IPV6 turned off for now.   The flip flop from IPV4 to IPV6 and back is a different twist on this.  

 

With the freebie pingplotter, the data will still be there, but you won't be able to access it.  Anyone with the Standard or Pro version will be able to access the file if you export the sample set.  I run the standard version.

 

 

Edit:  If you go Edit .... Copy as Image and then paste that into something like MS Paint, you will see the notes at the bottom of the image which includes the details of the events that are marked by the markers or arrows.  At least that's what the Standard version does.  I would expect the freebie version to do the same, unless they cut that off as well.   So, it should be a matter of waiting until one of these appears on the display, then run the Copy as Image function and dump it into something to see if the notes show up at the bottom left of the image. 

Re: CODA-4582 - Open Issues for Investigation

Jeffj
I plan to stick around

@Datalink wrote:

You can see red arrows on the bottom? And if you hover the mouse over the arrow it displays the IP address change?

 

I've seen that over the last couple of days, running a ping test to google.ca where the IP address for google changes, and then changes back.  The first change results in a slightly higher ping time and remains that way until it changes back.  This is all with IPV4 as I have IPV6 turned off for now.   The flip flop from IPV4 to IPV6 and back is a different twist on this.  

 

With the freebie pingplotter, the data will still be there, but you won't be able to access it.  Anyone with the Standard or Pro version will be able to access the file if you export the sample set.  I run the standard version.

 

What are you pinging that you see the shift from IPV4 to IPV6?

 

Edit:  If you go Edit .... Copy as Image and then paste that into something like MS Paint, you will see the notes at the bottom of the image which includes the details of the events that are marked by the markers or arrows.  At least that's what the Standard version does.  I would expect the freebie version to do the same, unless they cut that off as well. 


Yep seeing the red arrow and seeing the change from ipv4 to 6.  when it happens it flips and then 5 min later flops. I'll start up the test again and get some data and take a image.

download.xbox.com

 

Further more i am seeing it switch me from local CDN servers and then to the netherlands as well some times, and some tiems seeing my route go through Verizons layer in the states and soem times it doesnt. its realy unstable. 

Re: CODA-4582 - Open Issues for Investigation

Datalink
Resident Expert
Resident Expert

More than a little strange considering that Pingplotter should be using an IP address.  But, if you are using download.xbox.com as the target, I wonder if pingplotter goes out with a DNS request and gets a different response at some point?  That would show up on a Wireshark log perhaps?  What do you use for DNS servers?  I'm using OpenDNS and see and address changes which are all within the Rogers IP address ranges.  If you're using the Rogers DNS and seeing an address shift from IPV6 to IPV4 and back, thats a little interesting as well.  

 

Just thinking about this I'm using Google.ca as a target.  I'll have to give this a try with an IP address to see what happens.  Maybe this is nothing more than a DNS problem at the DNS servers?  

 

 

Re: CODA-4582 - Open Issues for Investigation

Jeffj
I plan to stick around

@Datalink wrote:

More than a little strange considering that Pingplotter should be using an IP address.  But, if you are using download.xbox.com as the target, I wonder if pingplotter goes out with a DNS request and gets a different response at some point?  That would show up on a Wireshark log perhaps?  What do you use for DNS servers?  I'm using OpenDNS and see and address changes which are all within the Rogers IP address ranges.  If you're using the Rogers DNS and seeing an address shift from IPV6 to IPV4 and back, thats a little interesting as well.  

 

Just thinking about this I'm using Google.ca as a target.  I'll have to give this a try with an IP address to see what happens.  Maybe this is nothing more than a DNS problem at the DNS servers?  

 

 


I am using Rogers DNS. I decided to stick with it for testing, i typically use googles DNS. 

 

 

THis could very well be a dns issue I agree, I just find it curious as once pingplotter resolves a name, and gets its target it should stick to it unless its forced to relookup?

Re: CODA-4582 - Open Issues for Investigation

Pejmank
I'm here a lot

Hi, 

 

When I found this forum I was happy that I'm not the only one having issues with Gig Internet.

I followed all the steps here:

 

-Changed modem to one with 2 black dots.

-reboot and tested with and without bridge mode

-asked tech to come and check my signal 

-tested with one device, multi devide, waited to have the latest firmware and anything else that I know of

 

but still having issues with download speed.

SpeedCheck.png

my Modem firmware

Hardware Version

1A

Software Version

2.0.10.24

 

also this is what my modem shows in the Docsis wan

Downstream Overview

 

Port ID

Frequency (MHz)

Modulation

Signal strength (dBmV)

Channel ID

Signal noise ratio (dB)

 

1

615000000

256QAM

5.500

11

38.983

 

2

561000000

256QAM

5.800

2

38.605

 

3

567000000

256QAM

5.900

3

37.636

 

4

573000000

256QAM

6.700

4

37.356

 

5

579000000

256QAM

6.400

5

38.605

 

6

585000000

256QAM

6.400

6

38.605

 

7

591000000

256QAM

6.000

7

38.605

 

8

597000000

256QAM

6.100

8

38.605

 

9

603000000

256QAM

5.700

9

38.605

 

10

609000000

256QAM

5.700

10

38.983

 

11

555000000

256QAM

6.300

1

38.605

 

12

621000000

256QAM

5.200

12

38.605

 

13

633000000

256QAM

4.800

13

38.605

 

14

639000000

256QAM

5.800

14

38.983

 

15

645000000

256QAM

4.800

15

38.605

 

16

651000000

256QAM

4.400

16

38.605

 

17

657000000

256QAM

5.700

17

38.605

 

18

663000000

256QAM

6.500

18

38.983

 

19

669000000

256QAM

5.200

19

38.605

 

20

675000000

256QAM

4.800

20

38.605

 

21

681000000

256QAM

5.800

21

38.605

 

22

687000000

256QAM

6.400

22

38.605

 

23

693000000

256QAM

5.100

23

37.356

 

24

699000000

256QAM

5.500

24

37.636

 

25

705000000

256QAM

5.700

25

37.636

 

26

711000000

256QAM

5.700

26

37.356

 

27

717000000

256QAM

5.600

27

37.356

 

28

723000000

256QAM

5.200

28

37.356

 

29

825000000

256QAM

2.500

29

36.387

 

30

831000000

256QAM

2.500

30

36.610

 

31

837000000

256QAM

2.200

31

36.610

 

32

843000000

256QAM

1.600

32

35.780

 

OFDM Downstream Overview

Receiver

FFT type

Subcarr 0 Frequency(MHz)

PLC locked

NCP locked

MDC1 locked

PLC power(dBmv)

0

NA

NA

NO

NO

NO

NA

1

4K

275600000

YES

YES

YES

6.000000

          

 

Upstream Overview

 

Port ID

Frequency (MHz)

Modulation

Signal strength (dBmV)

Channel ID

Bandwidth

 

1

38595609

ATDMA - 64QAM

45.750

3

3200000

 

2

30596000

ATDMA - 64QAM

41.500

1

6400000

 

3

23700000

ATDMA - 64QAM

40.750

2

6400000

 

OFDM/OFDMA Overview

 

Channel Index

State

lin Digital Att

Digital Att

BW (sc's*fft)

Report Power

Report Power1_6

FFT Size

0

DISABLED

0.5000

0.0000

0.0000

-inf

-1.0000

4K

1

DISABLED

0.5000

0.0000

0.0000

-inf

-1.0000

4K

             

 

 anyone have any idea what to try next?

@RogersDave I really appreciate if you can take a look and help me to figure this out. I'm dealing with this since Oct last year!

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a senior contributor

@RogersDave

 

Will .26 be ready by the end of the week? I can't wai to give you feedback on modem reboots, mine reboot average twice a day 🙂

Re: CODA-4582 - Open Issues for Investigation

RyzenFX
I'm a reliable contributor

@JohnBeaudin Hopefully it releases soon. I'm really curious to see how it performs in gaming with say 5-8 devices connected to it.

 

EDIT: Firmware .26 will be released today. Please let me know if the UDP issue is still present.

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a senior contributor

@RyzenFX

 

Fingers crossed!

Re: CODA-4582 - Open Issues for Investigation

JohnBeaudin
I'm a senior contributor

@RyzenFX

 

Out of curiosity how do you know .26 will be released today?

Re: CODA-4582 - Open Issues for Investigation

Good morning Community,

 

I just updated the release notes for firmware 2.0.10.26 in this post.

 

I will be starting the upgrades of non CODA modems today. For CODA-4582 I don't know yet if I'll be able to push them today or only start tomorrow.

 

For technical background, the CODA-4582 are currently configured to automatically upgrade on boot to version 2.0.10.24. If I upgrade any modem at this point, it will automatically downgrade back at reboot. We are making a change in our configuration file and I'll start the deployment as soon as the configuration is in place.

 

Now there is an important change in process for trial participants on the CODA modem. When I'll push the trial firmware, your modem will be a in state where it ignores upgrade commands from the network (so that it doesn't downgrade back to 2.0.10.24). However, if you perform a factory reset, the modem will automatically be set back in network upgrade mode and go back to 2.0.10.24.

 

The new configuration file will also include the "Pixel/Android fix" that greatly improves the stability of WiFi connections for Android devices. Because this will be included in the configuration file, all modems will receive this fix with a reboot (no need to be part of the trial or on firmware 2.0.10.26). 

 

As a side note, the performance/range of 5 GHz channels 36-40-44-48 on the CODA modem seems to be really poor. I recommend to everybody using WiFi functionality on the CODA-4582 to manually set their 5 GHz channel to 149-153-157-161.

 

Dave

Re: CODA-4582 - Open Issues for Investigation

Missing downsteam problem is back on a black sticker CODA-4582. Channels 2 to 10 are gone,  was connecting to all channels previously.  Not sure if it's realated to the main line break in my neighbourhood, a temp line was installed as a thaw is needed to bury it.  First RCS partial service error happened on March 14. 

Re: CODA-4582 - Open Issues for Investigation

Triple_Helix
I plan to stick around

@bahha wrote:

Missing downsteam problem is back on a black sticker CODA-4582. Channels 2 to 10 are gone,  was connecting to all channels previously.  Not sure if it's realated to the main line break in my neighbourhood, a temp line was installed as a thaw is needed to bury it.  First RCS partial service error happened on March 14. 


Unplug the Modem from the wall for about 10 to 15 seconds then plug it back in and check again when it's done.