CODA-4582 - Open Issues for Investigation

[RogersDave]

*** This post was last edited May 2, 2017 ***

Good morning Community,

As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.

However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.

In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂

UDP Packet Loss

The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.

Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).

Bufferbloat

When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.

Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.

The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.

Update April 22: This problem seems resolved in firmware 2.0.10.27

5 GHz WiFi Low range for channels 36 to 48

Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.

Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.

Workaround: manually select higher channels (149-153-157-161)

Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27

Loss of OFDM Channel Lock

Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.

Update April 12: This problem is resolved in 2.0.10.26T2

List of connected device does not get fully populated

This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.

The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.

NAT Loopback not working for wired clients

When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.

Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)

LAN Counters not working

Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.

This problem has been reported to Hitron for investigation.

Unexpected modem reboot

Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.

Update April 12: This problem is resolved in 2.0.10.26T2

Missing SC-QAM Channels

After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.

Investigation continues with Hitron.

WiFi Survey

The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.

Guest Network

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless".  This has been reported in firmware 2.0.10.26T2.

Update April 22: This issue has been resolved in firmware 2.0.10.27

Update May 2: It seems this issue is not fully resolved and still experienced by some users

---

Future Planned Improvements

The following are items that we are working on in parallel of the above.

• Improvement in WiFi speeds
• Improvement in latency / bufferbloat

Dave

*Edited Labels*

[Hwaiting]

---

@Adva wrote:

That's right. Rogers Fibre comes to my basement, then they have this converter I call it which turns the wire into a regular cable wire which is connected to my modem. I am on the old 250U package with the old advance wifi router/modem. (I do have access to the other competitor fibre as well, but I can get a better promo with Rogers)

I am just warry to signing up for gigtabit package and experience speed and other issues which folks have described here.

---

I just got a flyer/notice in the mail last week that Rogers (via Pentagon Group) will be upgrading buried services in my neighbourhood over the next few weeks/month.  At first, I thought this was simply a notice that they're going to be burying the temp drop lines that myself and my neighbours on both sides have (all of us got our Rogers cables accidentally cut last fall when Bell came and installed FTTH in my neighbourhood).  I'm wondering if the notice is for something bigger in scope upgrade wise though.

[4kgigabitguy]

Hey @RyzenFX

I ran another test on Leauge of Legends last night(result below). I posted my upstream & downstream logs as well. I also played 1 match of CSGO ( I suck at that game, I'm used to call of duty with a controller lol )

Downstream Overview
Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Signal noise ratio (dB)
1	609000000	256QAM	-5.100	10	37.636
2	561000000	256QAM	-5.400	2	37.356
3	567000000	256QAM	-5.600	3	37.356
4	573000000	256QAM	-5.400	4	37.356
5	579000000	256QAM	-5.300	5	37.636
6	585000000	256QAM	-4.800	6	38.605
7	591000000	256QAM	-4.900	7	37.356
8	597000000	256QAM	-4.900	8	37.636
9	603000000	256QAM	-5.000	9	38.983
10	555000000	256QAM	-5.400	1	37.356
11	615000000	256QAM	-4.800	11	37.636
12	621000000	256QAM	-4.900	12	38.605
13	633000000	256QAM	-4.400	13	38.605
14	639000000	256QAM	-4.900	14	37.636
15	645000000	256QAM	-4.700	15	37.636
16	651000000	256QAM	-5.000	16	37.636
17	657000000	256QAM	-4.800	17	37.636
18	663000000	256QAM	-4.800	18	37.636
19	669000000	256QAM	-5.200	19	37.356
20	675000000	256QAM	-5.100	20	37.636
21	681000000	256QAM	-5.500	21	37.356
22	687000000	256QAM	-5.400	22	37.636
23	693000000	256QAM	-5.900	23	37.356
24	699000000	256QAM	-6.200	24	37.356
25	705000000	256QAM	-6.500	25	37.356
26	711000000	256QAM	-6.700	26	37.356
27	717000000	256QAM	-6.500	27	37.636
28	723000000	256QAM	-6.700	28	37.356
29	825000000	256QAM	-8.600	29	37.636
30	831000000	256QAM	-9.000	30	37.356
31	837000000	256QAM	-8.900	31	37.636
32	843000000	256QAM	-9.100	32	37.356

Upstream Overview
Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Bandwidth
1	30596000	ATDMA - 64QAM	43.500	4	6400000
2	38595609	ATDMA - 64QAM	46.500	6	3200000
3	23700000	ATDMA - 64QAM	43.250	5	6400000

[JohnBeaudin]

@4kgigabitguy

Waiting for the image to get approved, I wan to look into that.

[4kgigabitguy]

---

@Telek wrote:

---

@JohnBeaudin wrote:

A few people also tested the 3552 for gaming vs the CODA and it's a lot better on the 3552, all theory points in the direction that the 3552 is handling better so far, CODA is still on beta and it should out perform the 3552 easily, however we might not see that until a few firmwares fix. but once that's done we should finally see what the CODA is capable of.

---

This is working on the assumption that the hardware is fit. We know that the chipset has major issues. We know that the hardware design itself has issues (hence the need for extra shielding, and known problems with WiFi interference).  I don't think that it's safe to assume that this modem will ever be better, and FirmWare fixes will likely only be to work around hardware limitations.

---

Interesting points, I never tried out the 3552......I honestly thought that modem did not perform as well due to the issues indicated in @Telek post.... I went from CGN3ACSMR to the CODA modem. Catching up on the discussions that happened in the past.

[4kgigabitguy]

Anyone doing testing with PingPlotter ? What's a good address to test & can anyone share ideal results that we should be seeing ? I did a test to google.ca & see small amounts of packet loss on some hops. Would be great to compare results with others.

4kgigabitguy 

[4kgigabitguy]

---

@JohnBeaudin wrote:

Well you can make him play CSGO as well

make him join official servers

make sure he runs net_Graph 3 in console so he can monitor the ping and packet loss.

---

@JohnBeaudin

I am new to CSGO and based on the information I found here  it indicated to use "net_graph 1"..... What's the difference with "net_graph 3" ??

4kgigabitguy

[JohnBeaudin]

@4kgigabitguy

slightly more details with net_graph3 but it's almost the same.

[p00kster]

Jitter Issues with CODA-4582

I just had Rogers Gigabit internet installed a couple days ago, and I've been having major jitter issues since the beginning.  When the jitter gets really bad, my upload speed also decreases a lot as well.  I'm starting to think this is more an issue with an overloaded node than with the modem.  I was wondering if anyone could provide some assistance.  I already had a technician come out, but he couldn't fix the problem.  Any help would really be appreciated.

 I've attached an image of my jitter results and as you can see the numbers are all over the place.  

Thanks.

[RyzenFX]

@4kgigabitguy @RogersDave Wow. You're barely losing any packets at all. I think this is because your upload bufferbloat is really low. Anyways, ran a couple more speedtests on DSL Reports and I found that when speedboost is on, the upload is also affected by bufferbloat as seen here below. When you click onto it, you can read that that my upload bufferbloat is as high as 148ms. This shouldn't be the case especially since speedboost is not enabled on upload. Ideally, I should be seeing upload bufferbloat similar to yours (32ms). I included 2 samples to show that the upload indeed affected by bufferbloat.

NOW, when the speedboost is turned off (i.e Download speed limited to 100Mbps), the upload is still being affected by bufferbloat in the 100's of ms. This could be the cause for the dropped packets in games, and inconsistencies of upload speeds. You can even see that the upload graph is showing inconsistent speed.

[p00kster]

Here are some more results of me pinging the gateway.  Something is definitely not right here.  Getting pings up to 86ms on the gateway alone.

[4kgigabitguy]

@RyzenFX

While the test is underway my buffer bloat bar tends to go up pretty high but then bounces back down....I presume the random sporadic surges are the reason I have not see any grade better then "C"....Pasted a picture of the bar at +211ms.....

[lethalsniper]

@4kgigabitguy can you use easo.ea.com on pingplotter and show us your test would like to compare to mine

[lethalsniper]

Im having the same issue as you , they first told me it was a node issue in my area which they called me back and told me it has been fixed but still getting high jitter/ latency which also drops the upload so I don't think it's just a node issue

[RyzenFX]

@4kgigabitguy Can you share the results of that test to me? I'm curious what your upload bufferbloat was.

[RyzenFX]

I ran a speedtest as of 10:06PM today. Speedtest showed the same inconsistent upload speeds I've been seeing. If I'm correct, the fluctuations (or spikes) on the upload graph of the CODA-4582 is bufferbloat. During the speedtest run on the CODA-4582, it does not maintain the upload speed at a steady rate. For instance, for the first 3-4 seconds of the upload test the speed would maintain 9.5Mbps, but then randomly shoot up (or spike) to 11Mbps (when it does this, it could mean that the modem is suffering from bufferbloat?). However, on the CGNM-3552, it would maintain the the upload speed at a steady rate, and not show any signs of bufferbloat upload wise (you can tell b/c the upload graph on the CGM-3552 is nice and steady). 

You can even start to question if the 1ms jitter is accurately reported. It certainly doesn't look like 1ms jitter is an accurate value given the fluctuations of upload speed seen in the graph of the CODA-4582.

[p00kster]

I signed up for the firmware trial to see if that helps with my situation.  Once I get confirmation that the new firmware is installed, I'll post updates here.

[JohnBeaudin]

@RogersDave

Just got D3.1

It's late at night I had the time to test it for 10 mins in CSGO, usually my ping fluctuate a lot, and during the test I made it stayed 61ms for the whole match, didn't move at all( first time ever since on Rogers) So I will do more testing tomorrow but by the look of it D3.1 is much more stable.

[MegnaGaming]

Interesting...

So first time streaming on the 2.0.10.26T2 software and the dropped frames I experienced were minimal. I have been seeing a fair amount of packet loss starting at the modem (Hop 2) and major issues with dropped frames during the broadcast but tonight was different. Almost no packet loss at the modem but a decent amount of packet loss occuring at the router. 

You can see in the image below that this packet loss doesn't propegate through the various hops (and so wasn't impacting anything) but I thought it was an interesting point to note. Will keep you guys (and @RogersDave) updated  if anything is different tomorrow.

https://i.gyazo.com/4a8667914311b21c3b29402e8be18170.png

[lowlevel]

Not sure if this is the right forum for this... but sounds like there have been some issues with the equipment some of which possibly fixed in the current firmware.  

I've been having packet loss/strange issues since my Sept 2016 install date... which may make my issues unreleated...  Generally my internet service would appear ok, but there would be random packet loss causing failed vpn connections to work, or difficulties starting the next show on netflix, or getting a website to load. SpeedTest.net would usually appear to be acceptable... 

Techs have been out several times to 'play around' with various things... I've been told all kinds of dumbed down stuff like the signals too weak, too strong, there are two signals one weak one strong, signals out of wack/etc. We had the 3552 initially... and just last week a tech 'upgraded' my modem to the CODA-4582 claiming they're excellent/no problems and previous one was very problematic. (Double black dot, firmware 2.0.10.24)

Anyway, it did seem to work more reliably for a while, but after roughly 7 days wifi was showing 'no internet access' on my iphone, and wired speed were dismal...showing 3-8mbps on speedtest, but practically unusable outside of that.   I tried to go into the webadmin, but that hung after a couple of clicks and became totally unresponsive. I had to power cycle the CODA, which restored full 300+mbps downstream.. and wifi.

Upstream appeard to be fine/full speed while I was experiencing the problems with downstream/wifi.

I'm interested to know if a newer firmware is on the horizon/or if it's known there are still issues with these units/etc.  Also willing to help if logs/etc would be beneficial. 

Thanks for being open about what's going on.

Stuart

[sikhness]

Hey Guys,

I have a weird issue that only happens on my Rogers router (on all machines at home connected to the router).  I have the CODA-4582 router with firmware version 2.0.10.24. 

When I try to use the Instagram app on any of my Windows 10 PCs, I notice that the images do not load and only text loads.  Even if I leave the app open for hours nothing will come up.  As soon as I connect my machine to another Wifi source (I have a Mi-fi, and other networks where I've taken my laptop to) everything loads up correctly and instantly.  The minute I connect back to my Rogers router, it goes back to the same behaviour.

I have tried the following and nothing has worked for me:

• Put a single machine on the DMZ and test on that machine
• Turn off the firewall on the router completely for both IPv4 and IPv6
• Restart the router
• Factory restore the router and set it up again

Any help would be greatly appreciated.

Thanks a lot!

[JohnBeaudin]

Hello Community, 

I have very good news to share, since I received the D3.1 update, I have been trying to reproduce the UDP issue with CSGO ( usually it was super easy it was doing it on any server at any time).

Now guess what? I can't reproduce it, 0% packet loss at all time Ping is stable my 3 tests

NY: 61 ms stable 0,02ms var (Time 1:30 AM Atlantic time)

Chicago: 47ms stable 0,001-0,002 ms (Time 11:00 AM Atlantic Time)

Chicago: 47ms stable 0,001-0,002 ms (Time 11:30 AM Atlantic Time)

Tests were done in CSGO, because it was in that game that it was the worst, my ping usually fluctuate between 50-75 in a matter of 2-3 minutes and keep going all way until I am done.

So now seeing a ping so stable is a FIRST on Rogers for me.