CODA-4582 - Open Issues for Investigation

[RogersDave]

*** This post was last edited May 2, 2017 ***

Good morning Community,

As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.

However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.

In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂

UDP Packet Loss

The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.

Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).

Bufferbloat

When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.

Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.

The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.

Update April 22: This problem seems resolved in firmware 2.0.10.27

5 GHz WiFi Low range for channels 36 to 48

Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.

Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.

Workaround: manually select higher channels (149-153-157-161)

Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27

Loss of OFDM Channel Lock

Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.

Update April 12: This problem is resolved in 2.0.10.26T2

List of connected device does not get fully populated

This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.

The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.

NAT Loopback not working for wired clients

When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.

Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)

LAN Counters not working

Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.

This problem has been reported to Hitron for investigation.

Unexpected modem reboot

Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.

Update April 12: This problem is resolved in 2.0.10.26T2

Missing SC-QAM Channels

After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.

Investigation continues with Hitron.

WiFi Survey

The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.

Guest Network

When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless".  This has been reported in firmware 2.0.10.26T2.

Update April 22: This issue has been resolved in firmware 2.0.10.27

Update May 2: It seems this issue is not fully resolved and still experienced by some users

---

Future Planned Improvements

The following are items that we are working on in parallel of the above.

• Improvement in WiFi speeds
• Improvement in latency / bufferbloat

Dave

*Edited Labels*

[damir]

Dave, there was a noise on my node, confirmed / fixed.

I can PM you more info about it (in case you need it / details - as i have them).
There is no more packet loss, however;

I am still facing a lot slower speeds compared to previous firmware versions.

I did try couple of times to factory reset the modem, that did not help.

I tried gateway mode, that did not help.

I tried different times of the day, same outputs, even 3am-7am.

If you have any  other ideas / if you require any specific tests, please let me know which / what, and i will provide it.

Thanks,

Damir

[Kyrluckechuck-2]

Thanks for actually testing that, I appreciate it Dave!

[DaveFro]

Thanks for all the info RogersDave! I sent you a PM with more details about the below.

I appear to be missing channels 22 and 23.... 

[toolcubed]

So if we're running our CODA modems in bridge mode, we should never experience the intereference since WiFi is completely disabled.  Is that right?  Or would you still recommend that we swap our modems out for the newer version (with the black dot)?

[RogersDave]

---

@toolcubed wrote:

So if we're running our CODA modems in bridge mode, we should never experience the intereference since WiFi is completely disabled.  Is that right?  Or would you still recommend that we swap our modems out for the newer version (with the black dot)?

---

It could still be affected in some cases, especially if your router is right beside your modem. Now to be clear, it's really a subset of modems that have this issue. In your case, I just reviewed your modem and don't see any sign of this impact.

As part of the trial firmware program, I do monitor more closely the performance of the modems. I will be contacting via private message, likely next week, everybody who is part of this program and could benefit from a modem swap.

Dave

[hptouchdroid13]

Here is my speedtest. I just got the CODA yesterday and it got .19 firmware last night. 

[gbssengo]

---

@hptouchdroid13 wrote:

Here is my speedtest. I just got the CODA yesterday and it got .19 firmware last night. 

---

Lucky you.

This is mine. on firmware .19

[gp-se]

---

@RogersDave wrote:

---

@gp-se wrote:

What are the symptoms of the interference issue? How would a user know if he is afftected by this.

@gp-se, It is hard for end users to detect this situation but it may be possible in some cases.

In the DOCSIS Status page, there could be missing channel (the channel ID column will completely skip a number). In some cases, there will be some channels with a SNR really low (10 to 15). There are also cases where none of the above shows a problem.

We are running scans on the network to flag affected modem and I'll be in contact with some of you to expedite a replacement.

---

@Alex4161 wrote:
@RyzenFX @RogersDave

What are the causes of this tuner interference? Is it related to placement, bad splitter, wiring, amps on the line?

thanks

---

@Alex4161, As stated previously, the interference is generally internal to the modem where 2.4 GHz WiFi can leak into the DOCSIS tuner and affect certain frequencies. This only affects a subset of modems but the newer generation modems (with a black dot) have extra protection against that.

---

@Double_K wrote:

@RyzenFX, thanks!

Did the FCC ID / IC number change on the back of the modem?

How does it have better shielding from interference?

---

@Double_K, There is no other indication, no difference in FCC ID or hardware revision. It strictly has an extra shield on top of the DOCSIS tuner.

Dave

---

I just checked my modem, all 32 channels and SNR between 38-40, so I'm guessing I am good. 

[hoob]

Random Friday evening prime time speed check on Rogers Speed Test:

Firmware .19

Uptime 38 hours

Friday the 13th 19h40 prime time stream-oclock

DOCSIS 3.0 4 of 32 download channels offline due to wifi interference issue

Download Speed: 671Mbps

Upload Speed: 26Mbps

Latency: 7ms

Jitter: 4ms

Things are looking good! No gradual or suddent modem slow downs as far as I can tell, however the modem reboot 38 hours ago was unsolicited.

[toolcubed]

Thanks Dave.

[14N]

Is Rogers recalling the first coda due to interference issue?

[toolcubed]

Dave,

Last night I decided to run some speed tests while I was wired directly to my RT-AC68U (CODA in bridge mode) and I couldn't break past 50M down and 3-5M up (I'm on the 100u plan). Usually I pull 200 down and 11-12 up. After I ran the slow speed tests, I checked my DOCSIS stats on the modem and everything was fine...signal levels were great as usual and no missing channels. Was the cause of the slow speed test due to local node congestion or could my CODA be exhibiting some form of interference?

[gp-se]

---

@toolcubed wrote:
Dave,

Last night I decided to run some speed tests while I was wired directly to my RT-AC68U (CODA in bridge mode) and I couldn't break past 50M down and 3-5M up (I'm on the 100u plan). Usually I pull 200 down and 11-12 up. After I ran the slow speed tests, I checked my DOCSIS stats on the modem and everything was fine...signal levels were great as usual and no missing channels. Was the cause of the slow speed test due to local node congestion or could my CODA be exhibiting some form of interference?

---

How far is your Asus from the CODA? Also is your speed consistanly low with your CODA/Asus, or just last night doing tests? If you have an interference issue, I would assume speed tests would be slow all the time. Make sure you keep your Asus at least 6 FEET from the CODA, and run some more tests, if speeds improve, then move the Asus closer to the CODA, if speeds drop then you have an interference issue.

[arrago]

I think, I'm starting to see conguestion in my area + Coda issues  you guys speak of here. Over time I notice the modem slows down. (speeds of 100mbps). Also if you don't reboot games stutter like crazy best way to describe it. Signals are good, so maybe chipset bug?? 

Anyways today sat I'm getting full speeds....

(as close as i'd get it without having rogers newest wiring in the area anyways)

451.76 mbps (max i've seen is 700 none existant on this coda modem 600 seems my max it can hit)

54.23mbps

ping 8ms

jitter 3ms

2nd test

558.63Mbps

54.16Mbps

ping 4ms

jitter 3ms

**Added Signal Levels**

DHCP Lease Time

😧 07 H: 00 M: 00 S: 00

Downstream Overview

Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Signal noise ratio (dB)
1	615000000	256QAM	0.800	35	38.983
2	363000000	256QAM	0.900	10	38.983
3	369000000	256QAM	0.500	11	38.605
4	375000000	256QAM	0.900	12	38.983
5	381000000	256QAM	1.000	13	38.983
6	387000000	256QAM	1.600	14	38.983
7	393000000	256QAM	1.700	15	40.366
8	399000000	256QAM	1.300	16	40.366
9	405000000	256QAM	1.000	17	38.983
10	411000000	256QAM	0.900	18	38.983
11	417000000	256QAM	1.000	19	38.983
12	423000000	256QAM	1.400	20	38.983
13	429000000	256QAM	1.500	21	38.605
14	435000000	256QAM	1.900	22	38.983
15	441000000	256QAM	2.000	23	40.366
16	447000000	256QAM	1.800	24	38.983
17	555000000	256QAM	-1.200	25	37.356
18	561000000	256QAM	-2.300	26	37.356
19	567000000	256QAM	-3.900	27	37.356
20	573000000	256QAM	-2.500	28	37.356
21	579000000	256QAM	-0.500	29	38.605
22	585000000	256QAM	0.300	30	38.983
23	591000000	256QAM	0.500	31	38.983
24	597000000	256QAM	0.500	32	38.983
25	603000000	256QAM	0.700	33	38.983
27	357000000	256QAM	1.400	9	38.605
28	621000000	256QAM	0.400	36	38.983
29	633000000	256QAM	0.400	37	38.983
30	639000000	256QAM	0.500	38	38.983
31	645000000	256QAM	0.800	39	38.605
32	651000000	256QAM	1.000	40	38.983

OFDM Downstream Overview

Receiver	FFT type	Subcarr 0 Frequency(MHz)	PLC locked	NCP locked	MDC1 locked	PLC power(dBmv)
0	NA	NA	NO	NO	NO	NA
1	NA	NA	NO	NO	NO	NA

Upstream Overview

Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Bandwidth
1	38596000	ATDMA - 64QAM	37.500	3	3200000
2	30596000	ATDMA - 64QAM	35.500	1	6400000
3	23700000	ATDMA - 64QAM	34.250	2	6400000

OFDM/OFDMA Overview

Channel Index	State	lin Digital Att	Digital Att	BW (sc's*fft)	Report Power	Report Power1_6	FFT Size
0	DISABLED	0.5000	0.0000	0.0000	-inf	-1.0000	4K
1	DISABLED	0.5000	0.0000	0.0000	-inf	-1.0000	4K

*edit again*

noticed T3 time up accomulate when speeds aspeed to be very slow on the modem before reboots. can post logs however levels are always good.

[gp-se]

you're missing channel 26, exchange the modem for the new one with the black dot on the box, and black dot on the modem beside the power cord connector

[prateeck7]

@RogersDaveSo after a week of 930+mbps I was getting ocnsistently, now I am not even getting past 100mbps speed for the past 3 days. Speed goes to 700mbps at max and then falls below 100mps. Can you, please, check what it going on in my area node? In the meantime I have asked @CommunityHelps to update me to .20.

[Dragus]

Almost 6 days in with .19 firmware. No more speed degradation issues and 10-20% packetloss.

Although my download speed is around 200-600 Mb/s down with 38-44 Mb/s upload during prime hours I can't complain, more than enough for my path of exile gaming 😜

But the midnight I did get the.19 firmware, the service technician did replace my previous CODA modem that same day. So really hard to tell if it was the previous CODA modem since people on .19-.20 are still having speed degradation issues.

Edit: forgot to mention I've always been running bridge mode on an Asus RT-AC3100 wired.

[arrago]

so I went in to swap my modem with the "black dot" version. I now have channel 26. I had one of the orginal batches. it has firmware version .13 on it not sure if i'll get newer any time soon.

Software Version

2.0.10.13

new levels

DHCP Lease Time

😧 06 H: 21 M: 55 S: 39

Downstream Overview

Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Signal noise ratio (dB)
1	591000000	256QAM	0.800	31	38.983
2	363000000	256QAM	1.900	10	38.983
3	369000000	256QAM	1.600	11	40.366
4	375000000	256QAM	1.800	12	38.983
5	381000000	256QAM	1.900	13	40.366
6	387000000	256QAM	2.500	14	40.946
7	393000000	256QAM	2.600	15	40.366
8	399000000	256QAM	2.400	16	40.366
9	405000000	256QAM	2.000	17	40.366
10	411000000	256QAM	1.900	18	40.366
11	417000000	256QAM	2.200	19	38.983
12	423000000	256QAM	2.500	20	38.605
13	429000000	256QAM	2.800	21	38.983
14	435000000	256QAM	3.100	22	40.366
15	441000000	256QAM	3.300	23	40.366
16	447000000	256QAM	3.300	24	40.366
17	555000000	256QAM	0.000	25	38.605
18	561000000	256QAM	-0.600	26	38.605
19	567000000	256QAM	-0.500	27	38.605
20	573000000	256QAM	-0.500	28	38.983
21	579000000	256QAM	-1.900	29	37.636
22	585000000	256QAM	-1.300	30	38.983
23	357000000	256QAM	2.400	9	38.983
24	597000000	256QAM	1.500	32	38.605
25	657000000	256QAM	1.300	41	38.983
26	663000000	256QAM	1.400	42	38.983
27	669000000	256QAM	1.900	43	40.366
28	675000000	256QAM	2.100	44	40.366
29	633000000	256QAM	1.000	37	40.366
30	639000000	256QAM	1.000	38	40.366
31	645000000	256QAM	1.300	39	38.983
32	651000000	256QAM	1.500	40	40.946

OFDM Downstream Overview

Receiver	FFT type	Subcarr 0 Frequency(MHz)	PLC locked	NCP locked	MDC1 locked	PLC power(dBmv)
0	NA	NA	NO	NO	NO	NA
1	NA	NA	NO	NO	NO	NA

Upstream Overview

Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Bandwidth
1	23700000	ATDMA - 64QAM	35.000	2	6400000
2	38596000	ATDMA - 64QAM	38.500	3	3200000
3	30596000	ATDMA - 64QAM	35.000	1	6400000

OFDM/OFDMA Overview

Channel Index	State	lin Digital Att	Digital Att	BW (sc's*fft)	Report Power	Report Power1_6	FFT Size
0	DISABLED	0.5000	0.0000	0.0000	-inf	-1.0000	4K
1	DISABLED	0.5000	0.0000	0.0000	-inf	-1.0000	4K

[Breadwinka]

Looks like im missing a few channels. DOCSIS WAN

This menu displays both upstream and downstream signal parameters

Downstream Overview

Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Signal noise ratio (dB)
1	591000000	256QAM	3.700	31	38.605
2	657000000	256QAM	4.700	41	36.387
3	663000000	256QAM	5.200	42	36.387
4	669000000	256QAM	5.500	43	36.387
5	381000000	256QAM	1.800	13	38.983
6	387000000	256QAM	2.200	14	40.366
7	393000000	256QAM	2.600	15	40.366
8	399000000	256QAM	3.000	16	40.366
9	405000000	256QAM	3.100	17	40.366
10	411000000	256QAM	2.900	18	40.366
11	417000000	256QAM	2.900	19	38.983
12	423000000	256QAM	3.000	20	38.983
13	429000000	256QAM	3.100	21	38.983
14	435000000	256QAM	3.200	22	40.366
15	441000000	256QAM	3.100	23	40.366
16	447000000	256QAM	3.300	24	40.946
23	675000000	256QAM	6.100	44	36.387
27	615000000	256QAM	4.000	35	37.356
28	621000000	256QAM	6.400	36	38.983
29	633000000	256QAM	5.000	37	38.605
30	639000000	256QAM	5.700	38	38.605
31	645000000	256QAM	5.500	39	37.636
32	651000000	256QAM	4.400	40	37.636

OFDM Downstream Overview

Receiver	FFT type	Subcarr 0 Frequency(MHz)	PLC locked	NCP locked	MDC1 locked	PLC power(dBmv)
0	NA	NA	NO	NO	NO	NA
1	NA	NA	NO	NO	NO	NA

Upstream Overview

Port ID	Frequency (MHz)	Modulation	Signal strength (dBmV)	Channel ID	Bandwidth
1	38595687	ATDMA - 64QAM	41.500	3	3200000
2	30596000	ATDMA - 64QAM	38.000	1	6400000
3	23700000	ATDMA - 64QAM	36.500	2	6400000

OFDM/OFDMA Overview

Channel Index	State	lin Digital Att	Digital Att	BW (sc's*fft)	Report Power	Report Power1_6	FFT Size
0	DISABLED	0.5000	0.0000	0.0000	-inf	-1.0000	4K
1	DISABLED	0.5000	0.0000	0.0000	-inf	-1.0000	4K

[Datalink]

Yup, time to swap the modem for a "black dot" version modem.   That black dot will be present on the outside of the packing box and near the power cord on the modem. 

[John86]

You can contact me re the below speed degradation issue noted in your post:

"2.4 GHz WiFi Degrades speed

There has been reports in the last few days that disabling 2.4 GHz WiFi can make a difference on the wired speeds observed (by 200-300 Mbps). I suspect that this occurs on modems that are impacted by interference (which we are working on a replacement program in the next 2-3 weeks) but want to be sure that there isn’t another underlying issue.

If you observe this behavior and are available to run some tests, let me know. I will be looking for a case where cable signal is within specification and there are no sign of interference on your modem."