01-05-2017 11:03 AM - edited 05-02-2017 07:09 AM
*** This post was last edited May 2, 2017 ***
Good morning Community,
As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.
However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.
In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂
UDP Packet Loss
The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.
Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).
Bufferbloat
When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.
Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.
The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.
Update April 22: This problem seems resolved in firmware 2.0.10.27
5 GHz WiFi Low range for channels 36 to 48
Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.
Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.
Workaround: manually select higher channels (149-153-157-161)
Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27
Loss of OFDM Channel Lock
Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.
Update April 12: This problem is resolved in 2.0.10.26T2
List of connected device does not get fully populated
This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.
The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.
NAT Loopback not working for wired clients
When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.
Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)
LAN Counters not working
Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.
This problem has been reported to Hitron for investigation.
Unexpected modem reboot
Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.
Update April 12: This problem is resolved in 2.0.10.26T2
Missing SC-QAM Channels
After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.
Investigation continues with Hitron.
WiFi Survey
The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.
Guest Network
When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless". This has been reported in firmware 2.0.10.26T2.
Update April 22: This issue has been resolved in firmware 2.0.10.27
Update May 2: It seems this issue is not fully resolved and still experienced by some users
Future Planned Improvements
The following are items that we are working on in parallel of the above.
Dave
*Edited Labels*
03-26-2017 01:27 PM
No problem. I'm thinking that the cause for the bufferbloat issue is because of the Speedboost feature and the CMTS. The bufferbloat could be the exact reason why I'm dropping packets.
03-26-2017 01:49 PM
Somewhere in all of these threads is a discussion on Bufferbloat as calculated by the DSLReports speedtest, and it has to do with the initial speeds versus the speeds as seen during the remainder of the test. The result is that the slower speeds as seen beyond the initial rise is considered as bufferbloat by the test and results in an inaccurate calculation. I think in that discussion Dave pointed out that you have to set the speed for the service that you're on, which should result in a more accurate calculation. Fwiw, my personal opinion is that the DSLReports servers are, for the most part, too far away to result in a useful indication of the last mile capability, that is to say, between the CMTS and the modem, especially for gigabit service. There are only two local servers, Toronto and Montreal, every other server is fairly distant, and the server speeds in many cases are low, compared to the Rogers, Beanfield and Fibrenoire servers in Toronto and Montreal. I found that testing somewhere between midnight to 8:00 am is the only time where you might see a useful result. Outside of those times, you might as well not bother, and that includes both speedtests and the mythical "bufferbloat".
Also fwiw, I see more random packet loss with the CODA-4582 than I have with other modems which I believe is a little unusual. That is seen during pingtests to the CMTS.
03-26-2017 03:51 PM
I think we are still several years away from FTTH.. ( 5years+ if not more.. )
With D3.1 full duplex I don't see any reason to start rolling FTTH any sooner.
03-26-2017 07:01 PM - edited 03-26-2017 07:32 PM
I can't add much to what actual problem with 4582 and UDP might be, but i can say, since we switched to 3552 i have had zero issues with browsing and playing games.
Like RyzenFX has noted, the speed test on the 3552 are usually steady, on the CODA was always spikes, would hit same speeds but very inconsistent. Bufferbloat was always fail on the CODA, still some on the 3552 but much better.
03-27-2017 07:13 AM
@RyzenFX wrote:@4kgigabitguy @RogersDave @Datalink I just switched my CGNM-3552 back to the CODA-4582. I ran some test games of LoL, however it's packet loss is nowhere near the packet loss I averaged with the 3552; it's worse. I averaged 6.7 packets lost/min. However, I do have a suspicion that the CMTS is not communicating with the modem properly (i.e CMTS is not giving the modem enough upstream signal strength, which needs to be adjusted).
Upstream Signal Strength:
Upstream Overview
Port ID Frequency (MHz) Modulation Signal strength (dBmV) Channel ID Bandwidth 1 30596000 ATDMA - 64QAM 29.250 1 6400000 2 38595785 ATDMA - 64QAM 33.250 3 3200000 3 23700000 ATDMA - 64QAM 29.250 2 6400000
The uplink signal strengh on the CODA is measured in a completely different way compared to previous devices and this is due to a change in the DOCSIS specification. The power level now takes into account the bandwidth of the channel.
For a 6.4 MHz channel, you signal level will be about 6 dB lower than what would be reported by a D3.0 modem. For a 3.2 MHz channel, the power level will be about 3 dB lower than a D3.0 modem.
In the case above, this means that if you were to connect a D3.0 modem in the exact same conditions, the power levels would be 35, 36, 35 wich is perfectly fine. The modem is also instructed by the CMTS how to regulate power level so that the signal arrives at the CMTS with a value close to 0. I checked your modem and all three uplink channels are received at the CMTS at 1 dB which is perfect.
Dave
03-27-2017 07:17 AM
@RyzenFX wrote:
@RogersDave @Datalink What do you think of this? I think it makes sense, bufferbloat can be due to the speedboost feature and CMTS, but I'm really curious what both of your opinions are.
It is possible that is has an impact with some applications. If the application estimates the available speed and then uses that as a baseline for the rest of the connection, there could be negative impact.
There is however positive impact to speedboost when browsing as it makes pages a lot more responsive.
I will however "downgrade" my test package and redo some of the tests.
Dave
03-27-2017 10:01 AM
This morning, I pushed firware 2.0.10.26T2 to all trial participants.
If I missed your modem, it is because I don't have your latest information. Send me a PM with your new MAC address and I'll update my records accordingly.
As stated previously, we have update on boot enabled on the CODA-4582 which means that the configuration file that your modem downloads at boot include additional information where to download the production firmware (2.0.10.24). When I manually upgrade your modem, I set a flag to instruct your modem to ignore that configuration statement but a factory reset will clear that flag and your modem will be downgraded to production code.
If that happens, send me a PM and I'll repush the test firmware to your modem.
Dave
03-27-2017 10:19 AM
@RogersDave wrote:This morning, I pushed firware 2.0.10.26T2 to all trial participants.
If I missed your modem, it is because I don't have your latest information. Send me a PM with your new MAC address and I'll update my records accordingly.
As stated previously, we have update on boot enabled on the CODA-4582 which means that the configuration file that your modem downloads at boot include additional information where to download the production firmware (2.0.10.24). When I manually upgrade your modem, I set a flag to instruct your modem to ignore that configuration statement but a factory reset will clear that flag and your modem will be downgraded to production code.
If that happens, send me a PM and I'll repush the test firmware to your modem.
Dave
Thanks Dave I got it!
Software Version | 2.0.10.26T2 |
03-27-2017 11:25 AM
03-27-2017 12:35 PM
03-27-2017 12:38 PM - edited 03-27-2017 12:38 PM
Can we a do a test Dave, can you turn off Speedboost on my modem? so I can test and see if I get better results in gaming?
03-27-2017 01:17 PM
People that are experiencing issues with gigabit, do you guys happen to be on FTTH by any chance or coax cable?
I am on 250/20 (FTTH) and have been solid. I fear if I change my package to gigabit i could experience issues.
03-27-2017 01:22 PM
@RyzenFX wrote:
I believe he's testing for bufferbloat that affects the modem on Internet Tiers below Ignite Gigabit which contain speedboost.
I think that the bufferbloat is way more than it should be on the CODA. I made a suspicion that the bufferbloat is causing the UDP issue. I'm pretty sure it affects upload aswell, which could be the reason why my upload speed is not as consistent when on the 3552.
As far as I'm aware Upload is not speedboosted at all only downstream.
03-27-2017 01:54 PM
@Adva wrote:People that are experiencing issues with gigabit, do you guys happen to be on FTTH by any chance or coax cable?
I am on 250/20 (FTTH) and have been solid. I fear if I change my package to gigabit i could experience issues.
you have FTTH?! Nice! In that case you're probably on a totally different system then we are, so gigabit shouldn't have any problems at all, no?
What modem do you have? Is it the strange thing where the fibre is converted to coax in your house, then you just have a normal modem on that?
03-27-2017 01:56 PM
You're the first one on FTTH to speak to these forums, we are all on COAX..
What kind of modem do you have?
03-27-2017 02:08 PM
@JohnBeaudin wrote:
Can we a do a test Dave, can you turn off Speedboost on my modem? so I can test and see if I get better results in gaming?
It is not something easy to do in production. The Speedboost parameters is tied to your Tier of service.
In order to disable Speedboost on a specific tier, we have to build a new configuration file, deploy in production and assign your modem to it.
I downgraded my test modem to 150u service and I will be running some test. In parallel, I downgraded the modem from one of the Hitron engineer to 150u and tasked him to play League of Legends for the rest of the day 🙂
Hopefully, he'll see some results and will be able to collect the appropriate logs.
Dave
03-27-2017 02:09 PM
03-27-2017 02:11 PM
03-27-2017 02:32 PM
That's right. Rogers Fibre comes to my basement, then they have this converter I call it which turns the wire into a regular cable wire which is connected to my modem. I am on the old 250U package with the old advance wifi router/modem. (I do have access to the other competitor fibre as well, but I can get a better promo with Rogers)
I am just warry to signing up for gigtabit package and experience speed and other issues which folks have described here.
03-27-2017 02:39 PM
03-27-2017 02:41 PM
Well you can make him play CSGO as well
make him join official servers
make sure he runs net_Graph 3 in console so he can monitor the ping and packet loss.