*** This post was last edited May 2, 2017 ***
Good morning Community,
As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.
However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.
In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂
UDP Packet Loss
The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.
Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).
Bufferbloat
When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.
Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.
The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.
Update April 22: This problem seems resolved in firmware 2.0.10.27
5 GHz WiFi Low range for channels 36 to 48
Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.
Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.
Workaround: manually select higher channels (149-153-157-161)
Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27
Loss of OFDM Channel Lock
Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.
Update April 12: This problem is resolved in 2.0.10.26T2
List of connected device does not get fully populated
This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.
The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.
NAT Loopback not working for wired clients
When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.
Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)
LAN Counters not working
Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.
This problem has been reported to Hitron for investigation.
Unexpected modem reboot
Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.
Update April 12: This problem is resolved in 2.0.10.26T2
Missing SC-QAM Channels
After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.
Investigation continues with Hitron.
WiFi Survey
The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.
Guest Network
When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless". This has been reported in firmware 2.0.10.26T2.
Update April 22: This issue has been resolved in firmware 2.0.10.27
Update May 2: It seems this issue is not fully resolved and still experienced by some users
Future Planned Improvements
The following are items that we are working on in parallel of the above.
- Improvement in WiFi speeds
- Improvement in latency / bufferbloat
Dave
*Edited Labels*
@Mike2011 wrote:
I checked this morning and I'm still on .24 firmware.
Is there something special that needs to be done to obtain the .26 ?
Fixed. I didn't have your latest MAC but I manually pulled it and pushed the firmware to your modem.
Dave
Hello Community,
Time for some quick updates on the status of the CODA. I have not updated the first post yet, too much going on but I’ll get to it eventually.
Based on the limited feedback received, it seems that firmware 2.0.10.26T1 resolved the issues around random reboots, at least for most users. I’d like to get additional feedback on this if possible.
I also had a discussion today with Hitron regarding modems with missing channels. In most cases for DOCSIS 3.1 enabled area, this will have very limited impact but it still requires a fix. It is not a physical problem with the modem but an issue where the modem decides too quickly at boot to exclude some channels. I should be getting another test build really soon with a fix for that problem.
This will leave us with the following 2 critical issues: UDP Packet Loss and WiFi performance and a few other minor issues.
Regarding WiFi performance, we made a configuration change on all CODA modems (takes effect after a reboot) to resolve Android phones (especially Pixel), frequently disconnecting from WiFi. This seems to have dramatically improved the performance and we are considering porting this fix to the CGN3 “AC” modems as well.
We are still validating if the difference in performance between lower channels and upper channels on 5 GHz is normal. Industry Canada allows much higher transmission power on channels 149-153-157-161 compared to 36-40-44-48 which is why I recommend manually setting the 5 GHz channel to the upper band. We are however validating that the lower band is operating as expected.
The UDP performance is still our top priority but we haven’t made much progress at this point. Multiple employees from Rogers and Hitron in Toronto, Denver and Taiwan tried to reproduce this problem without success. We are at the point where we will need assistance from the community to collect live logs in person from an impacted modem while the issue occurs. I will reach out in private to one or two of you who previously provided me logs for this issue. I need to run a series a tests firsts to select the best candidate for this and I will reach out in private to coordinate further testing.
Dave
Wireshark is a good candidate. But, the log files can become very large if the data capture runs for something like tens of minutes. You probably don't have to let the data capture run for very long in order to record sufficient data.
The file size becomes more of a problem if you are asked to send it somewhere. Beyond a certain file size, and I don't know what that is, you would have to be able to upload the file as it would probaby blow up an email inbox. So, if the file size isn't very large, you can simply attach it to an email.
Take notes on what game you are running, whether the modem is in Gateway or Bridge mode with a follow on router and whether or not you have port forwarding enabled with the ports set as per the game requirements.
After looking at a League of Legends data capture and from running UDP ping tests to pingplotter.com, I'm not convinced that the UDP problems are all attributable to the modem. I think the data path from the start point to the end point might be playing a big part in the UDP problems, but, that's for Dave to confirm or deny.
Its not just a matter of the ping time, its also a matter of whether or not the UDP packets are arriving at their end destination. If you run a UDP test to pingplotter.com, you will see an ICMP destination unreachable response from pingplotter for those UDP packets that actually arrive at their server. For the packets that time out along the way, you will see a Time to Live Exceeded response from the enroute servers. Pingplotter processes both as a response and plots them. The reality is that the packets that time out should probably be considered as failures as they don't arrive at the pingplotter server. If you run a game which uses UDP, look carefully at the Wireshark Internet Protocol Version 4 Identification field number and see if the inbound packets are sequential and increment by "1" for every packet recorded. I suspect that you might find that the packets are not sequential which to me suggests missing packets, but, I need Dave to confirm my thoughts on this one. If there are missing packets that brings up a number of possibilities ranging from network timeouts enroute to issues with the modem. Looking at Wireshark LOL and chat server records, the LOL is missing packets (I believe) while the chat server, which runs off of a separate server looks like it delivers all of its packets, based on the ID number. So, thats rather interesting, but this requires confirmation at this point. Why do a good number of LOL packets fail to arrive while the chat server packets all arrive? So, as I indicated earlier, I'm beginning to think that there is more to this than just the modem.
Yup, thats a little interesting allright. The question is, with LOL, is the gapped ID intentional or the result of data path and/or modem issues. Whats normal? To know that, one would need a modem that works without any UDP issues to use as a comparison device, or as you suggest, run your own server which gives you access to both sides of the transmit/receive records.
-
506752 views
-
295 Likes
-
-
