*** This post was last edited May 2, 2017 ***
Good morning Community,
As I mentioned in a post two days ago, we have received the next firmware 2.0.10.20 from Hitron. We are currently running initial testing on this version and will push it out to participants in the firmware trial program as soon as it passes initial testing.
However, while running these tests, we discovered abnormal behavior with ICMP and are awaiting feedback from Hitron today to asses how this will be addressed. As soon as I this is confirmed, I’ll update the change log with the correct version information and start pushing it out.
In parallel, we are still working on the following high priority items. In some cases below, I requested affected customers to reach out to me via private message. If you do so, please include your modem MAC address in the subject line (even if we exchange messages daily) as there are a lot of you reaching out to me daily 🙂
UDP Packet Loss
The investigation for what has been reported as UDP packet loss is still ongoing. We have deployed a probe at one fellow forum member on both a CODA-4582 and a CGNM-3552 to collect additional data. We are actively working with Hitron and Intel on the results observed.
Based on what we know so far, in most instances UDP packet loss is coupled with higher uplink usage in the area. Although the impact is noticeable in specific logs (League of Legends), the root cause for the perceivable impact (while playing) is likely related to bufferbloat (see next issue).
Bufferbloat
When comparing the performance of a CODA-4582 to a CGNM-3552 in the same network conditions, the CODA-4582 consistently reports higher bufferbloat when tested on DSLReports.
Update April 12: The solution for this problem will come in two folds. It will require a change in software which will possibly be included in 2.0.10.27 but more likely in 2.0.10.28 and a change in network configuration.
The network configuration change is not compatible with the current firmware so this change will only come after a vast majority of the modems are running the new code. We are however looking at a way to make the change only for specific modems to support testing in the community.
Update April 22: This problem seems resolved in firmware 2.0.10.27
5 GHz WiFi Low range for channels 36 to 48
Lower WiFi channels on the modem have a much smaller range. This is due in part to the limit imposed by Industry Canada to maximum transmit power.
Furthermore, the current automatic channel selection (auto mode) tends to select the lower channels when in similar load conditions.
Workaround: manually select higher channels (149-153-157-161)
Update April 22: The channel selection algorithm has been improved in firmware 2.0.10.27
Loss of OFDM Channel Lock
Under some RF conditions, the modem fails to lock properly on the OFDM channel. This typically result in variable performance.
Update April 12: This problem is resolved in 2.0.10.26T2
List of connected device does not get fully populated
This is a known issue that has been tracked since firmware 2.0.10.13. We are making improvements at every firmware but it is not a perfect system.
The situation is worst after a reboot or firmware upgrade as the list gets reset and must be repopulated as devices renew their DHCP lease.
NAT Loopback not working for wired clients
When setting up port forwarding to an internal server, it is possible for a client on WiFi to reach the server using the external IP/port. If the client is on a wired interface, it doesn't work.
Update April 12: This problem is resolved in 2.0.10.26T2 (not confirmed)
LAN Counters not working
Some customers reported that LAN counters (especially in bridge mode) are reporting inaccurate values.
This problem has been reported to Hitron for investigation.
Unexpected modem reboot
Some customers reported their modem reboots unexpectedly. We have also seen this behavior in our lab.
Update April 12: This problem is resolved in 2.0.10.26T2
Missing SC-QAM Channels
After a reboot, some modems are missing SC-QAM channels. A fix has been implemented in 2.0.10.26T2 to address this behavior but it has not corrected all scenarios.
Investigation continues with Hitron.
WiFi Survey
The WiFi Survey functionality in firmware 2.0.10.26T2 (and possibly before) reports incorrect SSID names.
Guest Network
When connecting to the Guest Network, an error message is displayed "only allow DHCP client to use this wireless". This has been reported in firmware 2.0.10.26T2.
Update April 22: This issue has been resolved in firmware 2.0.10.27
Update May 2: It seems this issue is not fully resolved and still experienced by some users
Future Planned Improvements
The following are items that we are working on in parallel of the above.
- Improvement in WiFi speeds
- Improvement in latency / bufferbloat
Dave
*Edited Labels*
@Telek the iperf numbers are definitely strange. I can repeatedly run up to 500 Mb/s UDP down without errors. From 500 Mb/s up, the bandwidth and datagram numbers start to diverge between the received data and the server data, but, the loss result will indicate no losses. And then you get to a tipping point and all of a sudden you see losses in the results. I don't know why that happens, so, I have some reading to do and further testing to do. My definition of success at this point with an iperf test is to end up with no errors and see a close association between the local data and the server data in terms of bandwidth and datagrams. It doesn't make any sense to me at this point to see those last two data points diverge and yet see no loss declared by iperf. So the numbers don't have to be exact, but, they should be pretty close unless someone with expertise in iperf tells me otherwise, as in "ah, its because of ........." as I nod my head in agreement.
What I was trying to highlight in that first section was the strange iperf behaviour where on one run I might see errors as you indicated, and yet, on the next run, it would turn out perfectly. I didn't do anything different, same command, and yet the results are drastically different.
If you look at the next section below that, running at 590 Mb/s, you can see that there are no errors. Thats the max Mb/s point that I saw today where I would see no lost datagrams. If I went to 600 Mb/s down, I would end up with lost datagrams and you would see that in the results and in the % loss.
In that section there are no errors, but, if you compare the bandwidth figures, bottom server data indicates 590 Mb/s, top receive data varies between 542 to 582 Mb/s. The datagram numbers also have diverged at this point, but yet, iperf is still happy to say that there are no errors. So, this is a little strange.
"why would 530mbit of UDP max out your high end i7 processor? What sort of processing is it doing on this??" I wish I had that answer. My immediate thought is that UDP doesn't follow the send and acknowledge routine. Its a firehose with the iperf test, so, you either catch it or you don't. Why is that any tougher than a much faster TCP/IP speedtest that I also run? I don't have that answer at the moment. One thing that I have to look at is the processor loading to see if the load stays on one processor or if it hops around. But, as I indicated, I'm not sure that I can do anything about it with the windows version of iperf.
That buffer size is actually the socket buffer size. What I found today is that if I didn't include that variable and increase the buffer size, I wouldn't have been able to run the UDP test at higher rates. That leads to the question of whether or not an end user would have to increase that buffer size for games or any applications that use UDP, or, are they self adjusting?
A Core 2 Duo, have one of those as well, what I'm typing on actually, and it runs rather well. A little slower on a speedtest since Windows 10 but it still runs rather well. My golden oldie Core 2 Quad Q6700 with its DDR2 runs 50 Mb/s up, 200 Mb/s down, UDP, without any errors. Thats with the same settings as before, just throttled back on the download to get to an error free point. That is also IPV4 only thru my Asus RT-AC68U.
If nothing else, today showed that there are more questions than answers. The modem looks like it can run fast UDP up and down, the question is, is there an approximate 590 Mb/s cap on the modem, or is that just a pc problem? That might take an overclocked CPU and 10 Gb/s nic card to determine that.
Maybe @RogersDave can explain some of the pitfalls of UDP.
The one thing that was demonstrated was the UDP IPV4 performance versus the UDP IPV6 performance. The IPV6 results are vastly superior to the IPV4 results, and the only explanation I can think of are either a NAT problem or some other issue that hasn't been identified as of yet. Perhaps looking at this would benefit the Puma 6 modems as well.
@JohnyR wrote:I was told by an agent that -10 to +10 is acceptable on the downstream. Upstream should be 35-58 according to him as well. Not sure how true these values are, especially since the CODA handles it a bit differently than the Rocket modems.
@JohnyR, I can't be sure of easily of what happened during that 3 hour or downtime you experienced but I suspect it was a segmentation activation (when we break an area in half to increase capacity).
I checked your signal levels and everything is fine on the downlink but your uplink is out of spec. This also affects your neighbors so I'll have maintenance dispatched to make adjustments to the line amplifiers on the street.
For reference, 35 dBmV to 53 dBmV is the acceptable signal level for upstream for a DOCSIS 3.0 modem. The levels for a DOCSIS 3.1 modem are much lower because of a correction factor that is dependent on the channel bandwidth:
3.2 MHz channel: 32 dBmV to 50 dBmV
6.4 MHz channel: 29 dBmV to 47 dBmV
In your specific case, the channel at 23.7 MHz (6.4 MHz wide) is too low at 27.75 dBmV. The channel at 30.596 MHz (6.4 MHz wide) is right at the limit at 29.00 dBmV.
Dave
Runnng a Factory Reset with trial firmware loaded in the past didn't result in a rollback. There is probably a technical reason this time around as to why .27 wasn't protected when a Factory Reset was run.
Ah, yes, I am positively, absolutely, 100% sure, after running numerous trial versions preceding .24 and .26. It wasn't standard procedure to roll back to a previous version if a Factory Reset was run. If a roll-back to a previous version happens, its due to some technical reason specific to the version in question. If Dave decides to make that standard procedure, ok, it is what it is. Until the recent change, my standard procedure was to run a Factory reset after a trial version was loaded, coming out of the reset and reboot with the trial version intact on the modem, so, yes, I'm sure.
-
473803 views
-
295 Likes
-
-
