XB7 in Bridge Mode to 2.5Gbps router - speed tests 300Mbps through port 4, 935Mbps through ports 1-3

[tenly2000]

I have a Mikrotik router with a 2.5Gbps port that connects to the XB7 and a 10Gbps fiber connection to a 10Gbps switch.

On the XB7, there are 4 ethernet ports.  Ports 1 to 3 are 1Gbps.  Port 4 is the only one capable of 2.5Gbps.
If I plug my router into any of the 1Gbps ports on the modem, I get 935Mbps speed test results - but when I move the router connection to port 4 (the 2500Mbps port), my speed test results drop to a pitiful 300Mbps instead of rising to the 1.35Gpbs I expect.
 

This appears to be because the modem is not properly advertising that port 4 is capable of 2.5G-baseT .  In the below output from my Mikrotik Router, the "link-partner-advertising" string does not contain the 2.5G-baseT speed (although the rate does get set/negotiated to 2.5Gbps)? 

This same home network and router setup all worked fine when I was with Bell's Fibe and I was able to consistently get speed tests of 1.35Gbps from my MacBook - but Roger's support is telling me that the problem "must be" with my Router because their modem is "working fine" (eye roll).

Has anybody else encountered this issue ?  Does anyone have an XB7 in bridge mode with a Mikrotik router that is working fine?  Do I need to demand an XB8 - or is there some setting I can tweak on my router or in their modem to make this work?  The router is running the latest firmware and the Rogers support rep also informed me there are no firmware updates available for the modem and that it is properly provisioned for 1.5Gpbs.

[admin@MikroTik] > /interface ethernet monitor ether1                 
                      name: ether1
                    status: link-ok
          auto-negotiation: done
                      rate: 2.5Gbps
               full-duplex: yes
           tx-flow-control: no
           rx-flow-control: no
                 supported: 10M-baseT-half,10M-baseT-full,100M-baseT-half,100M-baseT-full,1G-baseT-half,1G-baseT-full,2.5G-baseT
               advertising: 10M-baseT-half,10M-baseT-full,100M-baseT-half,100M-baseT-full,1G-baseT-half,1G-baseT-full,2.5G-baseT
  link-partner-advertising: 100M-baseT-half,100M-baseT-full,1G-baseT-half,1G-baseT-full

Additional info:
I have a NAS with dual 2.5Gbps ports and a MacBook with a 2.5Gbps adapter.  The NAS is plugged into a 5-port 2.5Gbps switch which itself is plugged into the 10Gpbs switch.  The MacBook is plugged directly into the 10Gbps switch.  Speedtests between the MacBook and the NAS server using iperf3 show 2.35Gpbs in both directions which proves the configuration of the network adapters on my devices and switches are working properly.  I say this to try and head off any suggestions that there may be a misconfiguration on my devices. The problem is definitely between my router and the XB7.
 

[-G-]

When you log into your Ignite Gateway, what do you see as the connection speed under "Gateway > Hardware > LAN Ethernet"?

The XB7 should be capable of attaining speeds in excess of 2 Gbps.  Comcast says that both the XB7 and XB8 can support a data throughput of 2.5 Gbps.

What system are you using to perform the speed test?  Your Mac?  Have you tried performing a speed test with any other device, even one with a comparatively slow Gigabit Ethernet connection?

[-G-]

I also recall seeing people with Microtik routers reporting a similar problem a few years ago where they were seeing slower speeds when connected to the 2.5 GigE port of an XB7.  If I remember correctly, it was due to some hardware incompatibility issue with their SFP module where it could not get a stable connection at 2.5 Gigabits.

[-G-]

FYI, found these posts on reddit:

https://www.reddit.com/r/mikrotik/comments/r80427/rb5009_xfinity_xb7_cgm4331com_slower_at_25g_vs_1g/

https://www.reddit.com/r/mikrotik/comments/rq7ytu/rb5009_25gbe_slower_than_1gbe/

People were also reporting similar issues with Ubiquiti gear where their SFP could get a stable connection when connected to GigE and 10 GigE switches but not 2.5 GigE.  Again, from what I remember, it took quite some time to find an Ethernet SFP module that played nicely with their router AND could connect to a 2.5 GigE switch.

[-G-]

@tenly2000  Was also thinking about what could happen with hardware acceleration and offloading, queuing strategies (in hardware and software), etc. when the line rate increases.... things that are very specific to Microtik and hardware-dependent... things that come into play when you have hardware that is technically capable of forwarding packets at high data rates but has a relatively slow CPU.

I found the following post on the Mikrotik support forum: https://forum.mikrotik.com/viewtopic.php?p=1068203#p1067463

[tenly2000]

Thanks for your replies and all the links. I did read them all and some additional posts that they linked to. My Router is the RB5009 that they all seem to mention - but I haven’t really seen anyone complaining about the issue with the XB8, only the XB7.

If I tell my router to stop advertising its port can do 2.5G, then they auto-negotiate1Gbps and everything works properly at that speed (speed test at 935Mbps). But when I advertise 2.5G, the modem UI shows 2500Mbps for the port and the microtik displays the negotiated rate at 2.5Gbps - but the speeds are only 300Mbps and the Mikrotik shows that the modem is not advertising its 2.5G capability - so I don’t know how they manage to negotiate that rate but I suspect that it’s part of the problem.

I don’t know if it will work properly with an XB8 - but I do know that it would work properly with Bell Fibe. So, if Roger’s will exchange my modem, I’ll try it - but if they can’t or won’t give me something that works, I guess I’ll be going back to Bell. Their equipment definitely works with my hardware at full speeds. I’ve seen the 1.4Gbps speed tests from my MacBook to the internet 8 months ago when I had Bell.

In any case, I do appreciate your replies and the information you provided. Thank you.

[-G-]

@tenly2000 The XB8 has a Wi-Fi 6E chipset and a slightly different internal antenna arrangement but, other than that, I'm pretty sure that it has the same SoC as the Technicolor variant of the XB7 and that their underlying software is built from the same code base.

Also, looks like Microtik testers are reporting the same issue with the XB8: https://www.reddit.com/r/mikrotik/comments/zwzs0h/multigig_oddity_with_a_rb5009_and_crs317/

[-G-]

@tenly2000  One other thing to think about: with the Bell Fibe service, you were (almost certainly) running PPPoE, which would have disabled hardware offloading on your WAN interface and would have put WAN traffic on a different forwarding path.

I don't know the Mikrotik hardware very well but I think that there is a fast forwarding path that is somewhere between total offload and exclusively processor-switched.  I would try disabling hardware offload on the WAN interface just to see what speeds you can attain in that configuration.