So, can you install routers in all of those rooms? That depends on what mode the modem is running in and what mode the routers are running in. The simple answer is yes, depending on the answer to those questions.
Typically you would do the following:
For the modem running in Gateway mode:
Incoming =====> modem =====> modem port 1 =====> cable to room 1
cable (Gateway) modem port 2 =====> cable to room 2
(mode) modem port 3 =====> cable to room 3
modem port 4 =====> cable to room 4
This assumes that the modem is located in or near the structured wiring cabinet where those cables to the rooms originate. This is usually not a good location for any wifi networks from the modem, or router if its located at the same location. Note that if you wanted to add more rooms, you would need a co-located gigabit switch to allow that.
So, you could in theory connect individual routers to those cable ports in the rooms where they would operate their own networks. If they are running in full router mode, then you would have a double Network Address Translation (double NAT) situation running, where the modem and routers interpret the external address to the internal address. You can do this but its not a good situation for gaming or any port forwarding requirements. You could also run those routers in Access Point mode where the modem is actually running the network and the routers provide access to the overall network.
The other alternative is to run the modem in Bridge mode:
Incoming ==> modem ==> modem port 1 ==> cable to room 1 (both IPV4 and IPV6)
cable (Bridge) modem port 2 ==> cable to room 2 (both IPV4 and IPV6)
(mode) modem port 3 ==> cable to room 3 (IPV6 only)
modem port 4 ==> cable to room 4 (IPV6 only)
In this case each router that is connected in those rooms would have full unimpeded access to the internet and would have to run in full router mode. Those routers would be responsible for their own networks only and would not provide access to any other networks in the house. The drawback to this is the limitation of the modem in Bridge mode. Only two ports of the modem will receive both IPV4 and IPV6 addresses. The other two will receive an IPV6 address only. Since a good many sites only run or use IPV4 addressing, the two IPV6 only routers would only be able to provide limited internet access to the external world, but, the list of web sites that run IPV6 is growing all of the time.
So, the modem and router would probably be co-located somewhere where the router can provide adequate wifi coverage for the home, and would connect via house ethernet to the gigabit switch. The gigabit switch in this case is an 8 port gigabit switch, where one port is connected to the router and the other 7 ports connect to various rooms in the home.