I have been having this same issue for the past month or 2 as well. I live in downtown Toronto and am on an Extreme Plus plan.
I get good speeds during the day (20-35mbps) and usually drop to about 1-2mbps in the early evenings. Sometimes even worse. Can't even watch an SD resolution Youtube clip without stalling every few seconds. This is what Rogers calls High-Speed?
I spoke with a Rogers tech at first. They asked I replace the modem at a Rogers location. I changed the SMC for the new Cisco modem which solved nothing. Tech came to look into the issue and spent 10 minutes poking around and said the problem was cabling outside and he would have to send a tech to look when the sun was out (it was the evening). He seemed uninterested in solving any sort of problem that day. Second tech came, re-cabled the entire outlet (including new sockets, cables, splitters for Rogers Digital TV) and also cabling inside the Cable box outside. At least he was helpful. This was during the afternoon so the speeds were already high. By the evening they were back to 2mbps or less and have been since.
My contract is done in December on internet and I am off contract on my my Mobile and TV. I will likely be switching all my services (Internet, TV, and possibly 2 mobile phones) to Bell Fibe, I have already been in talks with them. I am a long term customer on all 3 services and yet Rogers seems uninterested to fix the issue and keep me satisfied. I will of course be advertising my change of provider via Twitter, Facebook, Newspaper letters when I do change.