Southern Ontario where the bulk of the Canadian population lives has basically the same climate as Chicago or Detroit. It is cold in the winter and blistering hot in the summer with a 170-ish day growing season. If you don't consider those places to be brutally cold you wouldn't consider it that way here. In fact the areas on the north shore of Lake Ontario have a milder climate (due to lake effect) than those of upstate NY to the south of us.
I grew up in Alberta. It is cold and dry and has a short summer. I prefer the weather here even if the summers can be a bit too humid and hot for me.
Upstate NY here, and we get the worst weather in the US from Canadian winds and lake effect snow. For half the year it's always windy, always cold, always snowy, and always so, so gray.
But aside from that it's a great place to live. Land is cheap, you can get good local food anywhere, and we're safe from nearly every natural disaster you can think of. Just bring a jacket.
Seems like many Americans near the border have this perception as Canada being the 'source' of their cold weather.
In reality the bulk of the nastiest storms here in Ontario come over the lakes from northern Michigan, which is, yes, north of us. Look on a map, most of the heavily populated areas of Canada are as far south as northern California. The parts that are north of the 49th are much less populated.
Yes, periodically there is a nasty cold front coming down from Hudson's bay, but the prevailing winds and weather patterns are west to east. So most of our weather blows over from the midwest U.S.
The really nasty awful cold weather last year for example, hit the U.S. midwest harder and nastier first.
I see elsewhere in this thread you mention you live near Hamilton. Well, I work in Hamilton and commute from Niagara, and that puts you and I south of the following: Washington, Montana, North Dakota, Minnesota, and Maine, 2/3rds of Oregon, Idaho, South Dakota, Wisconsin, Vermont, and New Hampshire, 1/2 of Wyoming, Michigan, and New York.
I grew up in Alberta. It is cold and dry and has a short summer. I prefer the weather here even if the summers can be a bit too humid and hot for me.