The solution is density. Politicians talk a big game when it comes to affordable housing, but nobody has the cajones to take on the neighborhood groups who demand low density low rise buildings with big setbacks and expensive public amenities.
I agree to this in some aspects. i think density and or significant rezoning of land to allow for new stock is the away to alleviate housing affordability pressures. but, i don't think that necessarily means we need to throw up 20-30 floor buildings ad hoc a la some aspects of vancouver's planning (some of which i like, but their penchant for all of the sudden just tossing up a collection of 30 floor buildings is poorly thought out in my opinion). it can be as simple as saying on certain well utilised arterial roads you enable every site (subject to whatever heritage restrictions) to be 5 stories with no setback on the first three floors and the top two floors set back to avoid unnecessary overshadowing onto the street.
it means that you might have something like cook street in cook street village or quadra in quadra village all up to 5 stories. suddenly you have way more density than you currently have and are still able to maintain that village ambiance that nimby's and all people really, love so much.
the alternative is that you probably go super dense (15-30 floors) just in select areas of downtown. personall, for victoria, i think the limits of up to 20 floors are good, but something has to give and i think that is increased (though still comparatively low) density in the 'villages' around the downtown core.