There remains huge parts of downtown that could absorb much more density. Rock Bay alone could house 20,000 people if density isn't artificially stifled there.
No you are totally right, I was meaning comparatively to adjacent neighbourhoods but in terms of appropriate city building context, downtown is not dense