Not only the drilling, but the shoring up and walling of the tunnel. Also, people tend to be opposed to "move fast and break things" if they're likely to get buried under them.
People also tend to object to "move fast and break things" when they, and their very expensive buildings, are on top of said things and get broken as a result.
Point. Although I'd assume the majority of subway routes tend to be built under roads (also bad, but less terrible)?
Not sure what the geology of San Francisco / California tends to be like, but I believe most of the Manhattan skyscrapers are anchored down to the bedrock. Which I'd expect would require cutting and reinforcing vs just "drill on through".