I was thinking about this earlier today actually, when weighing the options between a VPS or dedicated box. The problem is finding a connection with a decent uplink. The two big ISPs in my area, Comcast and AT&T limit you to 768kbps, and I think both also block port 80.
And as far as I know, getting SDSL or a T1 is gonna set you back a few hundred a month, at which point you're better off getting a dedicated box.
Also, it's against the terms of service for basically all ISPs to let you run a server. They'll probably only enforce it, though, when you start getting a lot of traffic -- this means that you'll be turned off right as you are starting to gain traction. That's a bad time to have 404s. Also, what happens if the power flickers? Internet goes down?
Start with a VPS -- even if 30 bucks a month sounds like a big chunk of change, consider your other overhead (rent, groceries, etc). Not a big price to pay for the benefits.
slicehost.com is the best I've seen for a VPS so far, $20 beginner. They also say they're negotiable on bandwidth starting at $0.30/GB, which is a bonus if you're running a popular blog and for some reason need more than 100GB bandwidth.
As for power, you've always got UPS'.
Internet is the tricky one as all private hosts use redundancies, although I don't see why you couldn't sign up for regular internet and a cable connection.
I've seen a handful of ISPs that let you run a server (or more correctly, don't stop you), although I think most ISPs prohibit it to force you onto business packages (which you can get in your home too).
I know in the Toronto region there is Mountain Cable, which doesn't have anything in their Terms of Service preventing you, and have unlimited bandwidth (From speed tests I've seen, they're faster than Cogeco, and get an upload speeds in the 1+MB/s range). I think you've just got to settle with DynDNS, however from what I've read this is reccomended for static IP's too. If you did have two ISPs then this would solve the IP problem when you switch.
I'm actually considering going with Mountain Cable anyway as they're cheaper than Cogeco. Although their lack of information on their website freaks me out; I mean their service area map isn't even online. (Edit: They do have a service area map for their internet, however they apparently provide service outside of the area)
And as far as I know, getting SDSL or a T1 is gonna set you back a few hundred a month, at which point you're better off getting a dedicated box.