for worker, you might have to track the number of jobs in your queue. for dyno, you might have to track the number of requests you get within a time period.
This is such a good and obviously needed idea that it makes me want to run out and implement it tonight. (No guarantees that I'll do it, but I'm 100% sure someone will.)
You're missing an important detail: Start using the add-on in the Heroku app. Since the add-on provides feature x, you can simply depend on it and remove feature x from the Heroku app :-)
Way ahead of you. The tricky part is being able to bootstrap if the service crashes. It's pretty neat being able to basically provide artificial respiration for your web app in an emergency.
Definitely. I was just playing with kensa, which is a tool that they wrote for automatically testing your add-on. It checks to make sure everything is working properly. This includes your settings manifest, provisioning services, deprovisioning services, making sure the add-on is secure, etc.
I'm kind of disappointed at the "neutral marketplace" philosophy. I'd prefer a "curated marketplace" where the Herokuans could be opinionated about what works best (in general, and with Heroku). Metrics are great but I actually value the opinion of the people who built and run Heroku more. (Of course their opinion could be a metric, but it didn't sound like it in the announcement.)
Credit goes to the guy who got stuck with the $1300 Heroku bill... he inspired this idea.
link: http://news.ycombinator.com/item?id=1688904