Using SQLAlchemy has been my biggest (technical) regret in my current project; it was great at first, but overtime it's made testing a lot harder and session/object management has led to subtle bugs.
Getting Flask/Pyramid + SQLAlchemy + pytest to play nicely with sessions, transactions and rollbacks has often been the one thing that made me go "screw it" and start over with Django.
I feel like it’s a great tool to have available because it has solutions for everything you might want, but I found myself preferring it more for scripting than for application servers. Never quite figured out how to properly handle database sessions, really.
Speaking from experience, session/object management is always tricky with ORM's. SQLAlchemy has been good for us; but the usage has been very, very disciplined and doesn't look idiomatic.