I was thinking that you need to do that everytime refs change, but it's just a boost and I presume you can have some refs in the commit-graph and some not in there and it will still work, so you're probably right.