My experience is that while LLVM is very modular, it also has a pretty high amount of change in the boundaries, both in where they're drawn and in the interfaces between them. Maintaining a fork of LLVM with a new back-end is very hard.
Various reasons, like embargoes on information, stuff we didn't want to wait for review on before shipping, or features that don't make sense for upstream like `hipcc` which is an `nvcc` wrapper.
Our goal is to get most modifications not in the third category into upstream at some point which makes the maintenance load bearable.
I should have qualified: it's hard to do for an individual or very small team as a passion side-project. It's pretty time consuming to keep up with the rate of change in LLVM.