It’s only currently possible to start one in 7 states and the District of Columbia. So avoid Colorado, Iowa, Kansas, Montana, Oklahoma, Rhode Island, Texas, New Hampshire, and Utah. Though New Hampshire only accepts them for probate as a spouse can inherit without paying taxes etc.

Even in the remaining 6 states you generally have to represent yourselves as married in some context like filing taxes jointly. https://en.wikipedia.org/wiki/Common-law_marriage_in_the_Uni...

PS: California does accept common law marriages formed in another state.