It is. Car conversations are a serious issue with respect to distracted driving. I personally can't handle thinking about driving safely and thinking about a conversation. Humans are famously bad at multi-tasking and I don't see how being in a car gives us magical powers.
I thought I had read a study about talking to people in your car versus drinking, but here's a similar one with talking on a phone versus being drunk: https://www.ncbi.nlm.nih.gov/m/pubmed/16884056/
If I'm speaking with someone in the car and I detect a potentially dangerous situation, I immediately stop talking to focus.
It's hard to explain, but I would say my attention is unfocused and broad. I don't know if this is ultimately safe, but I've never had an issue with it.
In fact the passenger will also detect the potentially dangerous situation and stop taking focus from you. The person in the phone will not, and may continue to distract you involuntarily.
A coworker once told me that that is what makes the difference, based on some study. I immediately believed it because it seemed so reasonable, so not citation...
I've never had a problem with it, but I also don't look at them while driving, and so forth. Maybe I'm the drunk driver who thinks he can totally drive better when he's drunk, but I've been driving 20+ years and never had an issue.
OTOH, I also evaluate and will not talk if I don't feel comfortable (poor road conditions, etc).
I thought I had read a study about talking to people in your car versus drinking, but here's a similar one with talking on a phone versus being drunk: https://www.ncbi.nlm.nih.gov/m/pubmed/16884056/