We now have scientific evidence that talking on the phone while driving is more dangerous than chatting with the person in the car with you:

A cell phone conversation
cuts through a driver’s concentration and performance on the road in
ways that a chat with a front-seat passenger does not approach,
researchers here reported.

Drivers relating an event on a hands-free cell phone were four
times more likely to miss their exit compared with drivers who told a
similar story to a passenger (P<0.05), Frank A. Drews, Ph.D., of the
University of Utah, and colleagues reported in the December issue of
the Journal of Experimental Psychology: Applied.

Those who chatted on their cell phones were also significantly
more likely to drift in their lane (P<0.05) and fail to keep a
greater distance between their car and the one in front of them
(P<0.05).

The reason for this conclusion is clear:  The passenger is able to
better meter and negotiate the safety dyad between road and driver
while the cellphone caller has no idea of the circumstance of the
road.  In the cellphone conversation, the driver is forced to live in
the negotiated, neutral, ether of the cellular communication rather
than dealing with the immediate danger of the road facing driver and
passenger.