It depends on the time of day. I used to drive into the Bronx and occasionally had to drive into Manhattan and it's really a pain. I like Svish's suggestion of driving to White Plains and getting the train from there. But I have to say, one of my life goals is never driving in Manhattan again because of all the traffic, and I really think the train is faster anyway.
The fastest way would be by car along the Taconic and then splitting via Saw Mill or Sprain Brook (and other further roadways!) depending on where you are going in NYC. If you want to take the train to avoid parking hassles in the city, you can drive to Croton-on-Hudson or White Plains and park your car there and then take the train in - the train ride can be anywhere from 35 to 50 minutes depending on whether it is an express or not. There are convenient paid parking meters in both these location where you can use your charge card or cash.