There's something about Fifth Avenue. It's certainly stately, old, and historic, but it's also incontrovertibly part of the constant evolution that is New York. Start way uptown (I'm talking in the 100's) and hit the Museum of the City of New York and El Museo de Barrio. Beautiful places in authentic neighborhoods. The Conservatory is also right there (103rd), so you can have a romantic picnic that is unlikely to be interrupted since the area is rather quiet.

Then, of course, you have to go to the Met, that grand institution of art. I recommend you skip the European paintings (unless you're a tourist) and go to the Temple of Dendur (if you appreciate breathtaking sunlight) and the Asian art collection (first rate, and very quiet to boot).