The Financial District is weird, but not necessarily unsafe. It's pretty dead at night so I wouldn't go traipsing around after midnight. I would try to live as close to South Street Seaport if I were you, but no part is "unsafe" compared to the rest of New York.
