Does anyone know where I can buy a fishing license in Atlanta?

I think you can get them at Kmart. You might also be able to purchase them at your local state park.

Here is a link that might help you find additional locations for fishing licenses: http://georgiawildlife.dnr.state.ga.us/content/licenseagents.asp
You might also be able to buy them online (if you don't mind shopping on the web): https://www1.ga.wildlifelicense.com/start.php

Hope this helps.