Although Amherst is closer geographically to Wall Street Duke would be a better choice. Amherst is a fairly small school, located on a big hill. The town of Amherst is quite charming but located way out in Western Massachusetts. Once you leave the town there isn’t much to do.
As nerdychica points out Duke has a better record with Wall Street jobs. Honestly though both schools are great.