A fantastic guide written by my buddy Edwin D. He writes:

I generally only bass fish & keep to the ponds in Manhattan & Brooklyn.

1. Harlem Meer Lake (116th NE corner of Central Park) is my favorite
spot. NY Game & Fish stock it with pan fish & lots of bass. I use
small stick baits and dark green worms.

2. 72nd St. Boat House Central Park. Great walk around & man made
with big rock bottom/bank. I roll a mini spinner bait up the bank for
bass or bounce a stick bait with a bill off the bottom, up the
bank...bounce & pause.

3. Prospect Park Lake: this lake is not easy to fish as the bank is
mostly tall reeds. However, this cuts down on fishing traffic. I
walk the smaller creeks on the northeast tip of the lake for bass.

4. For some of America's best trout fishing (believe it!) you can fly
or spin fish the East branch Croton River without a car during season.
Pack your gear, waders & take the Harlem Line to Brewster and walk
down Main Street until you come to a bridge that crosses the East
Branch. Then just walk down in and start fishing. You need a DEP
Watershed Access permit, which can be obtained here http://nyc.gov/watershed.
its about an hour by train.

Places to shop:
1) I always use Cabelas for my fishing gear as there is no real
place to shop in the city that doesn't require a long train ride. I
recommend getting a nice four piece travel rod & a fishing backpack that
holds rod tubes. Makes moving from spot to spot much easier & people
on the train don't stare at you like an alien.

2) For trout only, there is a great place call Urban Angler on 25th &
Broadway. It is also the only place in NYC to buy a fishing license.
Call first & make sure the license machine is working. Excellent staff,
friendly, knowledgeable & for hire as a guide to aid with your first
trip.

3) NEVER GO to Capital Fishing & Tackle on 36th St. They only cater to
high end boat owners who use big saltwater rigs. Yes, they stock
lures/line, but its all old stuff, WAY overpriced & cheap quality.
Also, the service is HORRIBLE... I mean they are absolute jerks. I
practically have to beg them to ring me up as I usually only buy one or
two urgent items.

Hope this helps... fish on!