Boating in Harriman State Park is permitted from April 1 to November 30 from 5AM to 10PM. A boating permit is required. It is $15 and available at the ranger station at Tiorati Circle or at the Beaver Pond campgrounds. There is an additional $10 charge if you want access to lakes Welch, Stahahe, and Tiorati. When getting your permit, you must bring the boat and wearable Personal Floatation Devices (PFDs) for each person who will be on board. Children under 12 must wear a PFD at all times. Boats with electric motors must be registered with a state and have registration numbers on both sides of the bow. Gas-powered engines are prohibited. Motorized vessels must also carry a signalling device such as a whistle or horn and an anchor.
I don't know if any of the stages of this cache are on islands, but in the paperwork that accompanied my boating permit it states:
Boats must remain at least 300' from all islands, public beaches, and camp docks.
I haven't had any problems fishing from my boat less than 300' from islands; leaving your boat to search on an island might attract more attention.
I've also witnessed the Park Police writing tickets for equipment violations, e.g. lacking PFDs for all aboard the boat, no whistle, etc.
New York Boating Regs.
/my $0.02
Edit: added signalling info and link.