I know of fellow sea kayakers who have used the Garmin GPSMAP 60 Series Marine Mount (around 25-30 USD). This does involve drilling holes in your deck. Personally I fear that this exposed position will eventually get it knocked off, either during rescues or by my paddle when doing a quick low brace to maintain balance.
My solution is putting the 60CS in an Aquapac, securing the Aquapac to the deck shock cords by its lanyard, and keep the unit in its bag on the spraydeck in front of me. Sometimes I use a mesh deck bag and put the unit inside.
If I need a GPS just for the tracklog and for measuring speed I use a Garmin Forerunner training GPS that I wrap around the shock cords by the velcro wrist strap.
Happy kayaking!