I am usually going out with my 5 year old son. You would be amazed at the kind of trouble a kid can get into in as little as a half mile hike.
So, I usually have rain gear, first aid, water, blanket, extra clothes, camera, maps and compass, writing stuff, etc. There have been a few times when we left the pack in the car and just carried the GPS and swag. But I usually only do that when the car will be in site the whole time.