I haven't started my caching adventures yet, but I've been going through my house looking for SWAG.
I found some cute things. I know this one may be a bit controversial but I found a never used shot glass with my towns logo on it. I was planning on leaving it much later in my caching when I'm out of town and it's a more difficult find, that a child may not find.
I also found some "travel" themed stickers (perfect for cachers I think), and some beach themed stickers for a cash I may find close to a beach.
I have some nice candle holders, some glittery (unused) hair clips, key chains, small travel tools, I figure the more I can find and the less I can spend at first is better.
For Canadians, those Tim Hortons Rim Roller key chains woudl be a great find.