To Groundspeak: Permission to set up RSS feed for geocachers

[MagicTyger]

I am surprised that the Geocaching.com system still doesn't allow cachers to subscribe to an RSS feed for their caches or their finds.

 

It's easy to implement. I could write an RSS Feed generator in a couple of days that could do just that, plus generate a KML file for the log entry.

 

I want to know if Groundspeak has any objection to me doing this, on the understanding that it's a purely non-commercial project aimed at providing a service to geocachers that (in this day and age) should already be available. I'd be happy to turn it off as soon as there was a freely available service of similar functionality on the Geocaching site.

 

The RSS feed wouldn't require any sensitive information. The details of the sites you visit, or the visitors to a cache is publicly available information which can easily be viewed by anyone. all I'd be doing would be reformatting it as xml so people can put it in their blogs, webpages, phones and other cool stuff that people like doing with RSS.

 

Are there any problems with doing something like this?

[Motorcycle_Mama]

It's good that you are asking permission before you do this, but you might want to read the TOU (specifically #5).

 

http://www.geocaching.com/about/termsofuse.aspx

[MagicTyger]

It's good that you are asking permission before you do this, but you might want to read the TOU (specifically #5).

 

http://www.geocaching.com/about/termsofuse.aspx

 

Section 5 says:

 

5. Access and Interference

Much of the information on the Site is updated on a real time basis and is proprietary or is licensed to Groundspeak by our users or third parties. You agree that you will not use any robot, spider, scraper or other automated means to access the Site for any purpose without our express written permission. Additionally, you agree that you will not: (a) take any action that imposes, or may impose in our sole discretion an unreasonable or disproportionately large load on our infrastructure; or ( interfere or attempt to interfere with the proper working of the Site or any activities conducted on the Site or other measures we may use to prevent or restrict access to the Site.

 

I don't plan to do anything unless Groundspeak says it's ok.

 

As far as overloading the server goes, that's not going to happen. With intelligent caching, you'd only need to visit a user's log a max of once per day, and then only if requested. Same for a feed on a cache. Any subsequent requests that day would be cached. The bandwidth would be minimal.

[+mtn-man]

Still most likely unacceptable though. If you do it, others will want to. You should email the contact at geocaching email address to ask for permission.

[+Michael]

And that address is Contact@Groundspeak.com

[+The Last Airbender]

How about we discuss the merit of the idea then? Because I think it is a brilliant idea.

[+mtn-man]

That's why I moved it here and did not close the topic. Have at it.

[+Glenn]

How about we discuss the merit of the idea then? Because I think it is a brilliant idea.

 

This has already been discussed before but here goes.

 

RSS feeds are excellent for websites like News websites, Blogs, and Forums. When a new story, article or post is posted it is then sent to all the subscribers. Some websites have categories (World, Local, Late Breaking, Food, Investment, etc) an RSS feed is need for each category. RSS was developed to distribute posts from a relatively small number of categories to a large number of subscribers.

 

How to apply this to a website like Geocaching.com. Since Geocaching.com is world wide and not everyone cares when someone logs a cache that isn't in their home area then there will need to be more than one category. Do you divide the world up in to cities? What about large cities? Do you divide the city up even further while also providing a whole city feed? What about rural areas? What about areas that are divided by natural boundaries like rivers, lakes, and mountains? That is a lot of categories and we haven't even gotten to some of the sub-categories like division by cache type, terrain, difficulty, etc. Now how many people will actually sign up for the RSS feeds. In the citys maybe a few hundred but most likely when you average everything out the RSS feed to subscriber ratio will be very low when compared to other websites.

 

Will RSS work for a website like Geocaching.com? I don't see why not. Is RSS an efficient way of aggregating new cache notifications and new cache logs. No, not by a long shot. RSS is efficient at sending occasional posts from a small number of categories to a large number of subscribers. RSS totally sucks at sending a large number of posts in a large number of categories to a small number of subscribers. Which is the situation you have at Geocaching.com.

[MagicTyger]

How about we discuss the merit of the idea then? Because I think it is a brilliant idea.

 

This has already been discussed before but here goes.

 

RSS feeds are excellent for websites like News websites, Blogs, and Forums. When a new story, article or post is posted it is then sent to all the subscribers. Some websites have categories (World, Local, Late Breaking, Food, Investment, etc) an RSS feed is need for each category. RSS was developed to distribute posts from a relatively small number of categories to a large number of subscribers.

 

How to apply this to a website like Geocaching.com. Since Geocaching.com is world wide and not everyone cares when someone logs a cache that isn't in their home area then there will need to be more than one category. Do you divide the world up in to cities? What about large cities? Do you divide the city up even further while also providing a whole city feed? What about rural areas? What about areas that are divided by natural boundaries like rivers, lakes, and mountains? That is a lot of categories and we haven't even gotten to some of the sub-categories like division by cache type, terrain, difficulty, etc. Now how many people will actually sign up for the RSS feeds. In the citys maybe a few hundred but most likely when you average everything out the RSS feed to subscriber ratio will be very low when compared to other websites.

 

Will RSS work for a website like Geocaching.com? I don't see why not. Is RSS an efficient way of aggregating new cache notifications and new cache logs. No, not by a long shot. RSS is efficient at sending occasional posts from a small number of categories to a large number of subscribers. RSS totally sucks at sending a large number of posts in a large number of categories to a small number of subscribers. Which is the situation you have at Geocaching.com.

 

RSS WILL work for the situations I've outlined above. Namely:

 

1. Someone wants an RSS feed for a specific geocache.

2. Someone wants an RSS feed for caches they have found.

 

It would be on a request only basis, which would make it a small number of categories to a small number of users.

 

The feeds can be generated on the fly. E.g. something like:

http://SomeURL.com/RSS.aspx?User=MyUserName

or

http://SomeURL.com/RSS.aspx?Cache=MyCahceCode

 

No need to generate one for every person, and every cache - only those few who are interested in blogging them, or posting their activity on a website.

 

From a realworld point of view, someone's list of caches isn't much different from someone's blog. They might make one or two entries per week.

 

Individiual Caches - same thing - although the busier caches might get one or two entries per day.

[+The Last Airbender]

How about we discuss the merit of the idea then? Because I think it is a brilliant idea.

 

This has already been discussed before but here goes.

 

RSS feeds are excellent for websites like News websites, Blogs, and Forums. When a new story, article or post is posted it is then sent to all the subscribers. Some websites have categories (World, Local, Late Breaking, Food, Investment, etc) an RSS feed is need for each category. RSS was developed to distribute posts from a relatively small number of categories to a large number of subscribers.

 

How to apply this to a website like Geocaching.com. Since Geocaching.com is world wide and not everyone cares when someone logs a cache that isn't in their home area then there will need to be more than one category. Do you divide the world up in to cities? What about large cities? Do you divide the city up even further while also providing a whole city feed? What about rural areas? What about areas that are divided by natural boundaries like rivers, lakes, and mountains? That is a lot of categories and we haven't even gotten to some of the sub-categories like division by cache type, terrain, difficulty, etc. Now how many people will actually sign up for the RSS feeds. In the citys maybe a few hundred but most likely when you average everything out the RSS feed to subscriber ratio will be very low when compared to other websites.

 

Will RSS work for a website like Geocaching.com? I don't see why not. Is RSS an efficient way of aggregating new cache notifications and new cache logs. No, not by a long shot. RSS is efficient at sending occasional posts from a small number of categories to a large number of subscribers. RSS totally sucks at sending a large number of posts in a large number of categories to a small number of subscribers. Which is the situation you have at Geocaching.com.

 

RSS WILL work for the situations I've outlined above. Namely:

 

1. Someone wants an RSS feed for a specific geocache.

2. Someone wants an RSS feed for caches they have found.

 

It would be on a request only basis, which would make it a small number of categories to a small number of users.

 

The feeds can be generated on the fly. E.g. something like:

http://SomeURL.com/RSS.aspx?User=MyUserName

or

http://SomeURL.com/RSS.aspx?Cache=MyCahceCode

 

No need to generate one for every person, and every cache - only those few who are interested in blogging them, or posting their activity on a website.

 

From a realworld point of view, someone's list of caches isn't much different from someone's blog. They might make one or two entries per week.

 

Individiual Caches - same thing - although the busier caches might get one or two entries per day.

I agree the 2 above reason are more valid then what your are talking about Glenn. This would also be a fix to that My Friends feature also.

[+Glenn]

With 467946 active caches worldwide and 313340 new logs written by 45219 account holders in the past 7 days (as of 10/7/2007 11:18:27 PM (Pacific)) and the fact that RSS in now a feature or at the very least an add on for most browsers I doubt the number of users requesting RSS feeds and the amount of RSS feeds requested will be as low as you think.

[+StarBrand]

I don't see this as workable and easy as some try to make it sound.

 

As has been pointed out. RSS is excellent for sending a small snip of information to a large group of people but for RSS on the scale being suggested you are talking about highly customized, detailed RSS packets being generated for essentially 1 or 2 or maybe 3 users. Updated regularly. Multiply that times the number of caches and the number of logs generated and the resources necessary grow much larger than say a weather or news site very quickly.

[+moscow32]

It seems very like the concept of the watchlist, which is fully functional. Every time someone logs one of the caches or trackables on my watchlist, I get an email. I also get insta-notifies everytime a cache is published or archived within the zone I determine.

 

Why would a similar feature not be just as workable? I obviously don't understand the technological differences between the watchlist / insta-notify features and RSS, but in my over-simplified concept of the way that both would work, it seems doable.

 

I love RSS! Bring it on!

[+The Last Airbender]

I don't see this as workable and easy as some try to make it sound.

 

As has been pointed out. RSS is excellent for sending a small snip of information to a large group of people but for RSS on the scale being suggested you are talking about highly customized, detailed RSS packets being generated for essentially 1 or 2 or maybe 3 users. Updated regularly. Multiply that times the number of caches and the number of logs generated and the resources necessary grow much larger than say a weather or news site very quickly.

 

RSS doesnt send anything out but creates a file on the site for a reader to goto and read.

[OpinioNate]

It's an interesting request that I think a lot more people would find useful than some might at first think. RSS has been quite well received on Waymarking.com where you can get feeds for new waymarks and categories as well as for individual users.

 

In addition to MagicTyger's ideas I might also add that bookmark lists would probably be suitable for RSS. I'll pass the idea along to the boys upstairs. As always, implementation will depend on available resources and any technical considerations that my brain can't comprehend.

[+StarBrand]

I don't see this as workable and easy as some try to make it sound.

 

As has been pointed out. RSS is excellent for sending a small snip of information to a large group of people but for RSS on the scale being suggested you are talking about highly customized, detailed RSS packets being generated for essentially 1 or 2 or maybe 3 users. Updated regularly. Multiply that times the number of caches and the number of logs generated and the resources necessary grow much larger than say a weather or news site very quickly.

 

RSS doesnt send anything out but creates a file on the site for a reader to goto and read.

ummmm - when it gets "read" - that is information being sent somewhere......

[+Glenn]

<snip>

As always, implementation will depend on available resources and any technical considerations that my brain can't comprehend.

 

That has been my argument all along. I'm glad we have someone that can talk on behalf of Groundspeak once again.

[+BlessedBees]

With 467946 active caches worldwide and 313340 new logs written by 45219 account holders in the past 7 days (as of 10/7/2007 11:18:27 PM (Pacific)) and the fact that RSS in now a feature or at the very least an add on for most browsers I doubt the number of users requesting RSS feeds and the amount of RSS feeds requested will be as low as you think.

I just checked, these numbers aren't correct.

[+The Last Airbender]

It's an interesting request that I think a lot more people would find useful than some might at first think. RSS has been quite well received on Waymarking.com where you can get feeds for new waymarks and categories as well as for individual users.

 

In addition to MagicTyger's ideas I might also add that bookmark lists would probably be suitable for RSS. I'll pass the idea along to the boys upstairs. As always, implementation will depend on available resources and any technical considerations that my brain can't comprehend.

 

Thanks for the response.

[+The Last Airbender]

I don't see this as workable and easy as some try to make it sound.

 

As has been pointed out. RSS is excellent for sending a small snip of information to a large group of people but for RSS on the scale being suggested you are talking about highly customized, detailed RSS packets being generated for essentially 1 or 2 or maybe 3 users. Updated regularly. Multiply that times the number of caches and the number of logs generated and the resources necessary grow much larger than say a weather or news site very quickly.

 

RSS doesnt send anything out but creates a file on the site for a reader to goto and read.

ummmm - when it gets "read" - that is information being sent somewhere......

I think we are sayinig the same thing. RSS is on demand rather than a push to all which is what i thought you meant.

[+PittCaleb]

First let me say I'm an RSS addict. If I can't get it via RSS, I work with site owners to make that possible, so I applaud your effort here.

 

Here's the reason I would say it's a bad idea here. First, GeoCaching is big - tons of users, tons of caches and double tons of logs. I have my RSS reader set to check for new data every hour, that's on the 20 or so feeds I "subscribe" to. RSS is a pull system, not a push method.

 

With "watchlists" the site PUSHES out new log entries to those who want them. I can not even begin to estimate the site traffic it would generate if I did an RSS subscription to each of the 1700 cache finds I have. Every hour, it would pull what, the latest 8,500 logs. And it would do that 24 times a day - over 200,000 log requests daily. Do that for a thousand users...

 

I don't think the gc.com site was set up for this - regardless of the 'worth' of the project or not. There is a ton of cool things we could do if I could get live log data for all caches within 15 miles of my home, but the bandwidth and disk usage for gc.com would be overwhealming when scaled to the population of geeky geocachers. That is the reason I think it's a bad idea, not because of the desire for the data, just the fact that it's pull rather than push-based.

 

That said, if you could work within gc.com and find a way to push this data overnight on a digest-based format, that would be awesome!

 

Cheers,

PittCaleb

[MrMonk]

Most RSS readers have ETAG and/or conditional GET support, so it wouldn't be pulling the full feed with every single poll.

 

Personally I would love a single RSS feed for my friends' finds/logs (kind of like Facebook's News Feed). Feeds for a cache listing (or a single feed for multiple listings) would be nice to have (but not as hi-pri) so that my inbox doesn't get spammed with mail.

 

The "it doesn't scale" argument really isn't valid. Generally, well-designed systems are built to scale.

 

I think it could be done via a pub/sub kind of system. GC.com already knows which caches I'm interested in. Everytime a log entry is published, instead of sending me e-mail, it can update *my* feed, to which I'll subscribe. (I've only spent half an instant thinking about this.)

 

I would totally love it if you guys made *everything* in GC.com subscribable!

[+Glenn]

I just got done reading an interesting blog article titled A theory on why RSS traffic is growing out of control.

 

ETAGs and conditional GETs can reduce bandwidth. Although they don't reduce the number of request. Which is also a very important number. Also ETAGs and conditional GETs are very hard to implement properly on load-balanced systems. This is due to the fact that not all the servers will have the latest information. If the ETAGs are not identical it will look as if the information has changed and the RSS page will be resent. This defeats the whole ETAG system.

 

I've also read where some people think that ETAGs and conditional GETs could encourage people to reduce the time between RSS requests, increasing RSS requests, increasing the hits on the website and thereby negating any potential benefit. My personal belief is that most people will simply leave it set to whatever is default.

[mjar81]

Has this topic ceased discussion?

 

I'd love to get an RSS feed of my finds to put on my blog or an RSS feed of logs for any cache I may place in the future.

 

The discussion of "RSS totally sucks at sending a large number of posts in a large number of categories to a small number of subscribers" totally doesn't fly... Netflix does it for every single one of their users for tens of thousands of possible movies, why can't geocaching.com?

 

Just my 2¢ as a geocaching newbie and a web professional.

[+ReadyOrNot]

I think any potential concerns regarding bandwidth and the number of requests would be much less than the load for Groundspeak to push out emails to everyone. The RSS option would have to be a replacement for the current subscription system.

 

with version 2.0 coming out, considering RSS feeds already in use on Waymarking, my bet would be that we are going to see this in version 2.0 and that it's already in the works.

[lakeuk]

a side the T&C issue, it is easily possible to generate RSS feeds for geocaching using Yahoo Pipes.

 

I was shocked at how easy it makes it - http://pipes.yahoo.com/pipes/

[+ReadyOrNot]

a side the T&C issue, it is easily possible to generate RSS feeds for geocaching using Yahoo Pipes.

 

I was shocked at how easy it makes it - http://pipes.yahoo.com/pipes/

 

So in other words, never mind that it's against the T&C and you could get your account banned for life, but here's how you do it?!?!

[lakeuk]

a side the T&C issue, it is easily possible to generate RSS feeds for geocaching using Yahoo Pipes.

 

I was shocked at how easy it makes it - http://pipes.yahoo.com/pipes/

 

So in other words, never mind that it's against the T&C and you could get your account banned for life, but here's how you do it?!?!

 

I've not stated how to do it, meally stated a tool that's designed to generate rss from an ordinary webpage - you'd still need to work out how to do it for yourselve

 

The T&C is clear so doesn't need debating, but it's only enforceable against a registered member of site, there's nothing to stop non-member Joe Bloggs doing it

[+sbell111]

The T&C is clear so doesn't need debating, but it's only enforceable against a registered member of site, there's nothing to stop non-member Joe Bloggs doing it

Of course, as soon as TPTB notice the scraping, they'll shut down Joe Blogg's access to the site.

[lakeuk]

Of course, as soon as TPTB notice the scraping, they'll shut down Joe Blogg's access to the site.

 

I'm sure (quite rightly) action would be taken if scraping was noticed, but they wouldn't be able to notice Joe Blogg's doing this, he'd just look like any other non-member person accessing the website

[Guyute1210]

Of course, as soon as TPTB notice the scraping, they'll shut down Joe Blogg's access to the site.

 

I'm sure (quite rightly) action would be taken if scraping was noticed, but they wouldn't be able to notice Joe Blogg's doing this, he'd just look like any other non-member person accessing the website

Unless he has the same IP address. I'm sure GS logs all IP addresses.

[lakeuk]

Of course, as soon as TPTB notice the scraping, they'll shut down Joe Blogg's access to the site.

 

I'm sure (quite rightly) action would be taken if scraping was noticed, but they wouldn't be able to notice Joe Blogg's doing this, he'd just look like any other non-member person accessing the website

Unless he has the same IP address. I'm sure GS logs all IP addresses.

 

logging the IP address would make no difference as the amount of traffic used would be the same as an ordinary user - how would you differentiate the two?

[+ReadyOrNot]

Of course, as soon as TPTB notice the scraping, they'll shut down Joe Blogg's access to the site.

 

I'm sure (quite rightly) action would be taken if scraping was noticed, but they wouldn't be able to notice Joe Blogg's doing this, he'd just look like any other non-member person accessing the website

Unless he has the same IP address. I'm sure GS logs all IP addresses.

 

logging the IP address would make no difference as the amount of traffic used would be the same as an ordinary user - how would you differentiate the two?

 

A script behaves much different than a human being. Looking at response times between requests can clue them off. Again, the issue is not whether it can be done, but the fact that it should NOT be done.

[Darchmare]

I'd like to throw my support in the ring for feeds that at least provide the following:

 

> 1. Someone wants an RSS feed for a specific geocache.

> 2. Someone wants an RSS feed for caches they have found.

 

...particularly the 2nd one.

 

Both are perfectly useful and reasonable requests, and would provide significant benefits to both Groundspeak's users as well as Groundspeak itself.

 

For users, the benefits are obvious. Others have explained this in more detail than I can, but in today's "Web 2.0" world there are all sorts of useful things that can be done with RSS/Atom feeds.

 

For Groundspeak, there are benefits as well.

 

- For one thing, those who use such feeds will almost certainly turn off email notifications. Whatever additional resources consumed by people requesting static text files will be largely mitigated by the fact that you no longer have to send emails out to those end users.

 

- It's a selling point. Face it, in 2008 a lot of peoples' usage patterns have changed and sites/services that don't keep up risk falling behind. I understand that Groundspeak essentially "owns" geocaching as a hobby, but this is something useful enough that a competitor could use it to gain some inertia. For what it's worth, I'm not a premium member. If this were added, I would sign up in a heartbeat.

 

- It's a form of advertising. Any time someone syndicates a feed on their own web site, they will be linking back to the geocaching.com web site - multiple times. This will only solidify the association people will have with Groundspeak and the activity, potentially bringing new people into the fold.

 

If I may, I'd like to add another to the mix: It's simply the right thing to do.

 

After all, this service is built on information submitted by end users. Any time you log your finds, etc. you are adding value to the service. The least they can do is allow end users to actually use that information in a useful and helpful way.

 

Anyhow, while there are technical challenges, I think the above more than makes the case that such feeds are a 'really good idea' that should be implemented. Rather voluminous sites use RSS all over the place, so it's an insurmountable challenge. It'd be nice if Groundspeak could at least let it be known that this is something they know people find useful and would be interested in doing it if the technical hurdles could be handled.

 

- Jeff

[+amiantos]

I want to throw in my support for per-user feeds.

 

Last.FM uses RSS feeds for everything you listen to, and you can input those feeds into things like Facebook (in the case of geocaching you could have "Brad found "So and so"!" or "Brad couldn't find "so and so" in your news feed). Or you could use the feed to display your recent finds on your personal website, etc.

 

Per-cache feeds would be cool, too, but email notifications take care of that. It would just be nice if there was a way to spit my geocaching.com account activity onto another site somehow. In the case of Facebook, it's free geocaching.com publicity... and make the RSS feed a feature for premium-only accounts and then you've got yet another reason to pay for the site.

 

Geocaching.com needs to jump on the Web-2.0 bandwagon some more. (the google maps interface is slick!)

[+The Cooker]

It would be great to have all the email I currently get to show up in an RSS feed.

 

Say every user gets one Geocaching RSS Feed. And allow me to pick which items will be in the feed:

 

1. Notifications

2. Bookmarked items

3. Logs on owned caches

4. TB logs on owned TBs

5. Watch list items

6. Geocaching news

7. Forum threads

 

We get email for much of this already! Instead they can be put into an RSS feed. Man do I love Google Reader to organize a ton of information and allow me to see what is new in the world I care about.

 

You have my vote!

 

The Cooker

[+5 Caching Campers]

I would love to have an RSS feed of my finds. I've tried to emulate that by having my own blog (HayleeBugg Geocaching) which provides its own feed.... but it'd be much nice to have it be a bit more automated.

 

I would also be nice if there was a Geocaching Facebook application. There's one that will but your stats image on the "boxes" tab, but I'm talking something that does a little more. Nothing that would require you to use Facebook in order to geocache, but something that would put my finds into my mini feed. I've thought about something that would allow people to upload a GPX that would then generate stats and through things into the mini feed, but I don't have the time to do that right now .

Edited October 16, 2008 by HayleeBugg

[+coolerMietzer]

Hi,

 

I am new to GeoCaching and I am fascinated! But I wondered that there is no Feed or any other possibility to share my logs with other. On my blog there is a "lifestream", where everything I do on twitter, on my blog, on flickr, youtube, last.fm and so on is logged and shown in a timeline.

Like much others I would find it very nice to have the page "My logs", which is already generated, available as RSS or JSON or something else. For those who don't like to share this information there should be an option to disable this (per default disabled).

Perhaps a password-protected feed would be better, not everybody has to see everyone's logs.

It is the time of Web2.0, so it's time for this feed!