Jump to content

Geocaching Api?


protocoldroid

Recommended Posts

I'm curious if there's ever been a consideration for creating an interface for developers to build client applications for geocaching?

 

I know there's such applications as GSAK (which use the meta data from the GPX files [e.g. found / not found, owned / not owned, difficulty and terrain, etc], but, I'm wondering about client applications which access the live data on GC, talking directly to geocaching.com.

 

Anywho... some examples of website api's that i use...

 

Free to anyone -> LiveJournal Client Interface -> http://www.livejournal.com/doc/server/ljp.csp.guide.html

Pay to use -> EbayAPI -> http://developer.ebay.com/DevProgram/developer/api.asp

 

Anyhow, just curious. I built a small screen scraper (for my own personal use) to glean some cache info, and I started thinking it'd be awesome if there were an API for geocaching. I feel kinda guilty/dirty scraping the data (so I won't share my application with anyone else -- it's just not ethical), but... I have some decent ideas of what I could do with the data (which would probably be useful to more than just me), and I bet my ideas are miniscule compared to what other more seasoned geocaching developers might have.

Link to comment
they don't want to do it as that's the core of gc.com. If they allow anyone access to the data, why would gc.com receive funding for their site (membership)?

well, at least from my perspective...

 

if they were to make the data available on a user-per-user basis, and make it just like the GPX / Loc Model.

 

E.g. you can only see the metadata for waypoints if you subscribe to the service. So, only through the eyes of a premium member do you see the premium information.

 

Anyways, livejournal and ebay make money on it. why not think bigger?

 

...Although I had a feeling when I made this post that someone was going to say "it just doesn't work with the geocaching.com model".

Link to comment

I'd love this too. The instant I started geocaching I started writing scripts to produce simple text output that I could print out in small formats... An API would greatly help the things I'd like to do with some of my caches as well.

 

I'm not sure the best way to fund the model, and the thing to do would be to see how the other sites such as those mentioned (and google, for that matter) make their profits from their published APIs. One way around this is to only offer an API connection to paying premium members.

Link to comment

I guess a big concern would be the impact of external queries on their database. I know they have (according to their site) a separate machine to handle the PQs. I don't if the API would allow someone to, for example, select * from the database and bring the site/system to its knees.

Link to comment
I don't {know} if the API would allow someone to, for example, select * from the database and bring the site/system to its knees.

yep, there'd have to be a slew of design considerations that'd limit what you could see / do, cause, people certainly would end up doing something sloppy (e.g. extracting a small bit of data from a very large site inquery) and really bog down the GC machine(s).

 

anyways, they could mirror what they're doing with the web page right now, for starters... e.g. you must search by a specific metric (zip, keyword, state, etc) and then you get 20 results, then in a set interval of time, you can get the next 20 results. In fact, I kinda let my personal test screen scraper get a little busy, and geocaching.com even has a mechanism that knows you're downloading too much, and they just start redirecting you to the homepage (...which I assume is flat / mostly flat).

Link to comment

There are some things about the PQ's I'd like to see changed, but they've solved 90% of the things I wanted in an API. What do you want to do that you can't do with a PQ and external tools? A PQ and tools like GPSBabel can handle your polygon case, for example.

 

Now, I just want my logging interface to work again...

 

http://forums.Groundspeak.com/GC/index.php?showtopic=68251

 

On a related topic, we just added HTML and text output to GPSBabel to help with the "I just want a simple web/text page from a PQ" need.

Edited by robertlipe
Link to comment

I've often thought about this type of application to do things I'd like to do that GC or any other site (aka Navicache) don't do.

 

For example I can't see all my finds on a map, nor can I easily generate the Pocket Query to do that indirectly.

 

Also I like checking up on caches I found every now and then and see how they are doing, who's logging, and if they have gone on to that great cache archive in the sky. Pocket Queries only give you the last 5 logs. That's not enough to do the job.

 

Right now I think an API is possible with Navicache but I am not a computer programmer type who can confirm it.

Link to comment

I will go on record as being for the exposing of data in a manner other than by web only. That said, I also clarify that I am not talking about exposing the database for just anyone to query (that would be ridiculous).

 

The fact of the matter is, exposing the data via some other method than the web is honestly a low priority issue from the GC perspective. Priority should rightly be given to the web site upgrades/improvements and a whole list of other issues that are long past the original completion estimates. The fact is, us 'power users' who find a need to scrape data represent a very low percentage of the GC population.

 

That said, I will say that I personally would be willing to pay extra for lower-level access to the data, even if merely pre-defined queries. Something along the lines of this would be adequate:

 

Data retrieval:

 

1. Cache logs (list) by User

2. Caches hid (list) by User

3. Newest caches in Location (by state, country, etc. - list)

4. Closest 100 caches to coordinates (list) *

5. Cache Information by GC Number (including all logs, not just 5)

6. User Profile

7. Recent Logs (list)

8. Get Image (from img server)

 

Data submit:

 

1. Add/Update/Delete Log

2. Add/Update/Disable/Archive Cache

3. Update Profile

4. Add/Update/Delete Image

 

* Data retrieval #4 is probably the most query intensive due to the math involved (sorting on an aggregate); I'd be happy even with a 'caches within rectangular coordinates (>=minlat >=minlon <=maxlat <=maxlon)' style query that does no aggrgating or sorting at all (a much lesser cost query); With the rectangular coordinate query it would be the responsibility of the client app to do sorting, determining the closest 100, etc.

 

I might be missing something, but I can't think of anything that can't be derived from the results of the above list. Those of us scraping data currently (yes, I admit it) for various things (be it from personal interest to club stats) would be more than happy to hit the more robust (and more efficient) text only 'API server' instead of the web server.

 

Problem I can see with the concept:

With a text based API available, a whole variety of 'Bob & Sue's Super Geocaching Interface' programs would pop up thus reducing Geocaching.com's web traffic. While at first glance that seems like a good thing, it isn't, because it reduces the visibility of the sponsorship ads that geocaching.com places on the site. Reduced visibility equals reduced values for the ads equals less money for geocaching.com. Although I'd be willing to spend more for the access to the API, I don't know if I'd be willing to spend _that_ much more.

Link to comment

Don't forget the Google API, as another example of a site that's doing it.

 

I'd happily apply/pay for a developers license (above and beyond my charter membership) to be able to have more direct access to the live gc.com data such as this. If there's concern over people bogging down the site with bad queries, then they could (should?) require applicants to prove some minimum level of competence before being granted such a license. The developer's license agreement should also include a clause indicating what developers would NOT be allowed to do with the data (i.e. simply mirror geocaching.com under their own brand), etc.

License cost could be tiered, depending on the quantity of data a developer pulls. A hobbyist running one small query a day for their local community would pay far less than someone such as myself, who would be creating much larger/robust applications and therefore accessing a great deal more data every day.

To recoup the cost of the license, developers could take donations, or charge outright, for whatever apps they develop. I think this could be a gold mine for GC.com and open the floodgates of innovation and creative new ways to play the game. I believe that most, if not all, of the features and improvements users are asking for from GC.com, would quickly be provided by outside developers, once they could get the data to do so, and followed closely by amazing new tools that nobody has dared even dream of yet.

Edited by skydiver
Link to comment
I think this could be a gold mine for GC.com and open the floodgates of innovation and creative new ways to play the game. I believe that most, if not all, of the features and improvements users are asking for from GC.com, would quickly be provided by outside developers, once they could get the data to do so, and followed closely by amazing new tools that nobody has dared even dream of yet.

this is what i was trying to get at :lol: the nitty gritty of how it may or may not work, is details, I'm saying that I think that it would contribute to the success of GC.com. I know the site isn't filled to the teeth with power-users, but, there's definately a userbase which has technically savvy people who can independantly contribute.

Link to comment

We're already planning to create a sandbox area for the many different developers who have developed applications to work with the GPX Groundspeak namespace. It will be called the Sandbox, and the initial web service will be a cache logging service. It will be simple but supported, unlike the many other hacks against the geocaching.com site.

 

Stats are not on the plan at this point, nor are there any initial plans to have access to any of the site's user data, which includes stats. The features will be intended for individual users who wish to interact through the web site via alternate interfaces, such as ClayJar's Watcher, GSAK, etc. We'll expose these services to developers, who can in turn provide additional built-in features in their own apps.

 

I'm not at the point where I can answer questions about the Sandbox, but I wanted you to know that there are plans to offer services like this.

Link to comment

rock on jeremy, thanks for the info... i knew you guys had to have something (good, as usual) up your sleave :lol: anyways, can't wait to hear more about the sandbox, will be patiently watching and waiting.

 

also wanted to toss in another $.02 worth, i'm more interested about the cache information than the user information... have noticed there are threads around about the stat information... at least for me, that's not the juicy part: i'm into the spatial data -- not the social data :D

 

(edited, added that last paragraph above)

Edited by protocoldroid
Link to comment

One of the things I'd be willing to pay for as part of this is a starter data-drop, and then access to specific URL's that more humanely allow gleaning data from the site.

 

As an example of the former, I would gladly pay for a CD that contained all the relevant information about every cache in the system - no, not every log or anything like thiat - I mean the GC#, guid, date placed, lat/long, type, container type, status (active, archived, temporarily offline), etc. This would let me pre-populate my database with all the caches out there as a starting point.

 

As an example of the latter, being provided with either specific URL's, an RSS feed, or files that can be FTP'd - the caches placed/updated (status update) within the last 24 hours. This would let me keep my database up to date.

 

Having access to this data would allow me to generate code that would, for example:

 

a- List caches along I-4 from Orlando to Tampa within 1 mile of the interstate.

b- Build trip-based cache lists that aren't point-of-origin based.

c- Build stats that are geographically broken down by state, zip code, etc.

 

Stuff I'm willing to let my CPU churn and do, but may be of less interest to the general population of cachers (and therefore shouldn't be part of the GC.COM website.

 

That's my opion - and we all know about opinions... :tired:

Edited by TheNomad
Link to comment

Hi Jeremy,

 

What's the word on the 'Sandbox' project? Is there something that we could start testing against? Just curious and anxious :blink:

 

Regards,

-Fabien.

 

We're already planning to create a sandbox area for the many different developers who have developed applications to work with the GPX Groundspeak namespace. It will be called the Sandbox, and the initial web service will be a cache logging service. It will be simple but supported, unlike the many other hacks against the geocaching.com site.

 

Stats are not on the plan at this point, nor are there any initial plans to have access to any of the site's user data, which includes stats. The features will be intended for individual users who wish to interact through the web site via alternate interfaces, such as ClayJar's Watcher, GSAK, etc. We'll expose these services to developers, who can in turn provide additional built-in features in their own apps.

 

I'm not at the point where I can answer questions about the Sandbox, but I wanted you to know that there are plans to offer services like this.

Link to comment

I kept it quiet, but last week I was on vacation, so I had put a hold on new development projects until I returned. Now that I'm back I'll need to play catch-up and correct a few bugs, but I hope to get the sandbox site up soon. I'll update folks here as I get closer to launch.

Link to comment

What I'd Pay for in an API is the ability to update my GSAK Database directly from the GC.Com site.

 

I have a few GSAK DBs that are for routes/freeways that I frequent. I'd like to be able to keep the DB Current without having to figure out which PQ I ran to get all of the different Caches.

 

It would be nice if I could just tell GSAK to update the DB in the background with the current logs, etc.

 

It just sucks to have to keep reimporting the GPX files into the DB and updating it in a manual fashion. Seems like you should jsut be able to click UPDATE and GSAK just goes out and updates the Caches in its DB.

 

:(

 

My 2 Cents...

 

Thanks,

Scooter!

Link to comment

 

It would be nice if I could just tell GSAK to update the DB in the background with the current logs, etc.

 

It just sucks to have to keep reimporting the GPX files into the DB and updating it in a manual fashion. ;)

 

My 2 Cents...

 

Thanks,

Scooter!

THAT would be great. I have 8 PQ's running just to update all the Kansas and Missouri active and disabled caches . Then drag into GASK to updata the data bases.

 

It would be grrrreat if I could make this a one click operation somehow.

 

Glenn

Link to comment
Guest
This topic is now closed to further replies.
×
×
  • Create New...