Jump to content
Sign in to follow this  
Followers 14
OpinioNate

DeLorme et al

Recommended Posts

We recently made a change to the schema for our GPX files that negatively impacted some consumers of this data. After carefully weighing several options we have decided to temporarily go back to serving up the older version that we know works.

 

We have already begun to develop a way for users to choose between receiving the older version that you know and love, and the newer version containing geocache attributes. We hope this will give time for our partners to anticipate the changes and make any necessary accommodations. In the future when we make changes of this nature we will be more sensitive to our partner's needs and try to provide some backwards compatibility.

 

I will update the thread with more details on this option to choose between versions when downloading .GPX files when the plan solidifies.

 

The roll back has been completed at the time of this posting. You'll want to re-download any cache .GPX's and reschedule any PQs you received since the last release for this change to take effect.

 

Thanks! :)

Share this post


Link to post

We recently made a change to the schema for our GPX files that negatively impacted some consumers of this data. After carefully weighing several options we have decided to temporarily go back to serving up the older version that we know works.

 

Sigh. Unfortunately whatever app broke (I haven't been following any "this broke" thread) is at fault. Most of the point of XML is surrounded by the fact that applications should ignore new things it doesn't understand (unless some specific spec called for it not to).

 

Since GeoQO already supports parsing and using attributes and I was happily making use of the data, is there any time estimate for when it'll be turned back on.

Share this post


Link to post
Sigh. Unfortunately whatever app broke (I haven't been following any "this broke" thread) is at fault. Most of the point of XML is surrounded by the fact that applications should ignore new things it doesn't understand (unless some specific spec called for it not to).

Yep. Unfortunately this is going to be the norm for the foreseeable future, at least until we've got a generation of XML application programmers who've burnt their fingers a few times. Right now, the tail (people whose apps assume that the X in XML stands for something else apart from "eXtensible") is wagging the dog (services like Groundspeak which have to rely on people down the line actually doing their job right), partly because the tail is so much bigger than the dog.

Share this post


Link to post

We recently made a change to the schema for our GPX files that negatively impacted some consumers of this data. After carefully weighing several options we have decided to temporarily go back to serving up the older version that we know works.

 

Sigh. Unfortunately whatever app broke (I haven't been following any "this broke" thread) is at fault. Most of the point of XML is surrounded by the fact that applications should ignore new things it doesn't understand (unless some specific spec called for it not to).

I'm sorry but you cannot leave Groundspeak blameless here. They made a change to their API and didn't tell 3rd-party developers like they had previously said they would.

 

Yes, it can be argued that XML-reading applications should be more robust, but you cannot change an API without telling consumers of that API! Even if "it shouldn't break anything" people need the opportunity to perform regression tests to be sure.

Share this post


Link to post

Yep. Unfortunately this is going to be the norm for the foreseeable future, at least until we've got a generation of XML application programmers who've burnt their fingers a few times. Right now, the tail (people whose apps assume that the X in XML stands for something else apart from "eXtensible") is wagging the dog (services like Groundspeak which have to rely on people down the line actually doing their job right), partly because the tail is so much bigger than the dog.

I agree about the XML part, like the new GC-attributes. They should be ignored by the parser if it doesn't understand them.

 

But I still don't get how an XML parser would provide meaningful data when <gs:type> was changed from "Unknown cache" to "Mystery/puzzle cache". E.g. how would the GPSr know that it should draw a fat, blue questionmark for the symbol of "Mystery/puzzle cache".

Share this post


Link to post

The only app that I know of that broke was Delorme's: http://forum.delorme.com/viewtopic.php?f=141&t=21029 Remembering that PQ readers are baked into the firmware of Garmin and Lowrance receivers these days, caution when changing is in order but it doesn't mean nothing can ever change.

 

Adding tags and attributes should be totally transparent to any self-respecting consumer of XML. Changing the spelling - including capitalization - of existing values is bad. (I didn't know the spelling had changed - the one I approved definitely had "Unknown Cache".)

 

In GPSBabel, at least, if a gs:type of "Mystery/puzzle cache" had been seen it would have been treated exactly like "unknown" not by any great forward design but because of the same happenstance that a gs:type of "watermelon" would have been treated as "unknown"; we don't know what it is, so it's "unknown". (source) I suspect others may have "benefited" from this same accident.

 

If this is indeed the extent of the scope, instead of confusing users - and programs - with multiple types of PQs for something so simple, please just bump Schemaloc, leave in the attributes, and don't introduce a gratuitous spelling change for 'Unknown'.

 

Notifying developers, public[1] discussion of proposed change, and providing early samples would go a long way to keeping this path smooth.

 

[1] And "public" might mean programmers of the apps and device that matter and not the masses.

Share this post


Link to post

The only app that I know of that broke was Delorme's: http://forum.delorme.com/viewtopic.php?f=141&t=21029 Remembering that PQ readers are baked into the firmware of Garmin and Lowrance receivers these days, caution when changing is in order but it doesn't mean nothing can ever change.

There was at least one other referenced in this forum that broke. And GSAK made a change a month ago in anticipation of the change - perhaps it would have been impacted as well, if they didn't know?

 

And this wasn't the first time something like this has happened.

 

Adding tags and attributes should be totally transparent to any self-respecting consumer of XML. Changing the spelling - including capitalization - of existing values is bad. (I didn't know the spelling had changed - the one I approved definitely had "Unknown Cache".)
But in this case, that wasn't the only change. The schema was changed too.

 

In GPSBabel, at least, if a gs:type of "Mystery/puzzle cache" had been seen it would have been treated exactly like "unknown" not by any great forward design but because of the same happenstance that a gs:type of "watermelon" would have been treated as "unknown"; we don't know what it is, so it's "unknown". (source) I suspect others may have "benefited" from this same accident.
That's more a happy accident as a result of defensive coding. Had a new, non-"unknown" cache type been added by Groundspeak, GPSBabel would have presented it as Unknown, which would have made users unhappy as well.

 

Notifying developers, public[1] discussion of proposed change, and providing early samples would go a long way to keeping this path smooth.

 

[1] And "public" might mean programmers of the apps and device that matter and not the masses.

Either way, this seems to have not been followed through

Share this post


Link to post

But I still don't get how an XML parser would provide meaningful data when <gs:type> was changed from "Unknown cache" to "Mystery/puzzle cache". E.g. how would the GPSr know that it should draw a fat, blue questionmark for the symbol of "Mystery/puzzle cache".

 

Maybe they can add a unique number attribute to the <Groundspeak:type>, just like they did with <Groundspeak:attribute>. I don't even try to parse the text of those <gs:attribute> tags in my application.

 

Both <Groundspeak:container> and the type under <Groundspeak:log> could benefit from this as well. And maybe i have overlooked some other tags.

Share this post


Link to post

I won't participate in a pile-on, but I'll help size the problem and discuss how to fix it if Groundspeak is listening.OK Groundspeak is listening. I meant that I'm more interested in participating in a tech conversation than a "Groundspeak sucks" one. Wouldn't want to make anyone cry.

 

The non-beta builds of GSAK didn't have a problem with the new PQs - otherwise there would have been rioting in the streets. So GSAK had a head start on attributes but wouldn't have been broken without that.

 

I wasn't very clear. The "luck" was that it was the spelling of "unknown" specifically that changed because most programs that need to understand the cache types will *probably* treat something new (and that's what "Mystery" would be) as "unknown". If, say, "Earthcache" had been respelled to "The New Virtual" I suspect most programs would treat that as "unknown", too.

 

FraCaMen's suggestion is a good one. While there may be some skittishness in the current context to adding attributes to existting tags even in light that it *shouldn't* hose apps, I'd be supportive of ripping this band-aid off while we have programmer eyes on it. I'd gleefully change my reader to prefer a numeric attribute over a free-form textual tag in those fields and be out of the business of haggling over "Found it" vs "Found It" forever.

 

Publishing the tables would also be a key step - developers need to know that a attribute 42 is "needs maintenance" while a log type 15 (or whatever) is "needs maintenance" instead of building large PQs or looking at various page sources to divine them.

Edited by robertlipe

Share this post


Link to post
Sigh. Unfortunately whatever app broke (I haven't been following any "this broke" thread) is at fault. Most of the point of XML is surrounded by the fact that applications should ignore new things it doesn't understand (unless some specific spec called for it not to).

I am puzzled by this. The app broke because the attribute spelling changed from "Unknown Cache" to "Mystery/Puzzle Cache."

 

And somehow they were supposed to have written the app to have known that this ahead of time?

 

I was lucky that my FindStats app didn't break because I just build a map of the cache types that appear in the GPX file, but I can easily see how if I had pre-defined some tag values it could have broken.

Share this post


Link to post

There were two separate changes made: a change to the GPX schema, and a change to the wording of the "Unknown" cache type. It is the former that caused the more significant issues for various applications, and the one that should supposedly be handled by any well-coded parser.

Share this post


Link to post
FraCaMen's suggestion is a good one. While there may be some skittishness in the current context to adding attributes to existting tags even in light that it *shouldn't* hose apps, I'd be supportive of ripping this band-aid off while we have programmer eyes on it. I'd gleefully change my reader to prefer a numeric attribute over a free-form textual tag in those fields and be out of the business of haggling over "Found it" vs "Found It" forever.

 

Publishing the tables would also be a key step - developers need to know that a attribute 42 is "needs maintenance" while a log type 15 (or whatever) is "needs maintenance" instead of building large PQs or looking at various page sources to divine them.

Amen to this. Very constructive suggestion (as always) from robertlipe.

 

By the way, lest we be too hard on Groundspeak: I am in agreement that no application should have broken with the addition of the attributes. Conformant XML parsers should simply ignore attributes they don't understand.

Share this post


Link to post
Sigh. Unfortunately whatever app broke (I haven't been following any "this broke" thread) is at fault. Most of the point of XML is surrounded by the fact that applications should ignore new things it doesn't understand (unless some specific spec called for it not to).

 

I am puzzled by this. The app broke because the attribute spelling changed from "Unknown Cache" to "Mystery/Puzzle Cache."

 

 

Ah... I didn't see that change and assumed it was broken because of the attribute additions. Changing data *values* like that definitely is a bad thing to hard-coded applications. What's worse is that changes like that require smart code to be able to deal with old and new PQs, which isn't a good thing.

 

I wrote GeoQO to be generic so it can accept any type and you can create your own (ie, it handles much more than just Groundspeak data). But I realize that many applications would rather choose to hard-code type lists and things like that.

Share this post


Link to post

I will update the thread with more details on this option to choose between versions when downloading .GPX files when the plan solidifies.

Could you consider a new element (when all the parsers have been taught to ignore unkown ones :lol: ?

 

In addition to the current basic info

 

<Groundspeak:type>Traditional Cache</Groundspeak:type> // 1

<Groundspeak:container>Regular</Groundspeak:container> // 3

<Groundspeak:difficulty>2</Groundspeak:difficulty>

<Groundspeak:terrain>1.5</Groundspeak:terrain>

 

there would also be an element like

 

<Groundspeak:cacheinfo>01032015</Groundspeak:cacheinfo> //two digits per info

 

or, probably tidier

 

<Groundspeak:cacheinfo>1,3,2,1.5</Groundspeak:cacheinfo>

Share this post


Link to post

Gross! One of the key principles of database programming is to not have the same information in > 1 place or in > 1 form and one of the fundamentals of XML is to provide *structured* data, not comma separated values smuggled into a structured field.

 

The change that a couple of us are proposing (without looking to see if these numbers are probably "right") is to change

 

<Groundspeak:type>Traditional Cache</Groundspeak:type>

<Groundspeak:container>Regular</Groundspeak:container>

 

to

 

<Groundspeak:type id="1">Traditional Cache</Groundspeak:type>

<Groundspeak:container id="3">Regular</Groundspeak:container>

 

I would be very much against repeating these plus diff and terrain in another field.

Share this post


Link to post

there would also be an element like

 

<Groundspeak:cacheinfo>01032015</Groundspeak:cacheinfo> //two digits per info

 

or, probably tidier

 

<Groundspeak:cacheinfo>1,3,2,1.5</Groundspeak:cacheinfo>

Why include an element with redundant information? That's just bad design. A parser that reads the existing elements has all the information already.

Share this post


Link to post

There were two separate changes made: a change to the GPX schema, and a change to the wording of the "Unknown" cache type. It is the former that caused the more significant issues for various applications, and the one that should supposedly be handled by any well-coded parser.

 

Thanks for the clarification.

 

To prevent something like this in the future, maybe Groundspeak could create a test suite of GPXs which third parties can use to validate their code against. These could include:

 

1. Valid GPX as is dished out from the PQ generator

2. Valid GPX with some additional elements not produced by any PQ. Apps should ignore the additional elements.

3. Invalid GPX with XML syntax/nesting errors. Apps should handle these gracefully, like skipping to the next valid waypoint.

 

PS: You could even come up with a "Designed for Groundspeak GPX" label for applications which succesfully run the test suite. :lol:

Share this post


Link to post

there would also be an element like

 

<Groundspeak:cacheinfo>01032015</Groundspeak:cacheinfo> //two digits per info

 

or, probably tidier

 

<Groundspeak:cacheinfo>1,3,2,1.5</Groundspeak:cacheinfo>

Why include an element with redundant information? That's just bad design. A parser that reads the existing elements has all the information already.

 

Then Groundspeak is free to spell e.g. "Unknown cache" anyway they please, even "Tuntematon kätkö" :lol:

Share this post


Link to post

And if they declare "Unknown Cache" to be element id 423, an application is free to present element 423 in Finnish as Tuntematon kätkö. If it's free-form text subject to change at will, apps really can't localize the strings. It's much more practical for an app to know that 423 represents the concept of a cache of type 'mystery' and choose how to interpret/represent that than for it is to "read" the text.

Edited by robertlipe

Share this post


Link to post

Then Groundspeak is free to spell e.g. "Unknown cache" anyway they please, even "Tuntematon kätkö" B)

 

And if they declare "Unknown Cache" to be element id 423, an application is free to present element 423 in Finnish as Tuntematon kätkö. If it's free-form text subject to change at will, apps really can't localize the strings. It's much more practical for an app to know that 423 represents the concept of a cache of type 'mystery' and choose how to interpret/represent that than for it is to "read" the text.

 

The idea of having an ID instead of text for the type is good, but it would still be bad design to duplicate this it into a new field.

 

The cleaner way to do something like this would be, as robertlipe suggested, an attribute in the current field. Parsers could ignore the text and use the ID if they want to:

 

<Groundspeak:type id="1">Traditional Cache</Groundspeak:type>

<Groundspeak:container id="3">Regular</Groundspeak:container>

Share this post


Link to post
The change that a couple of us are proposing (without looking to see if these numbers are probably "right") is to change

 

<Groundspeak:type>Traditional Cache</Groundspeak:type>

<Groundspeak:container>Regular</Groundspeak:container>

 

to

 

<Groundspeak:type id="1">Traditional Cache</Groundspeak:type>

<Groundspeak:container id="3">Regular</Groundspeak:container>

I've been thinking more about this. Adding identifiers to the type would be excellent, and while we are tearing off the band-aid, maybe well-defined identifiers should be added elsewhere also.

 

Cache size and log type are the most obvious, but it would also be nice to have two-letter identifiers for the US states, as well as 2- or 3-letter standard identifiers for country. So the new code might look like this:

 

<Groundspeak:type id="4">Unknown Cache</Groundspeak:type>

<Groundspeak:container id="1">Regular</Groundspeak:container>

<Groundspeak:country id="USA">United States</Groundspeak:country>

<Groundspeak:state id="TX">Texas</Groundspeak:state>

 

What do people think?

Share this post


Link to post
The change that a couple of us are proposing (without looking to see if these numbers are probably "right") is to change

 

<Groundspeak:type>Traditional Cache</Groundspeak:type>

<Groundspeak:container>Regular</Groundspeak:container>

 

to

 

<Groundspeak:type id="1">Traditional Cache</Groundspeak:type>

<Groundspeak:container id="3">Regular</Groundspeak:container>

I've been thinking more about this. Adding identifiers to the type would be excellent, and while we are tearing off the band-aid, maybe well-defined identifiers should be added elsewhere also.

 

Cache size and log type are the most obvious, but it would also be nice to have two-letter identifiers for the US states, as well as 2- or 3-letter standard identifiers for country. So the new code might look like this:

 

<Groundspeak:type id="4">Unknown Cache</Groundspeak:type>

<Groundspeak:container id="1">Regular</Groundspeak:container>

<Groundspeak:country id="USA">United States</Groundspeak:country>

<Groundspeak:state id="TX">Texas</Groundspeak:state>

 

What do people think?

I like!!

Share this post


Link to post

Selfishly, since noen of *my* software ever parses state or country (hey, you have lat/longs...) but instead treats those totally as pass-throughs, I didn't remember those. I'd extend my rubber stamp to cover those, too..

 

Generically, any place that's "obviously" a separate table tied together, something like a primary key should be exposed.

Share this post


Link to post

The change that a couple of us are Cache size and log type are the most obvious, but it would also be nice to have two-letter identifiers for the US states, as well as 2- or 3-letter standard identifiers for country. So the new code might look like this:

 

<Groundspeak:type id="4">Unknown Cache</Groundspeak:type>

<Groundspeak:container id="1">Regular</Groundspeak:container>

<Groundspeak:country id="USA">United States</Groundspeak:country>

<Groundspeak:state id="TX">Texas</Groundspeak:state>

 

What do people think?

 

Adding the country id is a good plan. I would suggest to use the (two letter) ISO 3166 coding.

 

It may sound like a good plan to add that two letter id for states to caches in the US, but that doesn't work for international caches. Here in The Netherlands the provinces are not specified, but many other country do have their provinces, Bunderslander, or whatever they are called. Simply check the list of "states" when you search for a cache or create a pocket query and you will see what I mean. But a simple numeric id would do the job.

 

As long as Groundspeak publishes all those lookup tables it does not mind what number an id is.

Share this post


Link to post
Adding the country id is a good plan. I would suggest to use the (two letter) ISO 3166 coding.

I think that train left the station about 9 years ago. Without a rewrite of the site and the reassignment of codes for every cache :laughing: the only way I can think of to represent the country-as-defined-by-Groundspeak is the number corresponding to the country's name in the dropdown list.

 

Biting the ISO country codes "bullet" would be good when the site goes multilingual, along with ISO language codes.

Share this post


Link to post
Adding the country id is a good plan. I would suggest to use the (two letter) ISO 3166 coding.

I think that train left the station about 9 years ago. Without a rewrite of the site and the reassignment of codes for every cache :signalviolin: the only way I can think of to represent the country-as-defined-by-Groundspeak is the number corresponding to the country's name in the dropdown list.

I don't understand this. Why would a site rewrite be required to add two-character iSO codes to the countries? Are the countries used by Groundspeak different from the ISO countries?

Share this post


Link to post
Are the countries used by Groundspeak different from the ISO countries?

You just had to open that can o' worms, didn't you? :signalviolin:

Share this post


Link to post

The change that a couple of us are Cache size and log type are the most obvious, but it would also be nice to have two-letter identifiers for the US states, as well as 2- or 3-letter standard identifiers for country. So the new code might look like this:

 

<Groundspeak:type id="4">Unknown Cache</Groundspeak:type>

<Groundspeak:container id="1">Regular</Groundspeak:container>

<Groundspeak:country id="USA">United States</Groundspeak:country>

<Groundspeak:state id="TX">Texas</Groundspeak:state>

 

What do people think?

 

Adding the country id is a good plan. I would suggest to use the (two letter) ISO 3166 coding.

 

It may sound like a good plan to add that two letter id for states to caches in the US, but that doesn't work for international caches. Here in The Netherlands the provinces are not specified, but many other country do have their provinces, Bunderslander, or whatever they are called. Simply check the list of "states" when you search for a cache or create a pocket query and you will see what I mean. But a simple numeric id would do the job.

 

As long as Groundspeak publishes all those lookup tables it does not mind what number an id is.

 

Even the states are defined in ISO 3166-2 as a 2 character field.

 

E.G. Netherlands:

Drenthe NL-DR

Flevoland NL-FL

Fryslân NL-FR

Gelderland NL-GE

Groningen NL-GR

Limburg NL-LI

Noord-Brabant NL-NB

Noord-Holland NL-NH

Overijssel NL-OV

Zuid-Holland NL-ZH

Utrecht NL-UT

Zeeland NL-ZE

Share this post


Link to post

Here in The Netherlands the provinces are not specified, but many other country do have their provinces, Bunderslander, or whatever they are called.

 

Even the states are defined in ISO 3166-2 as a 2 character field.

 

E.G. Netherlands:

Drenthe NL-DR

Flevoland NL-FL

Fryslân NL-FR

Gelderland NL-GE

Groningen NL-GR

Limburg NL-LI

 

You are right. I forgot about that :) I knew that the states/regions/provinces are in ISO 3166-2, but unfortunately (off-topic for this thread) our Dutch provinces are not in the Groundspeak database.

 

From what I see in the html sources of various pages, Groundspeak simply uses a number to classify them. When (not if :D) they add those ids to the GPX file, they should publish the content of the (relevant) tables as well. A set of simple CSV files could easily do the job.

 

They have a country "Unknown" numbered "1". So a 1 on 1 mapping with ISO3166 could cause some issues, although the vast majority of countries should be matchable.

Out of curiosity I created a PQ for caches in country "Unknown" which resulted in zero caches.

Share this post


Link to post

The attributes in the PQs will be very useful to me, and I am really looking forward to it :angry:

So please let me know when the attributes will be back in the PQs.

Edited by Rhintl

Share this post


Link to post

Actually, since I don't think the attributes were the source of the problem, could we get them added back immediately? Then the Unknown issue could be solved later with the new needed checkbox or whatever.

Share this post


Link to post
The change that a couple of us are proposing (without looking to see if these numbers are probably "right") is to change

 

<Groundspeak:type>Traditional Cache</Groundspeak:type>

<Groundspeak:container>Regular</Groundspeak:container>

 

to

 

<Groundspeak:type id="1">Traditional Cache</Groundspeak:type>

<Groundspeak:container id="3">Regular</Groundspeak:container>

I've been thinking more about this. Adding identifiers to the type would be excellent, and while we are tearing off the band-aid, maybe well-defined identifiers should be added elsewhere also.

 

Cache size and log type are the most obvious, but it would also be nice to have two-letter identifiers for the US states, as well as 2- or 3-letter standard identifiers for country. So the new code might look like this:

 

<Groundspeak:type id="4">Unknown Cache</Groundspeak:type>

<Groundspeak:container id="1">Regular</Groundspeak:container>

<Groundspeak:country id="USA">United States</Groundspeak:country>

<Groundspeak:state id="TX">Texas</Groundspeak:state>

 

What do people think?

I like!!

Me too! Proper coding of textual info in GPX files would be a godsend and make parsing a whole bunch easier :)

Don't forget the log types! Eg:

<Groundspeak:log id="12345">
...
<Groundspeak:type id="1">Found it</Groundspeak:type>
...
</Groundspeak:log>

Share this post


Link to post

Actually, since I don't think the attributes were the source of the problem, could we get them added back immediately? Then the Unknown issue could be solved later with the new needed checkbox or whatever.

 

So, could we? :)

Share this post


Link to post

Sorry to be cumbersome here, but I'd would really appreciate an information about the time-schedule when we'll have the attributes back.

 

As a general idea, I'd suggest to implement changes like this as an option that every user can select in his personal settings.

E.g. if we could choose the PQ-version in our profiles, this would provide a much smoother possibility to everyone when introducing or testing anything new

...and the additional efforts were small compared to a full beta-testing, or the current irritations and roll-back alternative.

 

Best wishes

Rhintl

Share this post


Link to post

I'm for putting out a notification of the change with data provided for testing and a date of implementation and then just do it. Don't bother supporting multiple versions and allowing a choice, just declare what the change is and when it well happen.

Share this post


Link to post
Guest
This topic is now closed to further replies.
Sign in to follow this  
Followers 14

×