Jump to content

2500 Character Limit in Personal Cache Notes


hxdimpf

Recommended Posts

First of all, let me apologize if this has been discussed before. I have used the "Search ..." function to find out if this is the case but the search actually never completed. It kept saying "Waiting for geocaching.com ..." and the busy wheel in the browser tab kept spinning.

 

Now to the topic at hand: it is good practice to document what was learned in the context of searching and finding a geocache. For instance the solution of a mystery cache, the answers to field puzzles, the variables found out in the field at different stages of a multi cache. There is one logical place to put this documentation to and this is without any doubt the "Personal Cache Notes" field of the Geocache's listing at geocaching.com. Unfortunately this field is limited to 2500 characters input. While I understand that there have to be explicit limits, I don't understand why it has to be so small. Storage sizes grow and grow, price for storage goes continuously down.

 

Recently I tried to document a new Multi  (https://coord.info/GC8NDNY) and just for the list of required Variables {A .. Z, a .. d} and the associated waypoints, I ended up with 3800 characters, which had to be brutally truncated at 2500 characters ...

 

I would like to suggest that this limit is increased to 25,000 characters. 

Edited by hxdimpf
spelling
  • Upvote 3
  • Funny 2
Link to comment

I have also run into this limit a few times, an increase would be very welcome.

 

Just one example: https://coord.info/GC3WZKT

This challenge cache requires you to list 101 caches that you have found (one for each GC code suffix from 00 to 100).

Putting a list of what you have found so far into the Personal Cache Notes is completely impossible here - the sample list is close to 5000 characters. Of course you can store the list externally. But I think that shouldn't be necessary.

  • Upvote 1
Link to comment
1 hour ago, koeniglich said:

Just one example: https://coord.info/GC3WZKT

This challenge cache requires you to list 101 caches that you have found (one for each GC code suffix from 00 to 100).

Putting a list of what you have found so far into the Personal Cache Notes is completely impossible here - the sample list is close to 5000 characters. Of course you can store the list externally. But I think that shouldn't be necessary.

This is a perfect use for a list of caches. Create a list to hold those caches instead of listing them in the personal note. In my opinion, keeping a list of geocaches to document progress and documenting success for a challenge cache is the most important application of geocache lists.

 

On the one hand, I had a terrible time with personal notes when they first came out because they wouldn't hold much information. I think it was 250 characters back then. I frequently ran up against the limit, and it wasn't always very elegant about how it dealt with my excess data. So I'm sympathetic to the problem.

 

But I have to admit, I've never come anywhere near the 2500 character limit. I didn't even know there still was a limit. Once I have that much personal information for a cache, I want to keep it in a general purpose file on my local system for other reasons. Like, for example, because once there's that much data, I want to be able to bring a serious editor to bear and not be stuck with the geocaching.com text box to edit the data. So, as with all things in any user interface, I agree that it would be nice to have an infinite amount of space, but in this case, I'm also willing to say that the developers seem to have struck a reasonable compromise, so I'll trust them to have weighed the idea of no limit or a larger limit against whatever technical limitations they face in development.

  • Helpful 1
Link to comment
On 5/30/2020 at 12:45 AM, dprovan said:

[... some text deleted ...]

 

But I have to admit, I've never come anywhere near the 2500 character limit. I didn't even know there still was a limit. Once I have that much personal information for a cache, I want to keep it in a general purpose file on my local system for other reasons. Like, for example, because once there's that much data, I want to be able to bring a serious editor to bear and not be stuck with the geocaching.com text box to edit the data. So, as with all things in any user interface, I agree that it would be nice to have an infinite amount of space, but in this case, I'm also willing to say that the developers seem to have struck a reasonable compromise, so I'll trust them to have weighed the idea of no limit or a larger limit against whatever technical limitations they face in development.

 

I also frequently use external technologies as an assist when solving, and documenting mystery puzzles, for instance spreadsheets, however that data is disconnected from the geocache listing on geocaching.com. The authoritative place were at least a reasonably sufficient documentation should go is the personal cache notes field of the geocache listing. After all, that is the only data that gets synchronized to the mobile phone in the process of preparing for a field trip. Frequently I need information readily available in the field for instance in cases where a cache also has the "field puzzle" attribute set and the result of the prep work at home is needed in order to solve tasks out in the field. In many cases there is no mobile signal out there in the woods so it becomes essential to have the required data available offline, on the device in the field. The only available source for that information is the personal cache notes field of the geocache listing in question.

 

I am not asking for infinite amount of capacity, however, 2500 characters is a concern. Any increase would help. If 25,000 characters is considererd too much, I would settle for 10k as well.

 

Also, while at it: The personal cache notes field should be rendered using a monospaced font, otherwise it is not possible to even do a minimum type setting in that field. This has been suggested many times but still third party browser tools are required such as project-gc's tampermonkey script providing the capability to postprocess the rendered listing and turn the personal cache notes field to a monospaced font.

Link to comment
7 hours ago, hxdimpf said:

The authoritative place were at least a reasonably sufficient documentation should go is the personal cache notes field of the geocache listing.

If 2500 characters isn't enough, what would you consider reasonably sufficient? In my head, I imagine the developers explicitly deciding that 2500 was enough.

 

7 hours ago, hxdimpf said:

After all, that is the only data that gets synchronized to the mobile phone in the process of preparing for a field trip. Frequently I need information readily available in the field for instance in cases where a cache also has the "field puzzle" attribute set and the result of the prep work at home is needed in order to solve tasks out in the field. In many cases there is no mobile signal out there in the woods so it becomes essential to have the required data available offline, on the device in the field. The only available source for that information is the personal cache notes field of the geocache listing in question.

First, let me admit that the personal notes aren't downloaded to my GPSr through PQs at all, so I never have them in the field unless I bring them with me or am in a position to get on the net. So you're already getting 2500 more characters of personal data than I am.

 

Having said that, I'm having a really hard time imagining 2500 characters worth of information on my smartphone being useful in the field. That seems well over the point where I'd want to print it out to carry along for reasons completely unrelated to whether any of my devices can show it to me.

 

7 hours ago, hxdimpf said:

I am not asking for infinite amount of capacity, however, 2500 characters is a concern. Any increase would help. If 25,000 characters is considererd too much, I would settle for 10k as well.

I don't know what limitations the developers face, but way back when I did a lot with databases, there was a problem with "infinite space" because the database item had to allocate the maximum amount of space to every item regardless of how much space was actually being used in any given item. If there's any problem like that, increasing the space by 4x or 10x for all personal data users attach to all caches would need to be justified.

 

Even without any such technical limit, there's still a question of how much data is reasonable before the developers should conclude that this isn't someone casually using personal notes, this is someone abusing the database in an attempt to overwhelm their storage with garbage.

 

That's why I think the 2500 limit is a reasonable compromise: any longer, and the management of the data, both at home and in the field, seems so impractical that it would rarely be used. If that's the case -- and I suspect GS could produce evidence to this effect already if they wanted to -- then it becomes reasonable to force people that do want that much data to use other techniques.

 

7 hours ago, hxdimpf said:

Also, while at it: The personal cache notes field should be rendered using a monospaced font, otherwise it is not possible to even do a minimum type setting in that field.

I feel your pain. I certainly would like mono-spaced fonts in a wide range of user interfaces, and I particularly agree that it would be nice in personal notes pages just for my use on the web at home, never mind what happens in the field. But I bet it will never happen. Most people simply don't like mono-spaced fonts and, having grown up without them, they don't see any advantages that makes it worth using something they consider so ugly.

  • Upvote 2
Link to comment

Hard agree.  Especially useful for Mystery and Multi caches.  Also useful for those who use personal notes as a means for documenting their experience with said cache, especially a long-term cache that could take months to complete, which could lead to a particularly intriguing cache find entry. 

 

If this is a technical limitation, however, then there is little we can do.  But it's worth lighting the fire, for sure.

Edited by drdr88
Link to comment

When it comes to large amounts of data like that, depending on the database server, if it's stored as a text block instead of a set character length, the limit is arbitrary. If it were 5000 characters, the row would simple store a pointer to the text data which would be stored however long it is. Each row wouldn't have 5000 characters allocated, only the data required for the pointer. Then a row can have 100 characters of 5000 characters and no space is wasted. It's less optimal because the text data has to be looked up rather than retrieved implicitly with the row, but that's a tradeoff for data columns with large amounts of data. 

All that said, it really depends on how HQ stores their data and which server is used. They may compress the data if not already. And I highly doubt they use a format that allocates 2500 characters (at multiple bytes each) for every row.

If increasing the limit doesn't immediately add terrabytes of data, and the length of data is effectively arbitrary, then the only downfall would be people who immediately make maximum use of the additional characters to use up server space. In the grand scheme, that seems relatively infinitesimal overall. Especially if compressed (text is enormously compressable).

 

However, from my experience, the only time I've ever needed more than 2500 chars in the personal note was storing puzzle data, progress with solutions. And even those, only puzzles that may have large grids of content that I'd be updating.  Earthcache answers? point form, no need to copy questions as well, not an issue. It's really only been those large matrix-style puzzles where the whole thing is needed.  Other than that (even then it's a choice) I can't imagine a situation needing more than 2500 characters that can't be reduced in some manner.

  • Upvote 1
Link to comment
2 hours ago, thebruce0 said:

When it comes to large amounts of data like that, depending on the database server, if it's stored as a text block instead of a set character length, the limit is arbitrary. If it were 5000 characters, the row would simple store a pointer to the text data which would be stored however long it is. Each row wouldn't have 5000 characters allocated, only the data required for the pointer. Then a row can have 100 characters of 5000 characters and no space is wasted. It's less optimal because the text data has to be looked up rather than retrieved implicitly with the row, but that's a tradeoff for data columns with large amounts of data. 

All that said, it really depends on how HQ stores their data and which server is used. They may compress the data if not already. And I highly doubt they use a format that allocates 2500 characters (at multiple bytes each) for every row.

If increasing the limit doesn't immediately add terrabytes of data, and the length of data is effectively arbitrary, then the only downfall would be people who immediately make maximum use of the additional characters to use up server space. In the grand scheme, that seems relatively infinitesimal overall. Especially if compressed (text is enormously compressable).

 

However, from my experience, the only time I've ever needed more than 2500 chars in the personal note was storing puzzle data, progress with solutions. And even those, only puzzles that may have large grids of content that I'd be updating.  Earthcache answers? point form, no need to copy questions as well, not an issue. It's really only been those large matrix-style puzzles where the whole thing is needed.  Other than that (even then it's a choice) I can't imagine a situation needing more than 2500 characters that can't be reduced in some manner.

 

Yes, what you say here resonates well with me. And with respect to capacity: Basic members don't get personal cache notes. Premium members pay a yearly premium ... If I look at my own statistics: 11500 total, 7000 Traditionals, I'd rarly put anything into the PCNs for those, 450 Multis, for these it matters, however, there are perhaps 20 of them where the 2500 char limit is a problem. And if GC HQ would increase the limit, I would certainly not waste that additional capacity just because I can. In the end, the storage they allocate for my data anyway would perhaps increase by 500 ppm.

Link to comment
On 6/12/2020 at 7:47 AM, thebruce0 said:

When it comes to large amounts of data like that, depending on the database server, if it's stored as a text block instead of a set character length, the limit is arbitrary. If it were 5000 characters, the row would simple store a pointer to the text data which would be stored however long it is. Each row wouldn't have 5000 characters allocated, only the data required for the pointer. Then a row can have 100 characters of 5000 characters and no space is wasted. It's less optimal because the text data has to be looked up rather than retrieved implicitly with the row, but that's a tradeoff for data columns with large amounts of data. 

All that said, it really depends on how HQ stores their data and which server is used. They may compress the data if not already. And I highly doubt they use a format that allocates 2500 characters (at multiple bytes each) for every row.

If increasing the limit doesn't immediately add terrabytes of data, and the length of data is effectively arbitrary, then the only downfall would be people who immediately make maximum use of the additional characters to use up server space. In the grand scheme, that seems relatively infinitesimal overall. Especially if compressed (text is enormously compressable).

 

However, from my experience, the only time I've ever needed more than 2500 chars in the personal note was storing puzzle data, progress with solutions. And even those, only puzzles that may have large grids of content that I'd be updating.  Earthcache answers? point form, no need to copy questions as well, not an issue. It's really only been those large matrix-style puzzles where the whole thing is needed.  Other than that (even then it's a choice) I can't imagine a situation needing more than 2500 characters that can't be reduced in some manner.

Though I don't know, I'm fairly certain, GCHQ uses SQL Server and the notes are probably a column in some table of type nvarchar(x) where x can be specified from 1 to 4000 characters or "max". It's probably set to 2500 now. There would be no data size impacts to increase from nvarchar(2500) to nvarchar(4000). switching to max would impact things. Whether one can actually store the specified number of characters depends on what character encoding is used so the actual number may be less than specified. https://docs.microsoft.com/en-us/sql/t-sql/data-types/nchar-and-nvarchar-transact-sql?view=sql-server-ver15

Link to comment
48 minutes ago, Corfman Clan said:

Though I don't know, I'm fairly certain, GCHQ uses...

So you're saying the GS developer is an idiot for limiting the length to 2500 instead of 4000 for no good reason?

 

I figure the developer knows something about it while you and I are just guessing. Since, as I argued above, 2500 seems like plenty to me because any more that 2500 is unwieldy in a text box, I see no reason to second guess the developer and claim he made the wrong decision.

  • Helpful 1
Link to comment
49 minutes ago, dprovan said:

So you're saying the GS developer is an idiot for limiting the length to 2500 instead of 4000 for no good reason?

No, I'm not saying that at all. There are other concerns that would cause one to limit the field size. There are business decisions, UI/UX reasons etc. Ignoring whatever those forces are that caused the notes to be limited to 2500 characters, it would be trivial to increase the limit up to 4000 (probably but not necessarily) if it is an nvarchar column. Beyond that, other factors come into play.

Edited by Corfman Clan
Link to comment

Text and [n/varchar fields are stored differently. Text is there specifically to handle bulk text over 4000 characters. They both have benefits and drawbacks, as I mention in my comment. MS put the threshold at 4000 because that was to them a reasonable threshold to balance fast/small with slow/large.  You wouldn't create a 200 character field as a text type, and you can't create an 8000 character field as a varchar/nvarchar. As a datasystems developer that's my best understanding. A UI/UX reason for limiting to 2500 instead of 400 feels extremely arbitrary. I would assume there was statistical research and data covering everything from internal network bandwidth to user-end bandwidth to server CPU and storage requirements provided before presenting 2500 as the limit, especially with a data system the size of GC.

Link to comment
2 hours ago, thebruce0 said:

Text and [n/varchar fields are stored differently. Text is there specifically to handle bulk text over 4000 characters. They both have benefits and drawbacks, as I mention in my comment. MS put the threshold at 4000 because that was to them a reasonable threshold to balance fast/small with slow/large.  You wouldn't create a 200 character field as a text type, and you can't create an 8000 character field as a varchar/nvarchar. As a datasystems developer that's my best understanding. A UI/UX reason for limiting to 2500 instead of 400 feels extremely arbitrary. I would assume there was statistical research and data covering everything from internal network bandwidth to user-end bandwidth to server CPU and storage requirements provided before presenting 2500 as the limit, especially with a data system the size of GC.

You're out of date. text/ntext/image are deprecated and varchar(max), nvarchar(max), varbinary(max) should be used instead. Using, max instead of n <= 4000 changes the storage, as you mention and as I previously mentioned too.

Link to comment
On 6/26/2020 at 7:48 PM, Corfman Clan said:

text/ntext/image are deprecated and varchar(max), nvarchar(max), varbinary(max) should be used instead. Using, max instead of n <= 4000 changes the storage, as you mention and as I previously mentioned too.

 

Reading up a bit*, n/varchar(max) was introduced in SQL Server 2005. So now we're speaking about a specific database server (thus I want to focus in my comments more about the concept of inline data vs storing a blob elsewhere with a pointer).   However, yes it seems text/etc has been traded for the varchar(max) & variants - but it still accomplishes the same outcome - how the data is stored. From what I've just skimmed, the (max) still senses <=8000 bytes (4000 for nvarchar), and then decides whether to store the data inline or provide a pointer to a blob.  Basically instead of forcing inline (varchar) or forcing a pointer (text), the new unified data type removes that factor on the developer end so the system can make that decision.  varchar(max) with a string of 1000 will be stored inline, but with a string of 10,000 will be stored as a blob.  It would mean the PN field could be extended to any arbitrary length, and the database could decide whether to store a particular value inline or in a blob.

All that said, I'm pretty confident that HQ database developers have already considered all of this, if they are using MS SQL Server 2005 or later...

(on my part, we recently finally upgraded my company to sql 2005, and we are now running 2005, 2008, and 2016 sql servers; and I'm not the primary db developer but I work on them, so the varchar(max) data type addition is good to know)

* I found this stackexchange helpful

Edited by thebruce0
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...