Jump to content

Itsnotaboutthenumbers


Rick618

Recommended Posts

It doesn't look to be working today...

 

"Sigh.

 

Once again, the site has used up too many of the hosting server's resources and was turned off.

 

Not sure what I'm going to do at this point. My choices are pretty much to either completely reduce the number of stats that are reported or move the whole shebang to a dedicated server. Unfortunately, the second option costs about $200/month so that's pretty much out.

 

I'm probably going to go with the first, but I'm not sure how long that will take to accomplish. Please check back later.

 

Sorry.

 

-Dave (The Cheeseheads)"

 

:o

Link to comment

I would agree with Dave's inclination to reduce the number of reported stats rather than shut down the site. In my opinion, the detail in statistics that were being provided went far beyond what was reasonably needed. If removing superfluous detail is needed in order to keep even the maps, I would vote for that.

 

Hopefully Dave will find a way to keep the site up some way. Good Luck!

Link to comment

It doesn't look to be working today...

 

"Sigh.

 

Once again, the site has used up too many of the hosting server's resources and was turned off.

 

Not sure what I'm going to do at this point. My choices are pretty much to either completely reduce the number of stats that are reported or move the whole shebang to a dedicated server. Unfortunately, the second option costs about $200/month so that's pretty much out.

 

I'm probably going to go with the first, but I'm not sure how long that will take to accomplish. Please check back later.

 

Sorry.

 

-Dave (The Cheeseheads)"

 

:o

 

Dave-

 

Are you maxxing out the number of mysql queries you're allowed? How much page caching have you implemented?

Link to comment

If removing superfluous detail is needed in order to keep even the maps, I would vote for that.

 

Hopefully Dave will find a way to keep the site up some way. Good Luck!

 

I'll second the keeping the maps.

 

Also, I really hope Dave will find a way to keep up his great service.

Link to comment

Maybe he could have a "premium members" level where you get more stats than the free version. That would help pay for the dedicated server. I would pay $5 or $10 a month for it. Find 20 more cachers and its paid for!

There's was a link to make PayPal donations on the main page. Maybe he could add that link back to his "sigh" message?

Link to comment

Maybe he could have a "premium members" level where you get more stats than the free version. That would help pay for the dedicated server. I would pay $5 or $10 a month for it. Find 20 more cachers and its paid for!

 

I, for one, would probably support this. Perhaps a "free" version could generate basic statistics that would only need to be calculated once and could be stored as text, and the "premium" version could offer mapping capability for a few dollars a month.

Link to comment

How about offering an option screen that allows you to run what you are interested in, check boxes like the google map feature. Default is the least amount of stuff, then you can choose x-# of extra options per run and limit the number of runs per account. Maybe once every seven days like the my finds pg?

Link to comment

I haven't decided what I'm going to do yet. One problem is the table that holds all the log entries, which currently has 7.1 million rows. :D Even though I have that well-indexed, it still takes some effort to roll through that.

 

The other thing is the county maps. There's a fair amount of math involved to figure out in what county a cache falls. The county maps are really popular though (a lot of County Challenge caches use my site for verification), so I'd hate to get rid of them.

 

A lot of people are offering to make donations. Thanks, but I'm not going to put the Paypal links back up until I decide what I want to do. I'd hate to have people donate, then I end up paring the site down to a level that's not worth the $$$.

 

Thanks for all the support. It's nice to know my site is this popular! :)

Link to comment

I haven't decided what I'm going to do yet. One problem is the table that holds all the log entries, which currently has 7.1 million rows. :D Even though I have that well-indexed, it still takes some effort to roll through that.

 

The other thing is the county maps. There's a fair amount of math involved to figure out in what county a cache falls. The county maps are really popular though (a lot of County Challenge caches use my site for verification), so I'd hate to get rid of them.

 

A lot of people are offering to make donations. Thanks, but I'm not going to put the Paypal links back up until I decide what I want to do. I'd hate to have people donate, then I end up paring the site down to a level that's not worth the $$$.

 

Thanks for all the support. It's nice to know my site is this popular! :)

 

So why do you ignore all suggestions for help or improvements to the code?

 

I bet you could reduce your server load massively with some code optimizations, which many here are very willing to help with.

 

-Ben

Link to comment

I haven't decided what I'm going to do yet. One problem is the table that holds all the log entries, which currently has 7.1 million rows. :lol: Even though I have that well-indexed, it still takes some effort to roll through that.

 

The other thing is the county maps. There's a fair amount of math involved to figure out in what county a cache falls. The county maps are really popular though (a lot of County Challenge caches use my site for verification), so I'd hate to get rid of them.

 

A lot of people are offering to make donations. Thanks, but I'm not going to put the Paypal links back up until I decide what I want to do. I'd hate to have people donate, then I end up paring the site down to a level that's not worth the $$$.

 

Thanks for all the support. It's nice to know my site is this popular! :wub:

 

So why do you ignore all suggestions for help or improvements to the code?

 

I bet you could reduce your server load massively with some code optimizations, which many here are very willing to help with.

 

-Ben

 

How do you know he is not taking suggestions? Just because the discussions aren't public doesn't mean they are not happening.

 

How do you know the problem is code optimization? Maybe the problem simply is the site is getting hammered with lots of requests. He did say one thing that takes lots of compute is the county maps which are very popular. Low costs sites don't allow a lot of resource usage and I bet even with highly optimized code he would still trip the counters.

 

Jim

Link to comment

The "what county is this cache is?" function only needs to run once for each new cache found. That function already runs in background. I have no idea how many new caches need to be "researched" daily, but it would seem that the priority for that job could be lowered if that is causing the resource problem.

 

(Although, I guess, you could get to the point of never getting caught up...)

Link to comment

The "what county is this cache is?" function only needs to run once for each new cache found. That function already runs in background. I have no idea how many new caches need to be "researched" daily, but it would seem that the priority for that job could be lowered if that is causing the resource problem.

 

(Although, I guess, you could get to the point of never getting caught up...)

I haven't researched the numbers, but figure that every Monday hundreds of cachers are uploading they PQs filled with all their finds on the new caches that were listed over the weekend.

Link to comment

Would it be possible to split the load among two or more cheap shared servers? Have one server do nothing but process PQ's and transfer the data to a second server. The second server would handle all the front-door stuff, such as hosting images for display on people's profiles.

 

And here are my suggestions for a two-tiered approach:

 

Free service:

- Limited uploads (maximum of once every 14 days instead of every 7)

- No image hosting. If you want maps and stats to display on your profile, you have to upload them to geocaching.com or photobucket or imageshack for hosting.

 

Premium service (Say $2/month-$20/year or maybe $3/month-$30/year):

- Unlimited uploads (the only limit is Groundspeak's 7-day limit)

- Image/statistic hosting equal to current level

Link to comment
I haven't researched the numbers, but figure that every Monday hundreds of cachers are uploading they PQs filled with all their finds on the new caches that were listed over the weekend.

I'm curious. Are they uploading their entire Found PQ or just the finds from the past week?

 

You could also investigate using flash-based maps in place of some of the other created-on-the-fly maps you do have. Allow raster maps on a limited basis.

Link to comment
I'm curious. Are they uploading their entire Found PQ or just the finds from the past week?

The site does make you upload the entire zipped All Finds PQ. That is part of the ease of using the site. I don't have to modify my All Finds PQ and upload part of it. Easy on me but harder on the INATN site.

Link to comment

What kind of reduction on the server would you have if you eliminated the compare cachers feature?

 

I would also support a twice a month limit and no image hosting. I would imagine that the dynamic linking on the Groundspeak profile pages take a lot of hits. I have hosted my own images since the sites inception.

 

Some of the stats could also be a once a year special request. For instance, milestone caches. They never change and most people can keep track of their upcoming milestones for themselves.

Link to comment
There's a fair amount of math involved to figure out in what county a cache falls.

 

This is a perfect example of a place where you might be able to see huge rewards from optimization. Unless county boundaries are changing more often than I think they are, you should only have to do this calculation ONCE for each new cache and then store the value somewhere, whether in DB or TXT file. If counties DO change that often, maybe you'd have to refresh this data in the evening every month or so.

 

Or maybe you're already doing this?

 

Also, I have noticed on several of my sites that, for example, where a complicated query may take perhaps 15 seconds to complete, doing the same query as 3 or 4 consecutive smaller queries might get done in half a second. There's usually a bit more optimization if you experiment.

Edited by mvigor
Link to comment
There's a fair amount of math involved to figure out in what county a cache falls.

 

you should only have to do this calculation ONCE for each new cache and then store the value somewhere, whether in DB or TXT file.

 

Or maybe you're already doing this?

 

He does already do that. If you use his site, you may have noticed that after you upload your myfinds file, after a relatively short time all the county information is known about your caches, except possibly for a few that are clearly indicated. Those are the ones that the one-time process has to run to determine where the cache is.

 

And that is also why, when a cache‘s coordinates are corrected and that "moves" it to another county, he needs to intervene to fix it.

Link to comment
I'm curious. Are they uploading their entire Found PQ or just the finds from the past week?
The site does make you upload the entire zipped All Finds PQ. That is part of the ease of using the site. I don't have to modify my All Finds PQ and upload part of it. Easy on me but harder on the INATN site.

I haven't used it in a while except as to be curious about the site itself. It seems as though allowing power users to upload PQs with only those caches found in the last 7 days would drop the load tremendously. After all, previous finds aren't going to change and the site is doing useless work going through them. That could drop a weekly user's input from thousands to less than a hundred.

 

There might be the occasional cache that is archived before it exported, but those could be caught up with a full PQ only when needed.

 

Of course, that won't work for the occasional user, but any savings with help.

 

The idea is that even though computers are great for repetitive work, they can only do so much in a given time.

Link to comment

I'm convinced it's all my fault. You see, last week my daughter and I went on a 1750 mile spring break geo-roadtrip through nine states, for the sole purpose of getting the colors on our INATN maps to look pretty. (Well, we also did stuff like visiting Mammoth Cave and the Smokey Mountains, but please play along, I'm trying to be dramatic.) We ended up with cache finds in 39 different counties, 70 INATN map "color changes" for me in those counties, and 8 state map color changes.

 

So, I log all my finds, which took three days since I write unique logs for every cache (with an average length of 101 words, INATN tells me). I upload my query, then I do all my stats and spreadsheets and images and profile updating. Then I turn my attention to my daughter's account, and logging her finds based on the notes she made. I pressed the button to order her "All Finds" query, and sure enough, when I went to INATN to upload it, *that's* when the site went down. My daughter is none too pleased with me since *my* maps got updated.

 

I tell this story not to complain, but to illustrate how the INATN website has revolutionized and revitalized geocaching for us since its debut. We used it to keep track of our Pennsylvania travels last year as we met our goal of finding caches in all 67 counties, "coloring in our maps" along the way. We hope to do the same thing in Ohio's 88 counties this year. We are traveling more, and worrying less about cleaning out a radius of caches around our house. We derive enjoyment even from the wettest of film canisters under lamp posts, because that cache helped us achieve the goal of reaching X finds in Y county. I also rely on the site for helping verify finds by other geocachers on our state's "All Counties" challenge, of which I am a co-owner.

 

I hope the site will be back soon. I would be happy with a slimmed-down site that let me do the maps the way they were, and gave me some of my personal statistics. Since I don't regard geocaching as a competition, I could do without the comparative statistics, top ten lists, and even the ability to look up someone else's stats.

 

In its form prior to being taken down, I would be willing to pay up to $40 per year for using the site. For just the maps and personal statistics, I'd happily pay $20 per year.

 

Thank you to The Cheeseheads for the work on this site. Let us all know what we can do.

Link to comment

I tell this story not to complain, but to illustrate how the INATN website has revolutionized and revitalized geocaching for us since its debut.

 

That is certainly true for me too. Seeing the uncolored counties on the map was an incentive to finish the Ohio 88 and the Indiana 92. Recently my wife wanted to fly down to Key West. "Fine", I said "I'll drive and meet you there."

 

If you look at my US county finds you'll see the "breadcrumbs" I left on the trip down (and back, on a different path, of course.)

 

And I was just about to upload my file since I had turned the color of two nearby counties red (100+) ... and discovered The Leprechauns had broken this site. <_<

Link to comment

How about a pay structure like this:

 

Free:

Everyone that plays nice, and they get all the benefits that are offered now, one upload a week.

 

$30/Month:

Anyone that crashes the site while uploading their daughters information, just to see pretty colors.

 

But I like pretty colors.

 

Ooooooh.

Link to comment

Would it be possible to split the load among two or more cheap shared servers? Have one server do nothing but process PQ's and transfer the data to a second server. The second server would handle all the front-door stuff, such as hosting images for display on people's profiles.

 

And here are my suggestions for a two-tiered approach:

 

Free service:

- Limited uploads (maximum of once every 14 days instead of every 7)

- No image hosting. If you want maps and stats to display on your profile, you have to upload them to geocaching.com or photobucket or imageshack for hosting.

 

Premium service (Say $2/month-$20/year or maybe $3/month-$30/year):

- Unlimited uploads (the only limit is Groundspeak's 7-day limit)

- Image/statistic hosting equal to current level

 

I'd be up for a pay structure, though I would make one other change. Dave mentioned that the county issue was one of the most taxing parts of the site. If it's possible to make this aspect selective, then change the part that determines the county locations for caches a premium service. That way, people can get basic stats if they don't want to pay, and the detailed information if they are willing to pay. Since that seems to be one of the more popular aspects of the site, with a nice affordable price, I'm sure that many would be willing to pay a small fee for that service, thus bringing in money to pay for a dedicated server. (Is it just the county aspect that is most taxing? If there are other aspects, I'd say take the stuff that is hardest on the server and make them parts you get with premium service.)

 

Someone mentioned splitting up the resources onto more than one server. Sounds like an option to check out.

 

Of course, the pay structure involving the Leprechauns didn't sound too bad either. ;):grin:;)

Link to comment

Three points we would like to make

 

1) Agree totally with the comments about how INATN has helped transform caching - we may not have that many counties over here but we have really enjoyed watching our crumb trails grow. Likewise to all the other good things people have said about this great site. ;)

 

2) We would also be prepared to pay for the service (especially if we can fix the dollar pound exchange rate for the next five years now :grin: )

 

3) And, maybe most importantly, please don't confuse us with The Leprechauns - it wosn't us wot broke it :D;)

Link to comment

Add me to the list of willing to pay $20.00 a year.

 

Can't be as much as geocaching but $10 less is acceptable.

It was cool seeing your name in some of the same Tennessee logs recently.

 

With your statistics, maps and Cache Across America achievements, please tell us that you're willing to participate in this effort as a donor at the Leprechaun Level. ;)

Link to comment

Add me to the list of willing to pay $20.00 a year.

 

Can't be as much as geocaching but $10 less is acceptable.

It was cool seeing your name in some of the same Tennessee logs recently.

 

With your statistics, maps and Cache Across America achievements, please tell us that you're willing to participate in this effort as a donor at the Leprechaun Level. ;)

 

I signed a couple of logs right after you also.

 

That being said I will match the first year donation made by "The Leprechauns"

Link to comment

Add me to the list of willing to pay $20.00 a year.

 

Can't be as much as geocaching but $10 less is acceptable.

It was cool seeing your name in some of the same Tennessee logs recently.

 

With your statistics, maps and Cache Across America achievements, please tell us that you're willing to participate in this effort as a donor at the Leprechaun Level. :unsure:

 

So, do I need to make the Leprechaun donation in standard currency or in small, colorful marshmellow shapes?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...