Jump to content

Site Performance Being Addressed


mtn-man

Recommended Posts

In February, Jeremy posted that they are aware at Groundspeak that there are performance issues with the site. These issues come up on the weekends, usually on Sunday and Monday as people log finds from the weekends. Topics have been posted regarding these issues, but I decided to quote Jeremy's post here and stick the topic to the top of the Geocaching.com Web Site Forum until it is no longer needed.

 

Here is Jeremy's post, which is still located in the Geocaching Announcements Section.

 

The site has been performing a little better over the last few weeks, but today we had a short bout of "server too busy errors" and on Saturday there was an issue with one of the drives going sour causing disk errors. So, I figured I would provide an update on our ongoing quest for the scalable server.

 

As I've indicated before, the site was originally built on hobby code (to put it lightly). Fortunately we're now at the point where we can bring in the Big Guns - or People Smarter than Us. These are the folks that have had a lot of experience with high-traffic web sites like ours.

 

Of course the site itself, once a pretty basic site has now become an enormous Katamari ball, filled with just about every cobbled together feature. Although we have hired folks to work on the task the ball needs to be broken apart and examined to find out what needs to be fixed and what needs to be replaced.

 

We've been in this process for the past 6-8 weeks now and we're currently running tests against suggested changes to make sure they, in fact, make a difference. Some of the changes can be pretty dramatic so things like losing accounts or deleting logs would be a Bad Thing. So we're being careful. We're also doing a lot of house cleaning - losing old tables we don't use and removing functionality that never made its way to the site. This will just make it easier in the coming months as we do more retooling for the site.

 

Again, this is annoying to me far more than any of you can imagine. It's one of those things that have followed us around since the beginning and I will be happy to see the scalability issues go away. I assure you that the majority of our resources, both people and financial, are addressing this issue. I thank you again for your patience as we look for a permanent solution.

I am sure we all hope these issues will be resolved soon.

They are aware and they are working to correct it.

Link to comment

OK. I renewed my membership recently and am looking forward to the upgrades and/or code rewrites.

 

I understand that in the beginning it would have been very hard to work off of an established model because there really wasn't anything to emulate. GC.Com is rather unique in it's demands on coders and resources.

 

Me, I'll hang in here.

 

 

 

 

 

 

[How'd I do on the suck-up meter? :anibad: ]

Link to comment

This is getting really bad. Do I have to wait until Monday to log my finds because the site can't handle the load?!! Then what happens when Monday fills up?

It's been going on for quite a while now.

Is this issue really being addressed with any urgency? :(:ph34r::sad:

Link to comment
Is this issue really being addressed with any urgency? :(:ph34r::sad:

I think the fact that Jeremy hasn't been in the forums at all for several weeks speaks volumes. It tells me that he is working hard to fix the problem.

 

Would you rather have TPTB sitting around spending their time apologizing in the forum, or working on fixing the problem?

Link to comment
Would you rather have TPTB sitting around spending their time apologizing in the forum, or working on fixing the problem?

I need no apologies so spending time fixing the problem is the obvious answer. Furthermore, I make my living writing software and I know what a pain it can be to try and replace a poorly architected legacy. So I am not surprised that this is taking awhile.

 

But, having said that, I am distrustful of progress when I don't know what the plan of attack is and the prognosis is for improvement. So, what I would like is some details on what's being done and how long it's going to take to complete the work. As someone who is helping to pay those consultants, I don't think that's too much to ask...

Link to comment

The site has been developing for about six and a half years now. There are a lot of features integrated into this site over that time. I would not expect that you could interview prospective analyst, make the decision which ones would be the right ones to tackle the problem, have them take the site apart and analyze each and every aspect of the site and its features, examine possible solutions, present these recommendations and then analyze those recommendations to make sure one fix doesn't break something else in only five weeks.

 

My impression is that they not going for a short term, quick fix.

My impression is that they are now looking long term.

 

Trust me, I know it is hard to be patient. As reviewers, we have to look at not just one cache page, but also the nearby caches and any additional waypoints associated with them. These times when the site slows down affects us as well, in some ways more so. As Lil Devil says, Jeremy has been pretty vacant from the forums. I imagine he and the other guys are trying to focus.

Link to comment

and to think that all this time I've thought the "Server Busy" problems were due to not having enough processing power behind a fat enough pipe to handle the traffic... now we find out its just the cobbled together code base that is to blame. I can understand that it takes more time and grace to fix the software than it does to slap another box into the mix.

 

Edit: ...but that doesn't make it any less annoying :sad:

Edited by mini cacher
Link to comment
Is this issue really being addressed with any urgency? :sad::(:ph34r:

I think the fact that Jeremy hasn't been in the forums at all for several weeks speaks volumes. It tells me that he is working hard to fix the problem.

 

Would you rather have TPTB sitting around spending their time apologizing in the forum, or working on fixing the problem?

 

I would much rather have TPTB fixing than sitting

Link to comment

Update from today from Jeremy in the Announcements section once again.

:laughing::):D Thanks for the update Jeremy! :D:D:o

 

It's been a while since I've given an update on our work towards resolving the Server Too Busy issues.

 

With help from our consulting partner we have been digging into the database and resolving some of the big database hogs - resulting in an alleviation of some pressure but obviously not resolving the times when traffic simply overwhelms the site. Since this is a big issue there are many smaller things going on to defeat Goliath.

 

First, we have put a specification for a new machine. This is a very straightforward approach by adding more "iron" to the problem. It's a stopgap measure and certainly not the final solution. However it accomplishes several goals:

  • Creates a fresh install (always good)
  • Gives us time to reconfigure the database so it runs faster
  • Gives it an extra oomph with disk space, memory, etc.

The second point is actually the most important. Currently the database is configured in such a way that it causes the disk drives to work harder than they need to. By moving disks around to be dedicated to certain tasks, like backups, the database ultimately speeds up.

 

In addition to this we're still scouring the logs, especially during server too busy times, to tweak the most common queries that take too long. Since the Server Too Busy errors happen when there are too many queries in the queue, speeding up these queries will reduce the queue and help resolve the issue.

 

We're also starting a documentation stage for Geocaching.com which will look at the entire site and recommend a new version of the back-end code that helps to fix this for good. This will be a 6-8 month project so in parallel we'll be working hard to make sure the site functions well in the current system to allow us time to create a permanent solution.

 

Keep on geocaching - and remember - Sundays and Mondays are the slowest times of the week. If you visit Tuesday-Thursday you'll have a better experience logging caches. It isn't the right solution but it helps if you get aggrivated by the slow times.

 

And as always, thanks everyone for your patience and support. We're working hard on the problem even though it may seem like nothing on the site is changing.

Link to comment

One of the worst problems a company can have is growth from success. It’s easy to grow slowly. It’s hard to grow fast.

 

The problem is almost always money. The specific issue really doesn’t matter unless there is no solution. The problem is money because when you hit a wall with your company the next step means a major investment in equipment and people. The investment can kill a firm.

 

Take SD Cards. They are cheap. 24 bucks and you can store more music than you can listen to in a week. The factory to make a single SD card costs billions. If after building that factory you don’t sell enough SD Cards to cover costs…you go bankrupt. If you do nothing you lose customers to the company that does make the factory and having a newer factory they can sell them cheaper and you go bankrupt. There are more options but they all pretty much suck. It’s the problem most companies dream of having. Too much success.

 

If the formula holds true. GC.com (and any other listing site) will grow until they hit the maximum amount of traffic they can handle. Then they start having problems delivering services to customers. The fix, migrate to the next level of equipment, plus the staff or contract to take care of it. But you have to make an investment that is far and away more than originally anticipated back when they had no idea where this activity would take them. They will have to look, look at the viability of non paying customers. If they don’t have enough premium members to pay for the equipment and the cost of running it they will have to change something. Increase prices and you can lose paying customers. If everyone pays…he breaks a promise (one that I never saw as realistic because of this problem) and loses a lot of customers. If he holds back on the required improvements until premium members can cover costs, he loses customers due to ongoing performance issues.

 

I have no doubt there is equipment and people who can tackle the job. It’s a question of how to pay for it. There are options. Some will kill geoacaching.com, some may merely maim it. Others may take this company to the next level where it thrives but that may have a price that in the end spells out the doom of geocaching.com or a loss of at least some managerial control of the present PTB.

 

Whether or not Geocaching.com has reached this point, I can only speculate. The signs from my end as a user seem to indicate that it has. Long Story short. If I’m right the solution won’t be quick, any other listing site would have the same problem, there is potential for failure, and success may require changes in the way this site does business that can make or break this site.

 

Long story short, it’s not an easy fix and don’t expect quick turn around, but I’ll bet TPTB are working like never before to get it solved. (It's also 100% Speculation on my part).

Edited by Renegade Knight
Link to comment

Given the vast level of integration with logs, photos, listings, searches etc - I know this site works harder than most to deliver the information. I appreciate that a fix is in the works and that resources have been allocated to solving it in a financially viable manner. I have very seldom seen problems that others have but I do occasionally see a slow-down. Nice to know that it will be solved. Have patience.

Link to comment

Okay, well I was just dropping by to see if others were having the same issues. That question has been answered. I feel for everybody at the Headquarters, COST VS. REWARD. I'm thinking that I caused this whole thing. I forgot to renew my Premium membership at the end of February and that is when this whole ball of wax started. !!!!!Gosh, was I that piece of straw that broke the camels back?!!!!!

Well my thoughts are with Jeremy....and my wife, as she has to deal with me when this highly technologic POS staring me in the face becomes a Comcast Slowskys idol and I start turning red. My wife actually said to me," Well I guess you'll just have to stop Geocaching if that's how there website is going to be now." I just told her to get back to watching her Desperate Houswives. Well let's just hope this all gets taken care of before somebody goes out and designs the new Server Too Busy Geocoin. Oh man, did I just type that.....

Link to comment

Keep on geocaching - and remember - Sundays and Mondays are the slowest times of the week. If you visit Tuesday-Thursday you'll have a better experience logging caches. It isn't the right solution but it helps if you get aggrivated by the slow times.

 

OOh, now I think I'll save all my logging and searching till Friday's or Saturdays, that should do the trick at least until others take the same hint. But of course, ke others, it's most frustrating whenever you are actually trying to do something and can't.

Link to comment

In addition to this we're still scouring the logs, especially during server too busy times, to tweak the most common queries that take too long. Since the Server Too Busy errors happen when there are too many queries in the queue, speeding up these queries will reduce the queue and help resolve the issue.

Hey, cool we appreciate knowing what's going on. Could you by any chance make this information (the slowest queries) public, so we can know what drains the system and try to avoid those actions? For instance, if loading a cache page were the bottleneck, there's a greasemonkey script that adds a log link to the cache search page to bypass it. It would be helpful to know what to avoid and work around.

Link to comment

I'm realy angry with the GC.com:

 

Every day at 11:00 am I cannot get any acces to any GC. What that ****?????? It apperas only a "Time out"

 

I tried it with sereveral Browsers and serveral PCs every time the same ****.

 

What is the problem you have with the servers. Make them availible all the time!!!!!!!!!!!!!

 

GC.com, I'm paying 30 $ a year so make the server in Europe availible 24h/7days a week!!!!!!!!!

 

<_<<_<:ph34r::D:D:D:D:D:D:D:D:D

Link to comment

I think $30 a year is a very reasonable sum compared to the enjoyment I get out of this site. That alone buys a LOT of patience on my part! (And believe me, patience does not come easily in the Vanillahip household!)

Edited by Vanillahip
Link to comment

I think $30 a year is a very reasonable sum compared to the enjoyment I get out of this site.

 

Not ever had any real problems, I understand that on Sunday evening and Monday, all the weekend finds are going in and the server is busy. You see the same sort of back-up everyday on your commute. Things I will try to do:

 

1. Log on weekend at odd hours, or wait until Tuesday.

2. PQ for near-by caches, those close to home, is the only one set to run weekly. Will move to Thursday, or even check usefulness and maybe just run bi-weekly or as needed.

 

Thank you to all those working in the office to keep the system up and running.

Link to comment
:P Wow, a lotta angst out there over this. I enjoy the many features I get to use for the bargain price of $30 a year. If I have to be patient once in a while, so be it. Calm down folks. It's not as if the whole site has shut down completely, it's just slow once in a while. Read the paper, eat a cookie, or take a valium and chill out.
Link to comment

Yep, amazing what some people will post.

 

I am on the site either reviewing or posting logs every day. Ironically, I have been on a streak trying to find a cache every day at this point. I am logging finds just about every day, or maybe skipping a day. I have been listing caches just about every day. While I see slowdowns temporarily, the site still moves along. I don't review caches after noon EDT on Sunday though to stay out of folks way, for example.

 

Off to log finds for the last couple of days.

Link to comment

:laughing: Wow, a lotta angst out there over this. I enjoy the many features I get to use for the bargain price of $30 a year. If I have to be patient once in a while, so be it. Calm down folks. It's not as if the whole site has shut down completely, it's just slow once in a while. Read the paper, eat a cookie, or take a valium and chill out.

 

People were much nicer around here five years ago when we were a much smaller community. Too bad geocaching is so much darn fun and attracts all sorts of folks. Back in my day... I'm going to make a good grandfather in 40 years, I think!

 

Thanks for working on the server issues (which I almost never experience personally). Much appreciated.

Link to comment

:laughing: Wow, a lotta angst out there over this. I enjoy the many features I get to use for the bargain price of $30 a year. If I have to be patient once in a while, so be it. Calm down folks. It's not as if the whole site has shut down completely, it's just slow once in a while. Read the paper, eat a cookie, or take a valium and chill out.

 

People were much nicer around here five years ago when we were a much smaller community. Too bad geocaching is so much darn fun and attracts all sorts of folks. Back in my day... I'm going to make a good grandfather in 40 years, I think!

 

Thanks for working on the server issues (which I almost never experience personally). Much appreciated.

30.00 for a year of keeping our found caches in order, downloading new ones, the forums. What a lot of fun for just 30 bucks. We go out to eat and 30.00 in the minimum for a nice meal and we have forgotten about it in a day. Cool down folks....when things are a little slow do something else for pete's sake. I grumble a little sometime but just put it aside and do something else.

Link to comment

java script:add_smilie(":smile:","smid_15")

mad.gif

 

www.geocaching.com has been down for me the last several days. I couldn't even get the front page to load. Firefox would just go into "Waiting for www.geocaching.com..." and then do nothing. This happen on two different computers and at different times. Log onto the forms to see what was wrong and didn't see anyone writing about it. I then tried it using "Internet Explorer" and found that gc.com works find with IE, and seems to be much faster. If they have put up a new version, it looks to me as if the new version is using some feature that Firefix does not like and causing it to stop working.

 

Carl

Link to comment

There's been no upgrades to the website in recent weeks. All of the performance improvements are being tested on a development server.

 

What CRF describes is a fairly common Firefox problem. Try clearing your cookies and browser cache. Getting a fresh start usually clears up the problem. :smile:

Link to comment

Moderator's Note: This post was originally the OP in a separate thread entitled "Site gets slower and slower, then it stops." I merged the duplicate thread into this one.

 

Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

 

Exception Details: System.Data.SqlClient.SqlException: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

 

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

 

Stack Trace:

 

[sqlException: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.]

System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream) +742

System.Data.SqlClient.SqlCommand.ExecuteReader() +41

System.Web.SessionState.SqlStateClientManager.DoGet(String id, SqlCommand cmd) +380

 

[HttpException (0x80004005): Unable to connect to SQL Server session database.]

System.Web.SessionState.SqlStateClientManager.DoGet(String id, SqlCommand cmd) +535

System.Web.SessionState.SqlStateClientManager.GetExclusive(String id) +81

System.Web.SessionState.SqlStateClientManager.System.Web.SessionState.IStateClientManager.BeginGetExclusive(String id, AsyncCallback cb, Object state) +6

System.Web.SessionState.SessionStateModule.GetSessionStateItem() +67

System.Web.SessionState.SessionStateModule.BeginAcquireState(Object source, EventArgs e, AsyncCallback cb, Object extraData) +274

System.Web.AsyncEventExecutionStep.System.Web.HttpApplication+IExecutionStep.Execute() +66

System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +173

Edited by Keystone
Link to comment

There's been no upgrades to the website in recent weeks. All of the performance improvements are being tested on a development server.

 

What CRF describes is a fairly common Firefox problem. Try clearing your cookies and browser cache. Getting a fresh start usually clears up the problem. :P

 

sure, there is nothing wrong with geocaching.com performance wise.

 

keep on dreaming fool...

Link to comment
keep on dreaming fool...

From the Forum Guidelines linked at the top of every forum page:

Respect: Respect the guidelines for forum usage, and site usage. Respect Groundspeak, its employees, volunteers, yourself, fellow community members, and guests on these boards. Whether a community member has one post or 5,000 posts, they deserve the same respect.

 

Personal Attacks and Flames will not be tolerated. If you want to praise or criticize, give examples as to why it is good or bad, general attacks on a person or idea will not be tolerated.

While I understand that this is frustrating, further attacks of this nature in this topic will result in disabled posting rights. There is absolutely no call for this type of personal attack. What is sad is that you attack him calling him a "fool" when he said nothing about there being no problems with the site. He was offering the user a possible solution to the issue they were having. In fact, CRF says the site works fine in Internet Explorer for him and was talking about a Firefox problem, not a site problem. Perhaps instead of calling people names, you could instead be helpful, as Keystone was.

 

hogrod, please read the topic title and the early postings by me in this topic where I quote Jeremy. You will see that this is exactly what is taking place.

Edited by mtn-man
Link to comment

java script:add_smilie(":(","smid_15")

mad.gif

 

www.geocaching.com has been down for me the last several days. I couldn't even get the front page to load. Firefox would just go into "Waiting for www.geocaching.com..." and then do nothing. This happen on two different computers and at different times. Log onto the forms to see what was wrong and didn't see anyone writing about it. I then tried it using "Internet Explorer" and found that gc.com works find with IE, and seems to be much faster. If they have put up a new version, it looks to me as if the new version is using some feature that Firefix does not like and causing it to stop working.

 

Carl

 

There's been no upgrades to the website in recent weeks. All of the performance improvements are being tested on a development server.

 

What CRF describes is a fairly common Firefox problem. Try clearing your cookies and browser cache. Getting a fresh start usually clears up the problem. :)

 

Ah, that helps a lot!

 

I've been trying to log some finds using FF and it's died on me completely. Switched to IE and all seems fine now.

 

Think I'll use IE for logging and FF for everything else :D

 

Cheers for the info Keystone :)

Link to comment
CRF says the site works fine in Internet Explorer for him and was talking about a Firefox problem, not a site problem.
FWIW, just because it works in one browser and not the other does not always mean it is not a site problem. A site that has mal-formed code can work just fine in one browser and not in an other due to the nature of how *some* browsers will take any garbage code on par with its own ... while others adhire to a higher standard. I won't claim this is the case here... but you never know.

 

Edit: ... but name calling is never nice. :(

Edited by mini cacher
Link to comment

OK, I realize the server issue is a big one and we keep hearing that work is being done, but from the standpoint of the patient user, this is extremely frustrating. It just took me over an hour and a half to log 10 finds from yesterday. Where is the bloody progress? If it's true that things work better in IE, I will reluctantly use that browser.

Link to comment

Recently renewed only to find the same old performance problem today. Sunday 2:57 PDT

 

Server Error in '/' Application.

Server Too Busy

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

 

Exception Details: System.Web.HttpException: Server Too Busy

 

Source Error:

 

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

 

Stack Trace:

 

[HttpException (0x80004005): Server Too Busy]

System.Web.HttpRuntime.RejectRequestInternal(HttpWorkerRequest wr) +146

 

Version Information: Microsoft .NET Framework Version:1.1.4322.2300; ASP.NET Version:1.1.4322.2300

Link to comment

The general site problems have nothing, repeat nothing, to do with your browser.

 

Thousands of people use this site every day. If 10000 people are having a server-related slowdown and one person has a Firefox-related cache issue, that doesn't mean that Firefox is the problem. It's like one person gets a brain tumour and decides it's because they use their cell phone a lot.

 

Right now I'm watching the dials spin equally slowly on FF and IE.

 

(However, knowing the clear-cache-and-cookies trick in FF is useful. I have to do it about once a month to unblock Gmail.)

Link to comment

Finally...after a good 2 hours...the site works for a moment

 

Server Error in '/' Application.

________________________________________

Server Too Busy

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

 

Exception Details: System.Web.HttpException: Server Too Busy

 

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

 

Stack Trace:

 

[HttpException (0x80004005): Server Too Busy]

System.Web.HttpRuntime.RejectRequestInternal(HttpWorkerRequest wr) +146

 

________________________________________

Version Information: Microsoft .NET Framework Version:1.1.4322.2300; ASP.NET Version:1.1.4322.2300

Link to comment

The Geo-chain is only as strong as the weakest link...and GC.com is that link.

 

I have this nice, new computer with more bells and whistles than you can poke a stick at. I have a 5 meg cable service. and here I sit....looking at "Server Overload."

 

For the money made off of advertising and premium membership fees, it is unacceptable for this service to be slow.

And don't begin to tell me that I should just be patient. I only did *ONE* cache today...and it takes me 20 minutes to get it logged in.

 

If I go eat at a resteraunt, and the steak is cooked improper, it is replaced, and/or my bill is reduced.

 

If I pay for newspaper delivery, and I miss a paper, not only do I get refunded or an extra paper....the folks responsible actually say "We're sorry."

 

All we get from TPTB is...

 

well, I can't show you because I can't get the blasted site to load up...yet....

 

It's not Christmas anymore...

Link to comment
Guest
This topic is now closed to further replies.
×
×
  • Create New...