Jump to content

Server admin 101


power69

Recommended Posts

Why oh why oh why do updates have to happen on production servers. why is nothing tested before going live to ensure stuff actually works.

Why is everything (including both primary & secondary DNS servers) hosted in a single datacenter with no failover/offsite emergency server?

Edited by dakboy
Link to comment

why is nothing tested before going live to ensure stuff actually works.

If you're referring to the slowness of the maps, then it would be extremely difficult to test before going live, since it's most likely due to the large volume of hits.

There are quite a few tools on the market for performing load & performance testing of web apps. You can simulate very large workloads with them.

Link to comment

why is nothing tested before going live to ensure stuff actually works.

If you're referring to the slowness of the maps, then it would be extremely difficult to test before going live, since it's most likely due to the large volume of hits.

There are quite a few tools on the market for performing load & performance testing of web apps. You can simulate very large workloads with them.

Good to know. Thanks for clarifying!

Link to comment

There are quite a few tools on the market for performing load & performance testing of web apps. You can simulate very large workloads with them.

 

Even if they ran these tools, they still wouldn't be an accurate test as they would be simulating a very large workload only from Groundspeak's IP address(es). This might trigger throttling at the OSM side which would skew the test results.

 

Sadly the only true test was to put the thing in production and watch for smoke.

 

I wonder if they could get the Google maps back if they offered to put more stupid Google ads on the site? Or maybe embed the ad in the map?

Link to comment

Testing only tests so far. The real test comes from teh end users - the annoying ones who only know how to do something one way. And its that one way that you would never think to test for.

 

We always test everything before giving to the end user, but there will always be outages, and there will always be bugs after release. Until they can simulate people it ain't gonna happen.

Link to comment

There are quite a few tools on the market for performing load & performance testing of web apps. You can simulate very large workloads with them.

I doubt if any load test could simulate the hammering that all of us cachers have been doing to the new maps since the update. What we're most likely seeing now is a worst-case scenario where every cacher and his brother is pounding on the site to check out what all the fuss is about.

 

I plan to wait until the dust has settled before making any judgements.

 

--Larry

Link to comment

Testing only tests so far. The real test comes from teh end users - the annoying ones who only know how to do something one way. And its that one way that you would never think to test for.

 

We always test everything before giving to the end user, but there will always be outages, and there will always be bugs after release. Until they can simulate people it ain't gonna happen.

 

Perfect timing for what I'm dealing with elsewhere. :D

Link to comment

Testing only tests so far. The real test comes from teh end users - the annoying ones who only know how to do something one way. And its that one way that you would never think to test for.

 

We always test everything before giving to the end user, but there will always be outages, and there will always be bugs after release. Until they can simulate people it ain't gonna happen.

 

Perfect timing for what I'm dealing with elsewhere. :D

 

Yep, and that's why you have "backout plans". :rolleyes:

Link to comment

Testing only tests so far. The real test comes from teh end users - the annoying ones who only know how to do something one way. And its that one way that you would never think to test for.

 

We always test everything before giving to the end user, but there will always be outages, and there will always be bugs after release. Until they can simulate people it ain't gonna happen.

 

Perfect timing for what I'm dealing with elsewhere. :D

 

Yep, and that's why you have "backout plans". :rolleyes:

 

All I can think of right now is to smack a few people. <_<

 

Would that be bad? :rolleyes:

 

*sigh* I'm trying to learn patience and diplomacy, at the expense of my sanity.

 

Bring on the simulated humans, please.

 

 

B.

Edited by Pup Patrol
Link to comment

There are quite a few tools on the market for performing load & performance testing of web apps. You can simulate very large workloads with them.

I doubt if any load test could simulate the hammering that all of us cachers have been doing to the new maps since the update. What we're most likely seeing now is a worst-case scenario where every cacher and his brother is pounding on the site to check out what all the fuss is about.

--Larry

 

Load testing is just one kind of test, and mostly only tests for performance though it could also expose "configuration" issues. For example, setting the number of database connection an application might use too low might not be detected until everyone and their brother started making simultaneous requests. However, many of the problems that arise after an update are functional and in some cases, when we're "surprised" with a new feature (or a feature taken away) the first time any of us sees the problem is when it's in full production. Even on fairly small projets, I have a test machine that I have set up where I deploy any changes I make, then have others look at how it's behaving before I deploy the code to a production machine. I imagine Groundspeak could get quite a few volunteer beta testers if they'd only set up a "beta.geocaching.com" (available only to authorized users) site which a bunch of people could pound on (I'd even be willing to sign a non-disclosure agreement) for a few days to a week before a release is deployed to full production. Perhaps even setting up a restricted forum section available to beta testsr to promote the discussion about impeding features (rather than speculating about what we might see) might prevent some of the train wrecks we've seen.

Link to comment

Not to add to the whining...

 

Is there any way we as individual users can ovewrlay the Groundspeak data onto a google earth map real time without violating TOU from GS? My understanding is that the payments now required from GS by Google were being triggered by the volume of requests being funneled through GS. If we as individuals could instead request the Google data, perhaps we could have the best of both worlds?

Link to comment

There are quite a few tools on the market for performing load & performance testing of web apps. You can simulate very large workloads with them.

I doubt if any load test could simulate the hammering that all of us cachers have been doing to the new maps since the update. What we're most likely seeing now is a worst-case scenario where every cacher and his brother is pounding on the site to check out what all the fuss is about.

--Larry

 

Load testing is just one kind of test, and mostly only tests for performance though it could also expose "configuration" issues. For example, setting the number of database connection an application might use too low might not be detected until everyone and their brother started making simultaneous requests. However, many of the problems that arise after an update are functional and in some cases, when we're "surprised" with a new feature (or a feature taken away) the first time any of us sees the problem is when it's in full production. Even on fairly small projets, I have a test machine that I have set up where I deploy any changes I make, then have others look at how it's behaving before I deploy the code to a production machine. I imagine Groundspeak could get quite a few volunteer beta testers if they'd only set up a "beta.geocaching.com" (available only to authorized users) site which a bunch of people could pound on (I'd even be willing to sign a non-disclosure agreement) for a few days to a week before a release is deployed to full production. Perhaps even setting up a restricted forum section available to beta testsr to promote the discussion about impeding features (rather than speculating about what we might see) might prevent some of the train wrecks we've seen.

 

I thik that feature is only available to the platinum members

Link to comment

As a member of the ANDROID app testing team, I can tell you there ARE staging servers. Occasionally, updates to the app are made and a dev version is released to us that only acts on the staging servers. Anything we do using that version is not reflected on the actual site.

 

Whether or not all site updates are tested the same way, and whether their (TPTB) 'considered acceptable' results are the same as everyone else's (us users) is another question.

Link to comment

Are you refering to anything specific?

Just the typical something is always broke after an update because it wasn't tested on a non production server.

Well, if aren't talking in specifics then...

A lot of website are java based. Java run on the users system. When you are dealing with PC there is an almost impossible number of equipment combinations, software combination, configurations, etc. There are so many different combinations that some tracking websites can identify individuals by polling your system for what software you have installed and what version you are using. It is not surprisingly highly accurate.

 

So what's my point? With all those possible combinations it is near impossible to test very possible way that every computer can be configured. Some thing just have to be hashed out on a live server with live users. Microsoft arguably has the most resources for testing and yet they still release products that have bugs. It isn't because they didn't test it. It is because no one can reasonably test every possible situation in every possible environment.

Link to comment
A lot of website are java based. Java run on the users system.

Correction: If a website is Java based, the Java runs on the server. Java on the user's system is quite rare nowadays. JavaScript frequently runs on the user's system, but it's completely different than Java. A pox on whoever decided to name one after the other.

 

Besides, I think Groundspeak runs a Microsoft stack, which would mean something other than Java.

Link to comment
A lot of website are java based. Java run on the users system.

Correction: If a website is Java based, the Java runs on the server. Java on the user's system is quite rare nowadays. JavaScript frequently runs on the user's system, but it's completely different than Java. A pox on whoever decided to name one after the other.

 

Besides, I think Groundspeak runs a Microsoft stack, which would mean something other than Java.

Good catch. I did indeed mean javascript. I'm not familiar with Microsoft stack. Whatever the case we can only expect GS to fully test the client side. There is no reasonable way for them to test every possible variation of the client side.

Link to comment

Why oh why oh why do updates have to happen on production servers. why is nothing tested before going live to ensure stuff actually works.

How do you know they aren't?

Two incidents, a power failure, and a fire, prove they aren't.

You're confusing off site redundancy with testing. Groundspeak is a small company, and datacenters are not cheap.

Link to comment

Why oh why oh why do updates have to happen on production servers. why is nothing tested before going live to ensure stuff actually works.

How do you know they aren't?

Two incidents, a power failure, and a fire, prove they aren't.

You're confusing off site redundancy with testing. Groundspeak is a small company, and datacenters are not cheap.

An offsite server on shared hosting where a simple "we're down, here's why" page can be posted, with secondary DNS (or even all your DNS) can be had for less than $100 per year. When the power failure & fire happened, people were in the dark for 24+ hours because there was no central way to get the word out.

Edited by dakboy
Link to comment

An offsite server on shared hosting where a simple "we're down, here's why" page can be posted, with secondary DNS (or even all your DNS) can be had for less than $100 per year. When the power failure & fire happened, people were in the dark for 24+ hours because there was no central way to get the word out.

 

Redirection can cause issues especially at a DNS level with ISPs.

Edited by bladesedge
Link to comment

Not to add to the whining...

 

Is there any way we as individual users can ovewrlay the Groundspeak data onto a google earth map real time without violating TOU from GS? My understanding is that the payments now required from GS by Google were being triggered by the volume of requests being funneled through GS. If we as individuals could instead request the Google data, perhaps we could have the best of both worlds?

 

Over in the Release Notes thread are links to a couple of different Greasemonkey scripts that will add Google maps back in. It is said that they apparently do not use the API, so Groundspeak won't get charged, but if you overuse them, you yourself might be blocked for a day.

Link to comment
A lot of website are java based. Java run on the users system.

Correction: If a website is Java based, the Java runs on the server. Java on the user's system is quite rare nowadays. JavaScript frequently runs on the user's system, but it's completely different than Java. A pox on whoever decided to name one after the other.

 

Besides, I think Groundspeak runs a Microsoft stack, which would mean something other than Java.

Good catch. I did indeed mean javascript. I'm not familiar with Microsoft stack. Whatever the case we can only expect GS to fully test the client side. There is no reasonable way for them to test every possible variation of the client side.

 

If you look at many of the pages on the site you'll see pages which end with a .aspx extension. The asp stands for Active Server Pages, a Microsoft product for creating dynamic and interactive web sites.

 

As a software application developer I'm quite aware that client environments can be quite varied, both from those that are still running very old hardware and software to those that insist on running bleeding edge technologies and insist that my application should work with something that may be months from a production release. In the case of javascript there are a lot of often used libraries which create an abstraction of the functionality a site might need so the developer doesn't need worry too much if it'll run on a Mac or a PC or a Linux machine. I've got a PC, a Mac, and a Linux machine in my office and often test things I develop on all of them. If someone would be willing to give me an iPad and an ASUS Eee Pad transformer I'd be happy to test my apps on those platforms as well.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...