MnGCA Home MnGCA
Minnesota Geocaching Association
 
 FAQFAQ   SearchSearch   User listUser list   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Pocket Query Question
Goto page Previous  1, 2, 3, 4  Next
 
Post new topic   Reply to topic    MnGCA Forum Index -> Tech Talk
View previous topic :: View next topic  
Author Message
pfalstad
Geocacher


Joined: 02 Feb 2006

Posts: 1013

PostPosted: Wed May 09, 2007 11:07 am    Post subject: Reply with quote

sir_zman wrote:
Groundspeak's reply?

http://forums.groundspeak.com/GC/index.php?showtopic=162834&st=0&gopid=2855538&#entry2855538


I would have preferred a simple, "good idea" or "we'll look into that", then ignoring my idea. I guess if they don't want a smoothly running system, and timely PQs then they can do what ever they want...but one would think that they might be a bit nicer to the paying members and consider their ideas.

I don't think StarBrand speaks for groundspeak.

Personally, I want to get the latest logs. I just wish that you could select a shorter period of time than 7 days, so that the number of updated caches is less than 500.
Back to top
View user's profile Send private message Visit poster's website
GeoPierce
Geocacher


Joined: 16 Nov 2005

Posts: 1657

PostPosted: Wed May 09, 2007 11:10 am    Post subject: Reply with quote

sir_zman wrote:
Groundspeak's reply?

http://forums.groundspeak.com/GC/index.php?showtopic=162834&st=0&gopid=2855538&#entry2855538

I would have preferred a simple, "good idea" or "we'll look into that", then ignoring my idea. I guess if they don't want a smoothly running system, and timely PQs then they can do what ever they want...but one would think that they might be a bit nicer to the paying members and consider their ideas.

"No, you can't always get what you want
You can't always get what you want
You can't always get what you want
And if you try sometime you find
You get what you need"
_________________
Tupperware doesn't belong in the kitchen!
Back to top
View user's profile Send private message Visit poster's website
GeoPierce
Geocacher


Joined: 16 Nov 2005

Posts: 1657

PostPosted: Wed May 09, 2007 11:11 am    Post subject: Reply with quote

pfalstad wrote:
Personally, I want to get the latest logs. I just wish that you could select a shorter period of time than 7 days, so that the number of updated caches is less than 500.

You could filter by date placed but that's what we are currently doing so it puts us right back where we started.
_________________
Tupperware doesn't belong in the kitchen!
Back to top
View user's profile Send private message Visit poster's website
Pear Head
Past MnGCA President


Joined: 04 Apr 2004

Posts: 5708

PostPosted: Wed May 09, 2007 11:29 am    Post subject: Reply with quote

By selecting the last 7 days we would at least get smaller queries returned compared with what we get now. No sense in pulling what hasn't changed...

I also think if you pull with the last 7 days option that you could probably combine some of the queries, especially the ones for the older caches as they are no longer hit as frequently.
_________________
Hmm...
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger
sir_zman
Past MnGCA Board


Joined: 30 Jun 2005

Posts: 1766

PostPosted: Wed May 09, 2007 11:35 am    Post subject: Reply with quote

In the end, I can keep doing what I'm doing, I just thought that this was a simple and easy way to A) make what many users are doing easier. B) save strain on the GC servers, C) Make PQs faster to run.

Oh well.
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger MSN Messenger
Pear Head
Past MnGCA President


Joined: 04 Apr 2004

Posts: 5708

PostPosted: Wed May 09, 2007 11:42 am    Post subject: Reply with quote

sir_zman wrote:
In the end, I can keep doing what I'm doing, I just thought that this was a simple and easy way to A) make what many users are doing easier. B) save strain on the GC servers, C) Make PQs faster to run.


I'm guessing you don't know how much of a strain (or just the opposite) this may cause. I'm guessing that each additional 'filter' you add to the query ADDs processing time. It may reduce the size of the file when you're done, but it may be cheaper in processing time to apply less filters and send you more queries. gc.com's problem doesn't appear to be bandwidth, it appears to be processing time on the database server(s).

It's just a guess - I'm not a MySQL expert.
_________________
Hmm...
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger
GeoPierce
Geocacher


Joined: 16 Nov 2005

Posts: 1657

PostPosted: Wed May 09, 2007 11:44 am    Post subject: Reply with quote

Pear Head wrote:
By selecting the last 7 days we would at least get smaller queries returned compared with what we get now. No sense in pulling what hasn't changed...

I also think if you pull with the last 7 days option that you could probably combine some of the queries, especially the ones for the older caches as they are no longer hit as frequently.

True, I didn't realize how big of a difference it would make but by using the updated option, I can change the date range for my first query.

without updated checked: 01/01/2000 - 10/20/2003: 496 caches
with updated checked: 01/01/2000 - 08/05/2005: 497 caches

That is a huge difference and something I will definitely try.
_________________
Tupperware doesn't belong in the kitchen!
Back to top
View user's profile Send private message Visit poster's website
Pear Head
Past MnGCA President


Joined: 04 Apr 2004

Posts: 5708

PostPosted: Wed May 09, 2007 11:50 am    Post subject: Reply with quote

GeoPierce wrote:
Pear Head wrote:
By selecting the last 7 days we would at least get smaller queries returned compared with what we get now. No sense in pulling what hasn't changed...

I also think if you pull with the last 7 days option that you could probably combine some of the queries, especially the ones for the older caches as they are no longer hit as frequently.

True, I didn't realize how big of a difference it would make but by using the updated option, I can change the date range for my first query.

without updated checked: 01/01/2000 - 10/20/2003: 496 caches
with updated checked: 01/01/2000 - 08/05/2005: 497 caches

That is a huge difference and something I will definitely try.


I would just be cautious not to trim it too close. One week might be a slow week (and thus you'll get fewer caches), but the next week may not be. I'm thinking I'll try reorganizing my queries this way but only set them up so the dates give me about 400 per query so I have room for expansion.
_________________
Hmm...
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger
GeoPierce
Geocacher


Joined: 16 Nov 2005

Posts: 1657

PostPosted: Wed May 09, 2007 12:04 pm    Post subject: Reply with quote

Pear Head wrote:
I would just be cautious not to trim it too close. One week might be a slow week (and thus you'll get fewer caches), but the next week may not be. I'm thinking I'll try reorganizing my queries this way but only set them up so the dates give me about 400 per query so I have room for expansion.

Good point. I knew there was a caveat to this. I was able to trim my pocket queries down from from 11 to 5. I'll go back and adjust the dates.
_________________
Tupperware doesn't belong in the kitchen!
Back to top
View user's profile Send private message Visit poster's website
sir_zman
Past MnGCA Board


Joined: 30 Jun 2005

Posts: 1766

PostPosted: Wed May 09, 2007 12:05 pm    Post subject: Reply with quote

Pear Head wrote:
sir_zman wrote:
In the end, I can keep doing what I'm doing, I just thought that this was a simple and easy way to A) make what many users are doing easier. B) save strain on the GC servers, C) Make PQs faster to run.


I'm guessing you don't know how much of a strain (or just the opposite) this may cause. I'm guessing that each additional 'filter' you add to the query ADDs processing time. It may reduce the size of the file when you're done, but it may be cheaper in processing time to apply less filters and send you more queries. gc.com's problem doesn't appear to be bandwidth, it appears to be processing time on the database server(s).

It's just a guess - I'm not a MySQL expert.


I'm a DBA for a fairly hefty database that requires 98% up-time I understand what you are saying, but I disagree with it, if they create an index on their update_date, or status_date column and filter only on that one column, the response time would be extremely fast. It would reduce overhead. Granted they also need to filter on distance as well, but still because you are reducing the number of returned results, this would be very fast.
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger MSN Messenger
Pear Head
Past MnGCA President


Joined: 04 Apr 2004

Posts: 5708

PostPosted: Wed May 09, 2007 12:11 pm    Post subject: Reply with quote

sir_zman wrote:
I'm a DBA for a fairly hefty database that requires 98% up-time I understand what you are saying, but I disagree with it, if they create an index on their update_date, or status_date column and filter only on that one column, the response time would be extremely fast. It would reduce overhead. Granted they also need to filter on distance as well, but still because you are reducing the number of returned results, this would be very fast.


Then I stand corrected - thanks.

The other thing I recall reading recently is that they were hiring outside help to assist them with correcting db problems.. I wonder if that is still going on, and if we will see new features/optimizations when that happens?

I doubt it, but one can wonder, right? Smile
_________________
Hmm...
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger
sir_zman
Past MnGCA Board


Joined: 30 Jun 2005

Posts: 1766

PostPosted: Wed May 09, 2007 12:33 pm    Post subject: Reply with quote

maybe I can work for GC/GS? hmm, that could be fun!
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger MSN Messenger
GeoPierce
Geocacher


Joined: 16 Nov 2005

Posts: 1657

PostPosted: Wed May 09, 2007 12:39 pm    Post subject: Reply with quote

sir_zman wrote:
maybe I can work for GC/GS? hmm, that could be fun!

I bet they have some great fringe benefits! Do they get paid to go geocaching? Maybe Friday is geocaching day? Smile
_________________
Tupperware doesn't belong in the kitchen!
Back to top
View user's profile Send private message Visit poster's website
RubberToes
Geocacher


Joined: 13 Feb 2005

Posts: 222

PostPosted: Wed May 09, 2007 3:28 pm    Post subject: Reply with quote

OK, you've had an interesting discussion while I've been off working. So, some of you are going to use "updated" queries instead of pulling all the caches.

After you do this, how are you going to tell which caches have been archived? Archived caches don't show up in your PQ, and now caches that have not been updated won't show up in your PQ.

You're going to have a database full of archived caches that don't look any different from caches that have not been updated. How will you know which is which?
Back to top
View user's profile Send private message
GeoPierce
Geocacher


Joined: 16 Nov 2005

Posts: 1657

PostPosted: Wed May 09, 2007 3:32 pm    Post subject: Reply with quote

RubberToes wrote:
OK, you've had an interesting discussion while I've been off working. So, some of you are going to use "updated" queries instead of pulling all the caches.

After you do this, how are you going to tell which caches have been archived? Archived caches don't show up in your PQ, and now caches that have not been updated won't show up in your PQ.

You're going to have a database full of archived caches that don't look any different from caches that have not been updated. How will you know which is which?

It doesn't matter for me because I'm not using GSAK. I wrote a program to load my queries into a MySQL database and mark caches as archived if they haven't been updated in more than 7 days. But if the cache is unarchived in the future, it will be updated to active in the database.
_________________
Tupperware doesn't belong in the kitchen!
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    MnGCA Forum Index -> Tech Talk All times are GMT - 6 Hours
Goto page Previous  1, 2, 3, 4  Next
Page 2 of 4

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2002 phpBB Group
Geocaching Cache Icons, Copyright 2009, Groundspeak Inc. All rights reserved. Used with Permission.