cache rating - request for comments [closed]

Discussion about the Geocaching Australia web site
User avatar
Snuva
1500 or more caches found
1500 or more caches found
Posts: 1081
Joined: 05 July 04 5:38 pm
Twitter: Snuva
Location: Hobart
Contact:

Post by Snuva » 13 September 04 11:06 am

What if in our profile we can rate our finds- ie, go to a page listing all your finds, and number them 1 to whatever. But with the understanding that this is just your favourites, not a negative comment upon any that get a low or no rating. Then in the overall ratings, we could see a summary of caches that were rated and see on average how they were rated - did cachers rate them #1 or #15?<P>

This doesn't even get around someone just being in a fantastic mood or having great weather on the day they visit a cache. However the more ratings a cache got would help average this out. <P>
This also doesn't get around finding a system that can reflect the different reasons cachers like a cache. If only we could add some sort of free text when recording our finds :wink: . Maybe if you're concerned with the different reasons, you just read other cachers' logs. Not something you can download, but if everyone's looking for something different what do you do? This is what I've done when deciding what caches to attack when planning a trip to a cache rich area away from my own stomping ground.<P><GARBLED SENTANCE>If we start looking at the data this rating system might provide and link it to such things as 'Cachers who rated this cache highly also liked. . .' people would start to be able to get an idea of what caches cachers who like caches you like liked</GARBLED SENTANCE>. For example, I would rate clever hides/multis and caches that take me to stunning scenery highly. If someone else just liked fast finds and ranked them highly, I might not put much priority to visiting other cahes that come up high in the rating of a cache that's just an easy find. Again, this doesn't take into consideration the multiple reasons a cache could be rated highly, just assumes that over time there will be certain groups of caches that like certain caches. And maybe if someone is going to visit an area, it provides some sort of list of 'must do' caches.<P>Being in cache-poor area, I just appreciate any cache that is there!<P>I've obviously spent too much time ranting; SwampGecko just mentioned that logs usually contain people's wow-factor comments. Agree.

Mind Socket
Posts: 1329
Joined: 29 March 03 6:04 pm
Location: Gladesville, Sydney
Contact:

Post by Mind Socket » 13 September 04 2:26 pm

Wow, people sure like to type a lot on this. I hope I skimmed the points well enough. I think ...

... a rating system should be opt-in, only if the cache owner is willing to accept "numeric" feedback (let's face it, the log notes can be used for feedback too, ratings are just a systematic feedback facility). I for one would love to get more feedback on my caches at a detailed level. I also think that it would be good if a cache finder could add private notes to their cache logs that only previous finders (a la winners page) and/or the owner (for feedback) can see.

... all feedback should be available to the cache owner. Negative or statistically insufficient (eg <3 ratings) feedback should not be displayed publically. This allows good caches to be recognised by the public and bad ones to be reviewed by the owner. Negative feedback should always be constructive.

... the ratings should not factor into any awards structure, for the already mentioned reasons.

As for categories, I think a location rating, cache (container/hide) rating and innovation rating would suffice.

This will certainly be one of those divisive things for people, so it probably shouldn't be forced upon them. It would be good to get more feedback flowing, and a systematic approach means that the good stuff can be found recognised more easily.

Wow, this is a word heavy topic, I've just written my own essay. :P

- R

User avatar
caughtatwork
Posts: 17015
Joined: 17 May 04 12:11 pm
Location: Melbourne
Contact:

Post by caughtatwork » 13 September 04 4:07 pm

The statistical danger is that if everyone was forced (or felt obligated) to leave a rating, over time you would end up with most caches being rated a 5 (on a scale of 1-10).

Assuming that some people will like and some won't (for avrious reasons) the end result would be average. This wouldn't provide much of a distinction.

Then there are the newbies who haven't found many caches and leave votes appropriate to their current skill and experience, but in the overall scheme of things may weigh a cache up or down. Imagine if 10 newbies found a very difficult micro cache with a 5 kilometer hike and 10 waypoints. They may feel that it was 'too hard' and weight their vote down, even though the difficulty/terrain was up at 4 and 4.

Not everyone will have the same experience or understanding of difficulty/terrain and take that into account when placing their votes.

Then there are the caches that are easy to find but experienced cachers are voting low. Their own expectations expects something better, but newbies might find them challenging and interesting, or may have never seen a cache container quite like that one. Again, we end up with an average, or possibly with a lot of experienced cachers voting, the resultant score might be low (too easy, no wow factor, standard container) and this may discourage people from finding a worthwhile cache for their experience level.

I suppose what I'm getting at is that depending on who votes and how many votes a cache gets, we're likely to end up with average being the norm.

I can't think of a way to overcome this UNLESS the votes are weighted based on the voters experience. That opens an entirely different can of worms.

User avatar
caughtatwork
Posts: 17015
Joined: 17 May 04 12:11 pm
Location: Melbourne
Contact:

Post by caughtatwork » 13 September 04 4:10 pm

Just a thought to have.
What if there were two types of votes?
Experienced and Newbie.

Up to a certain number of caches and your vote gets counted towards the newbie rating. After that your votes start counting towards the experienced rating.

Then each cache would have two ratings.
One from experienced cachers and one from inexperienced.

A newbie could then look at the newbie voting and see what was good for them at their level of experience and the others could use the other voting to see if it was up to par for them.

Hmmm.

Mind Socket
Posts: 1329
Joined: 29 March 03 6:04 pm
Location: Gladesville, Sydney
Contact:

Post by Mind Socket » 13 September 04 5:04 pm

Assuming that some people will like and some won't (for avrious reasons) the end result would be average. This wouldn't provide much of a distinction.
Why assume that? There are caches that everyone likes and some that very few like (some people are impossible to annoy, believe me, I've tried ;) ). The end result would be _an_ average. If it works out that one only pleases half the finders, then there's room for improvement (entirely within the cache owner's interpretation). I expect my first cache wouldn't rate as highly as a similar cache that is hidden better, but I also think it would get a reasonable rating. That's what I want to find out.

I don't think it's fair to assume that people will feel obligated to rate a cache either. Some people don't even feel obligated to log a find. Entirely up to them.

As for newbies, again it seems unrealistic to assume they'll attempt a 5/5 and get annoyed. Difficulty and terrain are one of the first things you find out about. In general, a person might just have a bad experience and them's the breaks, but if the categories are not dependant on the finder's individual experience (eg location, innovation), then the ratings will be resistant to outliers. On that note, it may be worth removing outliers for calculating averages, given a sufficient body of data.

Ratings information is merely one indicator, and comes with the territory in a community-oriented activity. IMHO, the benefits outweigh _potential_ problems, especially compared to the current regime of self-appointed geocaching gods trying to call the shots.

"Statistical danger" ... now there's a buzzword I've gotta try out at the next meeting. ;)

- Rog
[/quote]

User avatar
caughtatwork
Posts: 17015
Joined: 17 May 04 12:11 pm
Location: Melbourne
Contact:

Post by caughtatwork » 13 September 04 7:42 pm

Your comments are very valid Mind Socket.

I suppose I've looked at a number of caches and there have been some that have comments like 'best ever cache' and 'this makes my top ten' while my opinion is 'eh'.

One score of 10 and one score of 1 would still average out at 5.5 which would make it, um, average.

I'm not a statistician (obviously from the statstical danger comment I made (what was I thinking)) and I suppose I can only go on the type of comments that I have read and the off line discussions I have with other cachers.

There are some that I love that they don't and visa verca, so taking only our two into consideration the cache would end up average. I presume that over time the volume of rankings might end up showing higher or lower than average, but I couldn't really guess at the distribution.

There are also the caches that we both love and I agree that this could end up with a higher rating. Whether that would be a distribution of 1 or 2 standard deviation from the norm I couldn't say, but I doubt you would end up with caches being at 2 or 9. I think they would end up more around the 4 to 6 range given enough ratings.

Of course, we all have our personal idiosyncrasies and I probably wouldn't rate a long trek as high as a short interesting walk, which would be again different to a drive-by.

Hikers on the other hand would rate them differently.

I'm concerned that if there is a lot of work that went into the rating system and then it turned out there was enough diversity in cachers that the end result would be average (or one or two standard deviations), negating the reason for the ratings.

I'm not trying to be negative and if there is a statistician in the house and they can suggest otherwise, it would be appreciated.

I would hate to think that a lot of work went into the ratings only to see ratings between 4 and 6 which may not provide enough divesity to draw a meaningful conclusion.

Just my thoughts.

User avatar
Facitman
1400 or more caches found
1400 or more caches found
Posts: 463
Joined: 18 June 04 3:58 pm
Location: Melbourne
Contact:

Post by Facitman » 13 September 04 8:53 pm

I think you need to focus on the purpose of the rating and make it very clear. If the purpose is to assist people in choosing caches to attempt then a simple overall 'enjoyment' rating may help.
<br><br>
example
I have time to do only 2 or 3 and there are 25 within reach, I'll pick the top 5 based on rating, read the logs and then choose the 3 that appeal to me.<br><br>

If you break it down into too many categories then people will start to over analyse the rating rather than using it to narrow down a search. As with most rating systems, having a reasonable volume of votes prior to publishing an average is a must.
.<br><br>
If I use the Internet Movie Database (imdb.com) a choose a film and only look at ratings then I'll be watching On Golden Pond (score 7.3) rather than Plan 9 from Outer Space (score 3.4), it maybe a crap film but it would be my choice out of the two. :D

User avatar
Derringer
2500 or more caches found
2500 or more caches found
Posts: 273
Joined: 02 April 03 9:48 pm
Location: Seymour Vic

Post by Derringer » 13 September 04 9:19 pm

A Numeric cache rating system is IMHO too subjective to be of any useful purpose.
As a bushwalker I would rate highly any cache with a high terrain rating.
The further the cache is from a city would also score highly.
This would be in contrast to most geocachers.
The cache logs provide an informal system of how a cache is rated.
The cache logs tells you who does and to some extent who doesn't like to find your caches.
Those that do find your caches then inform you of their opinion of the cache, sometimes by what is said and in others by what is not said.

This provides us with an adequate means of feedback.

K&M

User avatar
Papa Bear_Left
800 or more hollow logs searched
800 or more hollow logs searched
Posts: 2573
Joined: 03 April 03 12:28 am
Location: Kalamunda, WA
Contact:

Post by Papa Bear_Left » 14 September 04 8:31 am

Derringer wrote:A Numeric cache rating system is IMHO too subjective to be of any useful purpose.
...
The cache logs provide an informal system of how a cache is rated.
...
This provides us with an adequate means of feedback.
OK, say that you're going to Sydney for work and have a free Sunday and a hire car. There's several hundred caches in range, and still scores even if you select for higher terrain ratings, and a few dozen even if you plot a map and choose the ones in a large amount of green.

How do you pick which ones are more likely to reward your limited time?
Sure, you'll read the logs, but that's a LOT of logs to read! Take away your filter for bushwalking, and the choice gets even tougher.

You might miss a great new cache that doesn't have a rating yet, but hopefully you'll also miss the slog through the uninteresting swamp that you might otherwise end up wasting your time on.

swampgecko
It's all in how you get there....
It's all in how you get there....
Posts: 2185
Joined: 28 March 03 6:00 pm

Post by swampgecko » 14 September 04 9:02 am

Bear_Left wrote: hopefully you'll also miss the slog through the uninteresting swamp that you might otherwise end up wasting your time on.
and WHAT is uninteresting about a swamp? :wink:

Mind Socket
Posts: 1329
Joined: 29 March 03 6:04 pm
Location: Gladesville, Sydney
Contact:

Post by Mind Socket » 14 September 04 11:35 am

Derringer wrote:A Numeric cache rating system is IMHO too subjective to be of any useful purpose.
As a bushwalker I would rate highly any cache with a high terrain rating.
The further the cache is from a city would also score highly.
It _could_ be too subjective, but if the ratings are designed where you must specifically rate the location, the cache itself, trickery/puzzles etc, then any existing bias shouldn't apply. To say a cache 50 kms from the city is better than one 5 kms from the city before even doing it is ... odd, IMHO. It may be more appealing, but in doing both, you might discover that the city one is a better cache/location/experience/challenge.

If a cache has a high terrain rating, would you rate it highly in any category even if it was a 30m abseil into raw sewage (taking an extreme example)?

- R

Rabbitto
4000 or more? I'm officially obsessed.
4000 or more? I'm officially obsessed.
Posts: 793
Joined: 01 April 04 2:01 pm
Location: Rowville, Victoria

Post by Rabbitto » 14 September 04 3:19 pm

Just spent most of my lunch break wading through this thread and considering all the pros and cons to this discussion. How about this to try and hit the thin white line -

Nominations -
Everybody can nominate their top ten caches in each state.
Once ten caches have been noninated, a cacher may only nominate a new cache to their top ten if they first remove a previous nomination.
The nominations remain valid indefinitely.

Reports -
On the main page - a list of the caches with the most top ten votes Australia Wide.
On each state page - a list of the caches in each state with the most top ten votes.
On each cache page - a list of the cachers who are currently rating this cache in their top ten.
On each user page - a list of their personal top tens.

Why? -
Discretion - Anyone's cache not listed with a top ten vote will never know whether they are 11th or 311th.
Simplicity - One cacher - 10 votes only in each state. No comments - If you want more info, you can go and read their logs.
Common Threads - You do 1 cache that you like on someone's top ten - Chances are there's another 9 right up your alley on their list.
Accountablity - Your name is listed next to the cache. Pals vote pals? It's fine but in the open.

User avatar
Cached
2500 or more caches found
2500 or more caches found
Posts: 3087
Joined: 24 March 04 4:32 pm
Location: Launceston, Tasmania
Contact:

Post by Cached » 14 September 04 3:40 pm

I personally like Rabbito's idea better than what else has come up so far. It seems fair and hopefully workable, and nobody gets offended.

The other way would be to have monthly nominations - your nominations are zeroed on the first of the month and they start again - otherwise exactly the same as Rabbito. You might even have an "All Time" top ten list as well as the monthly ones.

A list could be maintained of previous top ten caches and they could be given a link to an appropriate graphic to include on the cache page that indicates that it was a "prevoius .com.au top ten cache" or is "SA - December 04 Cache of the Month" type logo. Then anybody using gc.com could see that this had at some time in the past been the best cache and is probably really worth your time to do.

I think this is a better system than numbers.

Should we be subscribing to cover the cost of somebody's efforts? I pay gc.com but feel I am now going to get more out of this site.

Thanks

User avatar
Snuva
1500 or more caches found
1500 or more caches found
Posts: 1081
Joined: 05 July 04 5:38 pm
Twitter: Snuva
Location: Hobart
Contact:

Post by Snuva » 14 September 04 4:32 pm

I too like Rabbito's suggestion - brings together what a lot of people were suggesting in this thread that has become so tangled!<P>
All Cached Up wrote:Should we be subscribing to cover the cost of somebody's efforts? I pay gc.com but feel I am now going to get more out of this site.
<P>
I also agree with this. I know some people have problems with gc.com asking for money, but I subscribe as I enjoy geocaching and maintaining the site doesn't come without any effort. I would be happy to pay a reasonable subscription fee for the Australian site to help defray the cost and reward effort. And I wouldn't expect to be treated differently from those who do not wish to or cannot afford to (no members-only caches!).

User avatar
caughtatwork
Posts: 17015
Joined: 17 May 04 12:11 pm
Location: Melbourne
Contact:

Post by caughtatwork » 14 September 04 4:43 pm

Sorry to be bah humbug boy, but I conceive a potential concern.

What if you are a fan of hiking and assign your top ten to all hiking caches? In order for me to determine what you are a fan of, I need to read your logs. The same may (I say may) go for someone who loves puzzle caches. I need to work out _why_ they like a particular cache before I can determine whether it would also fit my interests / abilities.

I think the premise started out as identification of the good caches in cache rich areas. I'm not sure how the application of that desire could be accomodated by a top 10.

Taking Facitmans comments about the comparison of two movies. One rated 9 (or in this case had 9 top ten votes) and one a 4 (with only 4 top ten rated votes). You still need to determine whether they are applicable to you.

Using this method you may find that no-one aplpies a top ten vote to a magic puzzle cache, but that may be where your interests lie, so you are back to the beginning. Unable to use the voting system to determine which caches to visit in a cache rich area.

I'm not trying to be negative, but I wonder if we may have moved away from the original premise of the rating system.

Post Reply