cache rating - request for comments [closed]

Discussion about the Geocaching Australia web site
Post Reply
User avatar
ideology
Posts: 2763
Joined: 28 March 03 4:01 pm
Location: Sydney
Contact:

cache rating - request for comments [closed]

Post by ideology » 09 September 04 12:42 pm

we are thinking of implementing a cache rating system
the purpose is to highlight those "don't miss!" caches

we are aware of the keen people rating system (thanks, ecodave) but it only rates whether or not a person enjoyed it. we'd like something a little more specific than that. for example, some people enjoy puzzle caches, others don't.

rating caches
we think it's a good idea because it means that on a reasonably nice day (like this) we might do a quick scan of nearby caches, see that one is rated really highly, and go for it. at the moment, it's a bit hit or miss. your thoughts?

anonymity
we'd prefer that it's public because it will be trivial to work out who voted. even if we don't display ratings until there is a quorum (eg 3 finders), if the result is in one direction, then it's obvious that they have all voted that way.

rating dimensions
the nsw cache awards (thanks again, ecodave) had categories of best traditional, multi, puzzle, location, waypoint and innovation. what does "best" mean? everyone has different opinions. should we try to deconstruct "best"? we think that it would take too long to fill in the rating if that was the case.

keeping it positive
we'd like to keep this positive. perhaps a two-ended scale might help, eg not measure something in terms of "innovation" as if more is better, but measure it on a scale of "traditional <-> innovative" so that a cache rated as traditional doesn't look un-innovative (if that makes sense)

proposal
the best we can come up with so far is three "wow-factors":
- eyes / location "wow" factor - irrespective of urban or country. is it a place that you were glad to visit irrespective of the cache?
- brain / hiding technique / puzzle "wow" factor - was there some cleverness involved in hiding the cache? could be physical or mental
- cache "wow" factor - was it a cool container, filled with great stuff, etc
the problem with these is the positiveness. maybe the lowest is "good" or something?

your thoughts?
Last edited by ideology on 02 July 06 10:19 am, edited 1 time in total.

User avatar
Cached
2500 or more caches found
2500 or more caches found
Posts: 3087
Joined: 24 March 04 4:32 pm
Location: Launceston, Tasmania
Contact:

Post by Cached » 09 September 04 1:14 pm

Filled with good stuff is probably not an ideal thing to be rated against - it could be completely different next week.

The idea sounds brilliant - would help me with my caching.

User avatar
CraigRat
850 or more found!!!
850 or more found!!!
Posts: 7015
Joined: 23 August 04 3:17 pm
Twitter: CraigRat
Facebook: http://facebook.com/CraigRat
Location: Launceston, TAS
Contact:

Post by CraigRat » 09 September 04 4:26 pm

Ideology wrote:proposal
the best we can come up with so far is three "wow-factors":
- eyes / location "wow" factor - irrespective of urban or country. is it a place that you were glad to visit irrespective of the cache?
- brain / hiding technique / puzzle "wow" factor - was there some cleverness involved in hiding the cache? could be physical or mental
- cache "wow" factor - was it a cool container, filled with great stuff, etc
the problem with these is the positiveness. maybe the lowest is "good" or something?
Thinking about the subjectivness of it, you could have it so that after (say 5) people had visited the cache, if >50% of people rated it a wow factor then you could show an icon (not a score) for that factor(location/technique/whatever)...maybe if it had a >95% or more you could put a "Top cache" icon on it or something....

Dunno... Something about scoring doesn't grab me.... but at the same time I'd like to know if it had great views etc.....

/me will stop rambling incoherently now..........

User avatar
Snuva
1500 or more caches found
1500 or more caches found
Posts: 1081
Joined: 05 July 04 5:38 pm
Twitter: Snuva
Location: Hobart
Contact:

Post by Snuva » 09 September 04 4:50 pm

Something about scoring doesn't grab me.... but at the same time I'd like to know if it had great views etc.....
<P>I agree on both points - a score would be difficult because there are so many reasons that different caches appeal. Yet it would be nice to know what other cachers thought. I read through logs, and people usually indicate if it was really clever or had fantastic views, but if a scoring could be worked out it would provide a nice snapshot. So something along the lines of the suggested 'wows' sounds good to me, although perhaps we should make certain they are clearly defined. The lines could get a bit blurred between cool hiding place and cool container, no?

Dingbats
100 or more tracks walked
100 or more tracks walked
Posts: 118
Joined: 30 March 04 3:41 pm
Location: Wollongong
Contact:

Post by Dingbats » 09 September 04 5:23 pm

I really want to see an improved cache rating system and I think you are on the right track.<p>

You've mentioned how it will be useful to seekers. I think it would also be a much needed feedback mechanism for cache owners. I think most of us generally bite our tongues when we write a log on a boring (or worse) cache. A better rating system may encourage honesty and it will be clear to all if the majority think that a particular cache is crap. Hopefully it will encourage all of us as cache owners to work harder to achieve better satisfaction ratings.<p>

We know that cache containers change with time and the quality of the contents tends to degrade quickly. Perhaps the container / contents wow factor could be dropped in favour of splitting the hiding wow factor from the mental wow factor? I imagine that most cachers gain a different satisfaction from solving a tricky puzzle to finding a well hidden cache. My guess is many prefer one over the other :)<p>

Perhaps it could include a physical challenge wow factor. With this, regardless of whether the cacher found it enjoyable or not, they could rate how physically difficult it was for them. Some find this rewarding, others don't.<p>

I think an overall satisfaction rating might be useful. This might encourage cachers to score the mental, well hidden, and physical aspects of the cache more objectively because they can express through this overall satisfaction rating that they didn't enjoy it because it was in their opinion too well hidden or too physically challenging. Of course it also gives the ability for cachers to rate a high level of satisfaction for urban drive-bys because these are the caches they enjoy doing.<p>

It would be nice to try and avoid this kind of problem:<p>

(thought process)The views were the best I have every seen, a 360 degree vista of rugged snow capped peaks breaking through the clouds. It did however take me 40 days to climb this hill, I had to fight frost-bite and altitude sickness and the support crew cost me $80K. It really wasn't worth it, I'd be just as happy catching the lift up centre point tower. (/thought process)<p>

location wow rating: 1/5<p>

alex
1550 or more caches found
1550 or more caches found
Posts: 259
Joined: 11 April 03 12:34 am
Location: Adelaide

Post by alex » 12 September 04 11:19 pm

Maybe I'm in a bad mood today. Maybe the new layout has put me there.

I think this idea is wrong, wrong, wrong. What a way to discourage new players from placing their first cache. What a way to limit innovation and tie us into the normality of formally sanctioned "good" caches. What a way to add to the disturbing trend of mainstream conformity that is increasingly pervading our game and this forum.

If you are a true cacher then you will appreciate the challenge of the unknown journey, not a predictable result!

Some ramblings from a concerned old hand. Comments welcomrd.

User avatar
The Ginger Loon
450 or more roots tripped over
450 or more roots tripped over
Posts: 824
Joined: 28 March 03 9:09 pm
Location: Tamworth
Contact:

Post by The Ginger Loon » 13 September 04 12:48 am

Actually, I'm with Alex...

I don't like the idea of a ratings system either. "One mans' meat is another mans' chewy" as they say and I can see that putting an "official" rating on a cache could quite easily get people offside and generate a whole lot of bad feeling between players. We already have some players and cache owners here in NSW that are way too sensitive for their own good (no names) but this game is open to all regardless of their level of ingenuity or their geographic location for that matter or their ability to take criticism.

As someone who has been in the game for a couple of years now I've seen caches of all sorts and varieties come across my screen, from the brilliant and outstanding in thought, complexity and execution to the downright pedestrian and boring, but I feel we need to have caches of all types and varieties to keep the game interesting.

Part of the intrigue of geocaching for me is that I never know what GZ will be like as a location until I get there. You roll the dice and you take your chances. Occasionally you get a location that is not particularly inspiring but thems the breaks.

The other thing that I've noticed is that peoples' caches generally get better as they go along and gain more experience in the game. If a new player gets a bad rating and leaves the game, we run the risk of having abandoned caches.

Like Alex just a few thoughts from a concerned old hand...

swampgecko
It's all in how you get there....
It's all in how you get there....
Posts: 2185
Joined: 28 March 03 6:00 pm

Post by swampgecko » 13 September 04 7:26 am

The Ginger Loon wrote:Actually, I'm with Alex...

I don't like the idea of a ratings system either. "One mans' meat is another mans' chewy" as they say and I can see that putting an "official" rating on a cache could quite easily get people offside and generate a whole lot of bad feeling between players. We already have some players and cache owners here in NSW that are way too sensitive for their own good (no names) but this game is open to all regardless of their level of ingenuity or their geographic location for that matter or their ability to take criticism.

As someone who has been in the game for a couple of years now I've seen caches of all sorts and varieties come across my screen, from the brilliant and outstanding in thought, complexity and execution to the downright pedestrian and boring, but I feel we need to have caches of all types and varieties to keep the game interesting.

Part of the intrigue of geocaching for me is that I never know what GZ will be like as a location until I get there. You roll the dice and you take your chances. Occasionally you get a location that is not particularly inspiring but thems the breaks.

The other thing that I've noticed is that peoples' caches generally get better as they go along and gain more experience in the game. If a new player gets a bad rating and leaves the game, we run the risk of having abandoned caches.

Like Alex just a few thoughts from a concerned old hand...
Ditto the Loon and Alex. I can't put it any better myself Loon

User avatar
caughtatwork
Posts: 17015
Joined: 17 May 04 12:11 pm
Location: Melbourne
Contact:

Post by caughtatwork » 13 September 04 9:00 am

Also consider the frustration factor for someone who spent an hour seeking a cache hunting through banks of grass. Is it likely to taint their opinion of the true rating factors 'cos they took too long and at the end were just simply pissed off?

We're pretty much dealing with opinions and emotions after the cache has been found and a cache hidden well (but frustrating to find) might sway the overall result towards the negative.

On a personal note, I go for any cache that is doable (I avoid trips of hundres on kilometers on multis for personal reasons). I don't think I would attempt a cache based on a rating, I think I would attempt the cache simple because it's there.

Take into consideration that some people like to walk / hike and others don't, so a long multi on foot would rate high on some peoples meters and low on others.

I wonder if the proposed rating system is too subjective. I'll think about methods of ratings that are less subjective.

User avatar
Team Piggy
Posts: 1601
Joined: 02 April 03 5:16 pm
Location: South Australia

Post by Team Piggy » 13 September 04 9:28 am

Me follwing Alex too. With this new rating system, I would be rating a hell of a lot of the caches I found as rubbish bin fodder...Not worth the plastic they are in..

The Rats
5000 or more caches found
5000 or more caches found
Posts: 436
Joined: 08 April 03 12:44 pm
Location: Wollongong NSW
Contact:

Post by The Rats » 13 September 04 9:36 am

How about we get a handle on rating the "Terrain and Difficulty" of our caches before we go rating the success of the caches.<P>Time and time again I see one star rated caches where theres no hope in hell of getting wheelchair anywhere near GZ. :evil: Maybe cachers should look at this rating system when they report a new cache........ Oh! I feel better now that I've got that off my chest 8)

User avatar
Papa Bear_Left
800 or more hollow logs searched
800 or more hollow logs searched
Posts: 2573
Joined: 03 April 03 12:28 am
Location: Kalamunda, WA
Contact:

Post by Papa Bear_Left » 13 September 04 9:40 am

The only time I'd like to see a 'user satisfaction rating' is when I'm travelling to a distant cache-rich area and trying to decide where to spend my limited time.

That's not much of the time, though, and I do think that the pernicious effects of a rating system probably outweigh the benefits.

Now, if you could only see the ratings if you could prove that were an out-of-towner.... naar....

User avatar
ideology
Posts: 2763
Joined: 28 March 03 4:01 pm
Location: Sydney
Contact:

Post by ideology » 13 September 04 9:46 am

alex, thanks, you make good points
  • risk of discouraging new players: you're right. we'd thought about the risk of discouraging people, but not necessarily new players. they would be particularly sensitive to comments about their cache.
  • limiting innovation: we want to increase innovation too. we were hoping that this would increase innovation as Dingbats mentioned
  • disturbing tread of maintream conformity in this game: we don't want disturbing conformity in the game either. that's one of the things we don't like about the secondary site! banning moveable caches etc really stifled the game for us. we were hoping that some kind of rating system would break the conformity because if there were a string of conforming caches, the wow level would decrease as people got sick of it and people would start developing new ideas.
  • disturbing trend of mainstream conformity in this forum: we're sorry you feel that way. pyrenees kid started it on the ezboard system, so he can tell you the intentions for starting it. for us, we liked it being away from the conformity of the USA. we offered to host it here so it didn't have to have advertisements or ask people for money to host it advertisement-free. we've never deleted or censored any post in this forum (apart from the occasional one where someone posted an error message which posed a minor security risk.) having said that, we don't run the forums - pyrenees kid runs them with state moderators, so we'll pass the buck to them, ha!
  • unknown journey / predictable result: yeah, maybe this is the central point. we like the unknown journey, but i guess we are after some kind of predictable result, at least insofar as "gee, i had a good time today!" we don't get out caching as much as we used to, so the times that we do, we want to enjoy it!
  • bad mood / new layout: sorry that the new layout could have put you in a bad mood. what don't you like about it? is it the clean/clinical layout or the information presented, eg leaderboards?
loon, thanks. swampy, excellent points!
  • official: hmm... yeah... maybe we should drop the "official" tagline from the website - we only put it up there to highlight that the secondary site is no more official than any other site
  • [cachers] way too sensitive: yes, we have been surprised some of the behaviour as well.
  • we need to have caches of all types and varieties to keep the game interesting/roll your dice and take your chances: we guess the pedestrian and boring caches might keep the game interesting on a lucky-dip level (ie, oh dear, that was pedestrian and boring!") and that's fine, but we'd rather do a cache which has made people go "wow!" (and not, "wow, that was pedestrian and boring!")
  • people's caches get better as they go along: true, take a look at i!1: nest!
thanks caughtatwork, good points
  • frustration factor/subjectivity: yeah it's horses for courses. we were thinking about trying to decompse the subjectivity into objective measures, but it got pretty messy - so many variables. in real life, we usually ask someone we know that likes similar caches
  • attempting the cache simply because it's there: yes, some people do that. the situation we are in is that we are surrounded by hundreds of caches and don't have much time to get them, so we are looking for pointers to good caches, and thought other people would too
thanks team piggy, you've hit the nail on the head for us!
  • "With this new rating system, I would be rating a hell of a lot of the caches I found as rubbish bin fodder...Not worth the plastic they are in..": that's our point. we don't want to do those caches! we respect loon's love of the lucky-dip, but we don't share it.
rats, we agree - most people don't understand the difference between terrain and difficulty. we did some plots of it on the old statistics page (which is still being updated) and saw that there was a huge correlation between caches. (we should re-do that stuff - there's some good data there!)

bear_left, yes: we are in a cache-rich area with little time. it's just that it's a local cache-rich area!

we're glad we opened this up for discussion, and we hope we are debating the issues and that it will lead to a better result. we were going to start work on a rating system soon but if people think it will detract from the game, we won't do it. unfortunately, we have been looking at it primarily from a finders' standpoint, not the hiders'. from the hider's standpoint, we realise that getting an email saying that someone has found your cache and seeing that they rated it really low could upset you. it gets to the "positive" thing that we mentioned in our first post. is there another way to identify good caches but not upset the owners of other caches?

how about a system where only the positive stuff gets mentioned? eg you can identify a cache as an "outstanding," "top 10" or perhaps "favourite" cache, but that's all. the idea is that it's a term used sparingly. as a hider, you don't expect that people will constantly identify your caches as "outstanding" caches. as a finder, you don't need to worry so much about hurting other people's feelings. perhaps it could tie into a "nomination" system for the cache of the year awards (we have them in nsw, not sure about the other states.) the nice thing about this is that people can only nominate a handful of caches, so the nominations are by definition very sparse.

great discussion! thanks! coments welcome

User avatar
caughtatwork
Posts: 17015
Joined: 17 May 04 12:11 pm
Location: Melbourne
Contact:

Post by caughtatwork » 13 September 04 10:40 am

ideology wrote:how about a system where only the positive stuff gets mentioned? eg you can identify a cache as an "outstanding," "top 10" or perhaps "favourite" cache, but that's all. the idea is that it's a term used sparingly. as a hider, you don't expect that people will constantly identify your caches as "outstanding" caches. as a finder, you don't need to worry so much about hurting other people's feelings. perhaps it could tie into a "nomination" system for the cache of the year awards (we have them in nsw, not sure about the other states.) the nice thing about this is that people can only nominate a handful of caches, so the nominations are by definition very sparse.
This is probably better as you don't feel obliged to rate every cache you find.

Are you thinking of a limited number of 'ratings' that everyone has (eg. 5 per year, 2 per month, etc) or is it unlimited (ie. Rate every cache outstanding if you think it is).

Tricks then come into what happens when you find 11 outstanding caches, you already used up your 10 ratings for this year what would be the process for removing one rating from a cache and re-allocating it.

The converse would be that maybe you want to hang onto your ratings until the end of the year and allocate them that way so you don't waste a rating on a cache just in case something better comes along.

Maybe on the last day of the month it's voting day. You have to log on on the last day of the month to allocate your votes. Sure some people won't and some people will and that's another hairy one. This may also stress the bandwidth of the server with everyone logging on on a given day.

The other thing to consider (if you're going into awards) is that caches which are hidden towards the 'end of the year' may garner less votes (due to time) than caches hidden at the beginning of the year (more time to find and vote). Would we see a lack of caches in the preceeding months and a rush at the beginning of the new voting year?

I realise I'm throwing up more problems that solutions although I think this is just indicitive of the nature of voting. It's not dissimilar from the Academy Awards. Not everyone will agree and there are logisitical issues as well.

Also, do we have any idea whether people actually want to join in the ratings? If only 10% of the active cachers joined in voting (others may be ambivilent) then are we getting a true representation of votes or only from those who are dedicated to the game? Not dissimilar from the Australian Voting system. If we weren't forced to vote, would we?

If I really enjoy a cache, I leave a comment in the log. Sure, it disappears off the bottom of the page sooner or later, but at least the cache owner knows that at some point, someone appreciated the efforts that they went to. This may be more personal than allocating votes.

Another thing that may cause irritation is people 'swapping' votes. This comes down to personal intergity, but if we have people swapping on a not-equal basis, then are they just as likely to offer votes to mates rather than be true to the cause. This may not be a problem, but it's open to abuse at a global level, especially if there is a 'prize' at the end of the year. Some people will do anything just to win or be first on the leaderboard.

I'm not into competition (well, unless I'm competing against my brother and then it's personal) and I just have a feeling that this may end up being a competition that adds another level of angst to some of the participants.

Okay, I think I raved enough for now.

Please keep up the good work that you do. It's quite awesome and I simple can't work out where you get the time to do everything that is requested. I think this is one of the most customer focussed site I have ever been to.

swampgecko
It's all in how you get there....
It's all in how you get there....
Posts: 2185
Joined: 28 March 03 6:00 pm

WOW factor....

Post by swampgecko » 13 September 04 11:01 am

But isn't that why we have logs? To record what we think of the cache? I could cite numerous examples of where cachers have stated in their logs of the WOW factor that met them, or the lack of it.........

When I am gearing up for a day of caching, I normally plot what caches I want to do then read the logs, if any. To get an idea of what to expect...

I'll admit that at times I am slack and don't write exactly how I feel about a cache, by the time you do #9 or #10 or so for the day, the caches tend to be a blur.

On a personal note, who doesn't get a kick from reading the rave reviews that you have received for a cache that you thought was good but not great.. but then you get an absolute killer of a cache log that just goes on about how good your cache is....I know I do.... thats how I gauge the success of my caches.

Post Reply