Rating Puzzle/Mystery Caches

[ForYourEyesOnly]

This kind of picks up on one of the other threads - what we love and hate about caching.

A comment was made about bad ratings. i.e. something rated far to high or far too low in difficulty (is a hate)

I have difficulty in knowing just how to rate the difficultly of puzzles. To one person the answer is sometimes just so obvious, and to another it will remain unsolved. I've had a puzzle cache rated as "so obvious that it didn't warrant the rating" and yet others say they were totally stumped by it.

When I'm solving a puzzle there is usually that "ah ha!!" moment when it all falls into place. But what if it doesn't? To me it seems that the difficulty rating has little to do with how difficult I find the puzzle.

So how do others go about rating the difficulty of puzzle caches?

[Richary]

The difficulty rating has to be a bit of a compromise between how hard you think the puzzle is, and how hard the actual find is once you have got to GZ (ignoring terrain of course). I think all you can do is make a guess on the difficulty of the puzzle, and take that into account.

To post an extreme example, you could have a really difficult cipher that will take the average cacher several evenings to solve if they can at all. And even though the final location may be a 1/1 if it was a traditional it isn't fair to list it as that as it will take a serious amount of work to get the answer.

You as the hider have to make a guess as to what to rate it. If you get a lot of comments saying it was too easy or too hard then adjust te difficulty accordingly. If the comments are balanced then your rating is probably about right. You are correct, a puzzle that may be obvious to some from their experience will be totally unsolvable to others unless given a nudge in the right direction. I have one planned that will be obvious to those people who used DOS and early Windows software (maybe) but those who grew up with Apples might have to do a bit more research into how to solve it. So who do I rate it for?

[Zalgariath]

I agree with Richary. My most recent Puzzle Cache I have rated 4 stars on the diff. At GZ as a traditional it would have been a 1.5. To people with my experience the solution comes naturally and 4 stars is a generous reward... I reckon maybe 1 in 3 have a fair clue instantly and with a little work can crack it in 10 or so mins. However, if your brain doesnt work the same as me it may take some lateral thinking, or a hint from a friend. Ive had others kept awake at nights for a week until the penny dropped

When setting a puzzle think of the average man and how tricky it will be for them to either recognise the path to the solution and/or research the solution. One thing to be careful of is using "well known" puzzles that while inherently are very difficult, but due to their popularity are Goggleable, removing the challenge!

I hope that helped... sort of..

[noikmeister]

My feeling is that the puzzles fall into 3 categories:
1: So trivial that it may as well be a traditional
2: Requires only Internet research
3: Requires 1 or more leaps of logic

With category 1 I rate it like a traditional with maybe an extra 1/2 star for the extra effort the finder has to put in to solve. For category 2 I start with 1.5-2 stars for the puzzle and then any extra for the hide. Once you get to category 3 I start with difficulty 3 and then go up depending on how deliberately difficult I might make it, i.e. red herrings, multiple layers etc.

[blossom*]

My feeling is that the puzzles fall into 3 categories:
1: So trivial that it may as well be a traditional
2: Requires only Internet research
3: Requires 1 or more leaps of logic

With category 1 I rate it like a traditional with maybe an extra 1/2 star for the extra effort the finder has to put in to solve. For category 2 I start with 1.5-2 stars for the puzzle and then any extra for the hide. Once you get to category 3 I start with difficulty 3 and then go up depending on how deliberately difficult I might make it, i.e. red herrings, multiple layers etc.

This sounds like a great way to rate these caches - and from someone who should know

[noikmeister]

This sounds like a great way to rate these caches - and from someone who should know

I don't know about that... I am a mere puzzle posing puffer fish in a pool of puzzle piranhas.

[Zytheran]

Hmmm, I guess I should probably throw in my 2 cents worth being a leading puzzle hider in Aus.
(bear in mind this probably applies to SA only)

Here are some rules of thumb I use.
1 How does this new puzzle compare to existing ones that are similar?
2 Substitution ciphers of any type vary from about 1.5 star up to 3. See my Metal cipher series.
3 2 layers of ciphers will put it into 3 to 4.5 . So doing a message as a substitution and then hiding using symbols and colours.
4 Something requiring a penny drop such as some steganography will add 0.5 to 1 star.
5 Always put at least one hint for each step required on difficulty of 4+.
6 Needing to using some special software such as an Enigma or Purple simulator will tend to be 4+. Same applies if the solution can be done using constraint based software to a logic puzzle.
7 Assuming part of the answer to eliminate all but one solution in multi-solution puzzles adds a bit depending on how obviously wrong the other solutions are. i.e. using a multi-solution Sudoku to fool computer solvers. Allowing zero in Kakuro or other rule changes in puzzles to force hand solutions.

Some of my latest 5 star will have 6 stages of converting something to something else. So rearranging or anagram is one stage, a cipher might be another, bit picking using a different base another and then wrap the whole thing up with steganography and some searching of internet info gives a 5 star. See Z's or 2-3-74 which are all 6 +stage and unfound .

The biggest thing though is making sure it matches the other similar puzzles in the area. Especially with a common puzzle like Sudoku. These days it's pretty hard to come up with a new idea so there should be something close. Get to know existing puzzle designers scales both here and OS to get a feel. And/or get someone to check the puzzle bit and see what they say. What may seem obvious to the designer may not be obvious to others. Not that this has ever happened in my case.

[noikmeister]

Purple simulator

Never heard of such a thing! But now that I have, then it means I might have a couple more puzzles to solve.

[Papa Bear_Left]

And, as I've opined before, the rating of a puzzle is almost up to the solver rather than the setter!

Some puzzle veterans can read Vignere ciphers almost without needing to decode them, while some others will consider them a fiendish challenge.

If a puzzle revolves around a particular topic, then aficionados will see it immediately, while those who've never heard of it before won't know where to start. (e.g. text written in Klingon would be child's play to a Trekkie, but just words with too many Ks and apostrophes to normal folks)

As others have said, the best you can aim at is consistency, both within your own puzzles and, as far as possible, with other puzzles in the area.

As a reviewer, I'm often amused at the 4+ ratings given by new placers on their basic Sudoku puzzle, or text hidden in a comment, or as white text, etc.
Of course, cliches are only cliches if you've seen them many times before!

[ForYourEyesOnly]

OK....so I guess it's largely internal consisitency - what you think is difficult or easier. Ultimately what is easy to one is difficult to another and vice versa.

[Zalgariath]

It should perhaps be noted that if the stars system went above 5 a number of Zytherans would qualify for a 6 to 7

[Zytheran]

It should perhaps be noted that if the stars system went above 5 a number of Zytherans would qualify for a 6 to 7

As someone else noticed, exactly this issue occurred with walking/hiking/scrambling/climbing up cliffs which used to stop at 5 under the US system. Then people climbed harder things and so it kept on going up 5.1, 5.2 etc.
However I'm quite sure this will not occur because there is simply no point in making impossible to solve puzzles.
Yes, I could add extra layers and extra stuff to put the level up but then I'd have to add extra clues because no-one would get it, and the level would go down.
I would like to suggest that my two 5 stars puzzles that are currently not found are about as difficult as it's worth getting. Although I'm still not convinced they are really worth setting and/or solving until I get some feedback from future solvers.
(And 2-3-74 would be my most favorite puzzle to date, incorporating some of my favorite puzzle types rather neatly. http://coord.info/GC27E2P)

At the end of the day puzzles are meant to be solved and caches found, hopefully within a year. And then a few finders at least per year to keep things ticking over.

What might be nice for ratings would be maybe some symbols or icons for puzzle type so finders could easily search within Groundspeak or GSAK for puzzle types they like rather than having to inspect each puzzle, especially those with a graphical basis, which no automatic sorter is ever going to work out. People could then get on a theme and do all the sudoku/cipher/word puzzle's they like and not have to worry about the ones they might not. (e.g. ones by yours truly. ) However this is assuming enough people are even interested in unknown/puzzle caches and I'd say that's a minority as it isn't really part of the original Geocaching idea. (However someone has to do it )

Oh, another thing I should mention about puzzle setting. After one year see how many finds it has had and adjust the difficulty if it wasn't quite right or add more or less clues. This should probably apply to both difficulty and terrain of all caches, learn from your finder feedback.

[setsujoku]

learn from your finder feedback.

Assuming that they come back alive

[Zytheran]

learn from your finder feedback.

Assuming that they come back alive

Well, that could explain the 2 solved but virgin caches in terrain 5 I have waiting to be found and logged.