Below you will learn

Of course: which logos win with which group

but also:

Which group is the strictest and which is the most lenient
That designers and laypeople agree with each other better than designers and marketers
That the work of an academically educated professional evokes the same enthusiasm (and embarrassment) among designers as the work of an enthusiastic amateur
What the logos look like on which all groups agree
Which logos caused the most controversy
How it turns out when we apply Marty Neumeier's Good/Different tool

Why this survey and how did we think of it?

Font magazine organizes an evaluation of logos from a certain field in every issue. It is always evaluated by 4 people from the industry (I am one of them) and we give points (from 1 to 5). Agreement between us is definitely not 100%.

And for a long time, my colleagues from AnFas and I wondered, what if there were more than 4 evaluators? And how would those who commission our work evaluate them? And the end customers?

Because in our work, we always need to reach an agreement with someone. The designer with the client, the client with their customers.

So we took this survey from Font magazine (No. 177 with the evaluation of castle and chateau logos) and approached 82 respondents (designers, clients, and consumers with no relation to the industry).

Data-experienced friends Eva Hubená and Honza Kostkan helped us process the results. Thanks Eva, thanks Honza.

3 groups evaluate: designers, marketers, and laypeople. We monitor where they agree and where they diverge.

82 respondents evaluated 22 castle and chateau logos in the same way it is done in Font magazine.

Respondents rated with stars (1* yuck, 5* great) just like Font magazine did – via Google Forms.

Yes, we admit that judging a logo on its own without knowing the entire identity, context, and product is not correct. But this is how it has been done in Font magazine for many years, and most similar polls suffer from the same drawback. However, we believe that enough aesthetic parameters remain for evaluation (quality of artistic stylization, graphic balance, recognizability, originality, quality/appropriateness of the typography used).

In addition to evaluating the logos themselves, we monitored how respondents agreed within their group and how the results of individual groups differed.

This is what the evaluation output of one of the castle logos looks like. In addition to the rating in each group, we monitor the consensus within each group. Meaning, how consistently the group rates.

Are castle and chateau logos suitable for this type of survey? Yes! More than any other.

Castles and chateaux are not a product driven by conventional marketing. When judging, all groups start on a similar footing – they evaluate the logo itself and its execution much more than the brand that the logo represents. And another thing: We clearly see multiple kinds of work: professional, academic, and enthusiastic amateur. So we waited to see if the individual groups would recognize it.

22 logos chosen by Font magazine participated in the survey

Designers:

They give good logos better grades and agree among themselves

They rate 10 out of 22 logos as above average (giving them more than 3* on average). They are more appreciative than marketers (who rate only 7 out of 22 logos as above average (more than 3*).

Designers evaluated these logos as of higher quality than average (more than 3*)

The winner causes slight disagreement among designers

The logo of Zámek Žďár won among designers. It received the most "five-star" ratings, but paradoxically also quite a few 3*, 2*, and even 1*.

Designers agree quite well elsewhere, but exactly with the winner, their rating variance is quite large.

What designers agree is good

Now we look for consensus. If we look at the logos that received a good grade from designers and at the same time there is a decent consensus in the group, then these logos come out:

These 6 logos received a score higher than 3.2* and good consensus (variance < 0.75*). From this point of view, the highly rated logos of Zámek Žďár and Slezskoostravský hrad dropped out precisely because of the large variance – group disagreement.

Unfortuantely, we do not know the answer to that. It is also worth mentioning that among the 22 logos there were 5 foreign ones. And three of them made it into this selection of "uncontroversial quality." There is no Czech logo with an illustration of a castle here. Did Font select good foreign logos? Or do foreign designers have a better ability to create high-quality, uncontroversial things? Unfortunately, we do not know the answer to that.

What designers agree is not good

Again, we look for consensus. This time on poor quality. These are the logos that received a poor grade from designers, with a decent consensus on that within the group.

These 6 logos received a grade lower than 2.7* and designers agreed well on that (variance < 0.75*). There are other logos with low ratings, e.g., Karlštejn, Zlínský zámek, or Napajedla chateau, but there is not such group consensus over them.

Simply put, designers like neither literal illustrations nor clumsy stylizations.

Where designers' evaluations differed the most

When we look at where the ratings diverged the most, the winners of this category are the logos of the Hartenberg ruins and Prague's Vyšehrad.

Although the group gave a grade slightly above average, the disagreement is the greatest. The listed logos received more or less evenly all ratings (from "bad" to "brilliant").

An observation: According to the response from the castle administration, the Hartenberg logo is the work of a volunteer and was created during an art workcamp. In contrast, the Vyšehrad logo (according to this source) is the work of academically educated designers, passed successfully through a selection procedure, and was chosen by a jury. With the professional audience, both logos fared the same. Designers gave both logos the exact same score to the hundredth and had a similarly large variance. It's as if they couldn't agree on either one whether it's "good" or "bad." The amateur volunteer imprinted some charm into the Hartenberg logo so that a lot of designers forgave the "oddities." On the other hand, the professionalism of the Vyšehrad logo remained unrecognized by many designers.

Good news – it turns out that designers share quality criteria

With designers especially, I was most curious to see if the existence of shared metrics of quality would manifest – precisely through a small variance in ratings. Or if uncompromising distinctiveness and unappreciation of others' work would prevail, which should manifest, among other things, in a large variance of ratings.

Designers agreed on the quality or lack thereof for 2/3 of the logos (15 out of 22). That is good news. It looks as though shared criteria of quality, although unwritten or difficult to explain, exist and designers perceive and acknowledge them.

Ranking of logos by consensus (the lower the variance, the better the position). It shows consensus and does not address whether they agree on quality or lack thereof. Just consensus. Logos with a solid consensus (variance less than 0.75*) are outlined in yellow. Designers reasonably agreed on the quality or lack thereof for 15 out of 22 logos. That is the most of all groups.

Marketers:

As a group, they are critical and do not really share evaluation criteria

Marketers, i.e., those who commission the designers' work, rated more strictly than laypeople. Only 7 logos emerged as above average.

What scored best with marketers:

The logo of the French water castle appealed to marketers the most. And they even had a significant consensus on it (small variance). With that, the consensus in this group more or less ends.

If they were to agree on quality (rating > 3.2*, variance less than 0.75*), then they agreed on only two. With the same parameters, designers agreed on 6 logos.

What marketers agree is not good

Marketers fairly agreed on the below-average quality of these logos. They rated some logos even worse, but not with such consensus.

Notice. Marketers agree that they do not really like abstract stylizations. They are probably missing a depiction of the building there.

What marketers disagree on the most

These logos almost split the marketer group. Zámek Žďár, the winner among designers, is second to last here.

As a group, they send these logos below average, but with a variance like in no other group. The Vyšehrad logo is the record-holder for divisiveness.

Poor Zlínský zámek

Zlínský zámek is the sad loser of the test. Both marketers and laypeople picked on it and put it in last place.

Zlínský zámek received the worst ratings. However, marketers as a group are not so sure about it. Interestingly, designers do not see this logo so bleakly (it got 2.35* and left 5 worse logos behind them).

Marketers: Their attitudes have the largest variance

With designers, we hoped for consistency in their ratings and we got it. We expected it to be similar for marketers. After all, they should be closer to designers than to laypeople. And one could expect some "shared quality criteria" with them as well.

However, it was surprising that marketers as a group rate less consistently than laypeople.

It's probably because a marketer, who bears greater responsibility for their decisions, is more cautious and less optimistic. and caution and strictness manifest both in the overall rating and in a larger variance (lower consistency) of the evaluation.

Logos with a solid consensus in each group (variance less than 0.75*) are outlined in yellow. Designers reasonably agreed on the quality or lack thereof for 15 out of 22 logos, laypeople on 10, and marketers on only 7.

Laypeople:

They give the best grades and surprisingly agree well with each other

Laypeople are the most satisfied group. Their eyes were flattered by 17 out of 22 logos, which they evaluated as above average. Overall, they gave the best grades.

Laypeople appreciated what designers condemned

The logo of the fortress from the Louny region received a record rating from laypeople. So this is what a proper castle logo is supposed to look like according to laypeople. Tvrz Divice, which designers threw in the trash, is practically adored by laypeople (half gave it 4* and a quarter 5*).

Laypeople want an image of a castle in the logo

Yes, according to them, a proper castle or chateau logo should show, hint at what the castle or chateau looks like. We are looking for consensus on a grade better than 3.2*.

On these five logos, laypeople agreed (variance < 0.75*) that they are of high quality (more than 3.2*). Foreign logos predominate. Laypeople agree that there should be a "little house" in the logo.

Laypeople also dislike abstract illustrations

Unclear motifs, references to deeper principles miss expectations…

No little house? No logo! Laypeople also agree that they do not like abstract stylizations… Zlínský zámek has the worst score. Notice by how much of a margin.

Comparison of all groups:

Where do the groups agree and where do they differ? What do the evaluations of laypeople, marketers, and designers have in common?

Reaching a consensus on a project (designer, client, target audience) is essentially the goal of "normal business." Take a look at what patterns can be traced.

General consensus on quality: In only three cases

Now we are looking for agreement between the groups. What do the logos that received good grades in all groups look like? Where did designers, marketers, and consumers agree on quality?

Logos where all groups gave a grade higher than 3.2*.

What do these logos have in common? They have a castle icon in an interesting stylization. The moment the stylization is too literal or clearly clumsy, designers dislike it, and marketers also become hesitant. The moment the "little house" is missing, or the illustration is too abstract, laypeople dislike it (and marketers don't really like it either).

And interestingly, even into this selection of logos with cross-group consensus, two foreign logos made it.

The "coloring book" phenomenon: Professionals are disgusted, laypeople enjoy it quite a bit

"Coloring books" is one respondent's naming for an unsuccessful stylization of a castle. Designers don't like coloring books, marketers timidly echo them. Laypeople, however, like them!

Clumsy stylizations that fail with designers appeal to laypeople

Je zajímavé porovnat tuto skupinu („omalovánky“) s předchozí skupinou („všeobecná shoda“). To, co grafici hodnotí jako pokles kvality, laici nevnímají. Prostě mezi těmito dvěma skupinami log rozdíl nevidí.

Fenomén Tvrz Divice

U loga Tvrz Divice se vyskytují další extrémy. Podle laiků takhle vypadá správné logo hradu. Takhle vysokou známku jiné logo v jiné skupině nedostalo. A žádné logo nevyvolalo takový rozdíl v hodnocení mezi laiky a grafiky.

Fenomén Zámek Žďár

Logo Zámku Žďár nad Sázavou štěpí. Mezi grafiky je toto dílo na prvním místě. Hezké je, že toto neprvoplánové dílo dobře ocenili laici. Dokonce je u nich na 4. místě.

Marketers think the opposite of the other groups

If we admit that among designers (pros) there are shared criteria of quality and they put it in first place, then it is the winner of the professional public. And it is great that the logo resonated with laypeople. Even though it is different and doesn't have a "little house." And as I mentioned above, marketers condemned it as a group.

Here it can be seen that for some projects, you need a visionary marketer.

Zámek Žďár – the winner among designers – was sent to the tail end by marketers (i.e., potential clients). Could it be the fulfillment of the wisdom: A good idea scares everyone?

Further details on agreements and disagreements between groups

In the document Survey of consensus between designers, marketers, and laypeople you will find even more observations from evaluating the differences between groups of evaluators. The Zámek Žďár phenomenon is manifest in several other logos too.

Selected pages:

In the study on page 95, there is a comparison of the rating differences for all logos between marketers and designers. There is a set of designs highly rated by both groups. So-called "safe logos." There are only 5 of them! Then a set well-rated by designers and poorly by marketers (designated as the tension zone). And a set that both groups send below average.

For designers, it can be useful to realize with how good designs they can fail. And that their marketing counterpart can have completely different quality criteria.

Page 96 shows the differences in evaluation between designers and laypeople. And it turns out that laypeople and designers would understand each other quite well. They have a large area of consensus (there are 8 of them). And where they disagree, laypeople would be lenient. In only two cases did designers mark as significantly better something that laypeople marked as below average (the Vyšehrad logo and Zámek Pacov).

Page 97 shows the differences in evaluation between marketers and laypeople. Marketers evaluated the logo better than laypeople in only two cases.

Conclusion? Marty Neumeier will help

In trying to formulate a conclusion, we enlisted the help of the Good / Different tool presented by brand guru Marty Neumeier in his book ZAG (the Good/Different chart). It serves to evaluate elements or attributes of products, services, or brands in general.

But with a certain license we can use it to compare logos with each other.

Let's try to base the Good / Not Good evaluation on the shared opinion of the designer group…
and for the Different / Not Different division, we will choose simply whether the author used an illustration of the building ("little house")

The logos will then be divided as follows:

The GOOD/NOT GOOD division is based on the evaluation of designers. The DIFFERENT/NOT DIFFERENT division is based on whether the author chose a different symbology than the shape of the building. The Hartenberg and Vyšehrad logos caused such strong disagreements that we left them on the borderline.

Now let's see what happens in each quadrant and what can be deduced from it:

Good & Different (2nd quadrant)

Logos from this quadrant were well rated by designers and mostly poorly by everyone else. In a consumer test, they probably wouldn't score very well.

All of us (designers and marketers) try to make things that are good and different. It brings the greatest benefit to companies and brands in the long term.

That difference scares the consumer (layperson) is to be expected. But in this test, it turned out that difference ultimately scared marketers more than laypeople. So if you want to succeed with difference, you need to be able to deliver quality and have a highly educated client. One who has a vision that will overcome the initial animosity of the audience.
If you find each other, wonderful. Your work will function for a long time. It will become an important recognizing feature of the brand it represents. No one will want to change it. And it will age very slowly.

I'll try to remember in 10 years :) Although that's not such a long time.

Good BUT Not Different (1st quadrant)

Logos from this quadrant meet expectations and scored well. In addition to designers, also with laypeople and marketers.

If you as a creator want to reduce problems with the acceptance of your work, or if you have a client who is not very well-informed, it is better to get into this quadrant. Not being too original and innovative. Your work will then bother the fewest people. Consumers won't really care anyway. In their eyes, you win by not differing from "how it usually is." You will manage to push such work through more easily. But your work will help and represent decently. Over time, it will get "old" and "worn out," because it probably won't be timeless. And then it gets replaced. In any case, even such a result is worth it.

Not Good & Not Different (3rd quadrant)

Logos that were rated poorly by designers were rated quite favorably, especially by laypeople.

If you as a creator are conformist, predictable, your imperfections and craft flaws will pass. Cliché is welcomed by a certain part of the audience. Where there are no high demands, your work will be accepted and will somehow function until its shortcomings force someone responsible to replace it with a higher quality one (for instance, from the "good but not different" quadrant).

Not Good BUT Different (4th quadrant)

Logos from this quadrant were rated poorly across all groups.

If you want to be original, different, it is necessary to be able to deliver work that carries the proverbial "touch of the artist" and lacks craft flaws. Otherwise, your work falls into this quadrant. It suffers from misunderstanding from all sides, is not accepted, or is accepted but not respected, and is exchanged, replaced in a relatively short time.

For good, different, and timeless works, courage and alignment are needed. With both the designer and the client

The way I see it, the client's courage should rest on a strong vision, the designer's courage on experience and humility toward the craft. And if the designer and the client align on a similar view of the world, it is a joy to watch the results of the collaboration.

I wish all the courageous ones to cross paths more often.

Appendix

Full study: https://docs.google.com/presentation/d/1gdidUuLfEXRg1hcoFt3CGmKYmeGwoiW0JnxHmyzaQAg
Methodology and disclaimer: Pages 5–11
Evaluation of individual logos, verbal evaluation: Pages 12–78
Summaries and comparisons: Pages 79–103

Offer for designers

If you are designers and would like to use the test for your own purposes, write to me, I will provide the test to you. For instance, you can entertain your client and show them how much the attitudes of their marketing team members differ. Or you might be surprised by what they like.