Research Methods 101: Robust Design

Cartoon from XKCD.com

Guest Article by Efe Sevin

I ended my last post with a cliffhanger. I argued how taking ‘brand’ as a whole is an unforgivable mistake. You cannot measure the brand of a country, give it a number, and rank them. You need to figure out what a country’s brand stands for, and whether the target audience is willing to ‘buy’ what that brand stands for or not. In this post, I want to elaborate on this point, and to take a quick look at some of the measurement scales we have to introduce a couple of discussion points.

Now, basically, a measurement should have reliability and validity. Again basically speaking, reliability is the consistency of your measure. Let’s say you want to measure the length of a piece of wood. Your ‘measurement’ scale should show the same length if I measure the same length twice. A reliable measurement scale enables us to compare results across subjects and/or years.

Validity – the more complicated of the duo – is the accuracy of your scale. When you measure the length of the wooden piece, and your result is 25 miles per hour, you have a problem. Your scale should measure what you are trying to measure. Of course, an abstract concept – like a nation brand – is more complicated than a physical one – like length -.

Therefore, your measurement scale should:

  1. Define what a ‘nation brand’ is,
  2. Find out valid measurement scales,
  3. Ensure reliability for comparison across cases and times.

My first question to current measurement scales is this: what does it mean to rank #1 (let it be in NBI, CBI, or any other)? Will other countries listen to Brand #1? Will everyone want to do business with the country? Will there be an overflow of tourists coming into the country?

My second question is this: how do you explain the changes? Let’s take Nation Brands Index for instance. There is a very nice GUI that allows the users to play with the data on http://www.earthspeak.com/. According to the data, Turks ranked US 18th out of 50 in tourism in 2008, and 9th in 2009. Was this change caused by a branding project, or is it just a measurement mistake? Also, going back to my first question, does that mean Turks are more likely to go to US for a summer vacation? Even though, there are several post hoc explanations about the changes, none is ‘scientifically’ satisfying.

Also, before I forget. As most of the measurement scales we have right now are for market purposes. They are products/services sold to places. I believe the cartoon below summarizes all of their publicly declared methodologies.

Just like Coca-Cola....

Long story short, even though it is difficult to clearly define what a place brand is, measurement scales have to provide a very clear cut definition. Their definitions do not need to be accepted by every practitioner and/or scholar, but should be used to constitute the basis for measurement. In my next post, I’ll write about one of the few models that aim to systematically explain place branding, rather than create an air of mystery around the concept.

About Efe

I read and write about political communication stuff and I play with data to see what they have to say. I also love to cook.
This entry was posted in miscellaneous, research and tagged , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s