????教師

????然后是無(wú)所不在的教師評(píng)分,，尤其是對(duì)大學(xué)教授的評(píng)分。幾十年來(lái),，高等教育機(jī)構(gòu)一直把對(duì)學(xué)生的問(wèn)卷調(diào)查成績(jī)作為衡量教職員工工作表現(xiàn)的一部分,，現(xiàn)在大多數(shù)教育機(jī)構(gòu)都要求課程結(jié)束后收集學(xué)生的評(píng)價(jià)。如果你也像我一樣,，相信教師的本職工作是教書(shū),，好讓學(xué)生能學(xué)到并記住知識(shí)，而不是給學(xué)生講段子,、稱兄道弟,，那么你在衡量學(xué)生的評(píng)分時(shí)就要睜大眼睛，要根據(jù)客觀衡量標(biāo)準(zhǔn)看看學(xué)生究竟學(xué)到了什么,，然后再看看它與教師評(píng)分之間的關(guān)系,，你才知道學(xué)生的打分是否準(zhǔn)確。

????好在給教師打分的做法已經(jīng)實(shí)施了很多年,，對(duì)于學(xué)生評(píng)分與學(xué)生成績(jī)之間的關(guān)系也有大量的研究,。不過(guò)壞消息是，學(xué)生的課程評(píng)價(jià),，幾乎與客觀標(biāo)準(zhǔn)反映的學(xué)生學(xué)習(xí)成果沒(méi)有任何關(guān)系——人們了解這個(gè)事實(shí)已經(jīng)40多年了,。比如，一篇發(fā)表于1972年的論文研究了293名正在學(xué)習(xí)微積分課程的在校大學(xué)生,，結(jié)果發(fā)現(xiàn)：“學(xué)生主觀評(píng)分最低的教員,，獲得了最高的客觀得分?！辈贿^(guò),，盡管學(xué)生評(píng)分對(duì)他們的學(xué)習(xí)成果沒(méi)有任何有價(jià)值的參考意義，但這還是沒(méi)有影響評(píng)分在教育機(jī)構(gòu)中的普遍使用,。

????餐館

????和找醫(yī)生找老師相比,，對(duì)餐館質(zhì)量和用餐體驗(yàn)的打分往往會(huì)客觀一些，引起的后果也更少,。米其林公司從1926年起,，便聘請(qǐng)知識(shí)淵博、經(jīng)驗(yàn)豐富的匿名專家,，到全球各大城市尋找最好的用餐地點(diǎn),。我們可以在TripAdvisor等網(wǎng)站上,，將米其林的評(píng)級(jí)方式與一般老百姓的評(píng)分方式進(jìn)行對(duì)比。

????我選擇了兩座城市,，一座是離我住的地方很近的舊金山,，另一座是我和我妻子最近剛?cè)ミ^(guò)的巴塞羅那。我在2015年的米其林榜單上找到了這兩座城市所有獲得米其林星的餐廳,，然后看了它們?cè)赥ripAdvisor上的評(píng)分,。以下是我的發(fā)現(xiàn)：

????巴塞羅那有21家一星或兩星的米其林餐廳。按理說(shuō),，這些應(yīng)該是這座城市里最好的餐廳,，但其中卻只有一家登上了TripAdvisor的前10名，只有兩家進(jìn)入了前50名,，只有7家進(jìn)入了前100名,。其中一家名叫Nectari的餐廳擁有一顆米其林星，但它在TripAdvisor上僅排在第2262名,；另一家名叫Enoteca的米其林星級(jí)餐廳僅排在第1333名,。

????舊金山的食客兼打分員們對(duì)米其林的尊重也只是多了一點(diǎn)點(diǎn)。在舊金山的24家米其林星級(jí)餐廳中,，只有一家名叫Gary Danko的餐廳在TripAdvisor上排名前10,，但有6家米其林星級(jí)餐廳擠進(jìn)了TripAdvisor的前50名。但作為整個(gè)舊金灣區(qū)僅有的4家二星級(jí)米其林餐廳之一的Coi,，在TripAdvisor上的排名卻僅為第562名,。

????至少對(duì)于這三個(gè)領(lǐng)域，可能還有許多其它領(lǐng)域來(lái)說(shuō),，消費(fèi)者的評(píng)分往往與專家意見(jiàn)或客觀的衡量標(biāo)準(zhǔn)是不相關(guān)的,。當(dāng)然，正因如此,，許多聲譽(yù)管理公司才會(huì)做得這樣成功,，因?yàn)槁曌u(yù)是可以“管理”的（你可以從最好和最差的意義上理解這個(gè)詞匯），而與實(shí)際的質(zhì)量無(wú)關(guān),。

????為什么評(píng)分會(huì)鼓勵(lì)錯(cuò)誤的行為

????由于評(píng)分及其產(chǎn)生的口碑會(huì)產(chǎn)生經(jīng)濟(jì)后果,，人們自然就會(huì)獲得足夠大的激勵(lì)去操控這套體系。一種越來(lái)越普遍的方式就是雇傭“水軍”來(lái)發(fā)布虛假評(píng)價(jià)（或是開(kāi)發(fā)軟件,，不過(guò)軟件是比較容易探測(cè)和預(yù)防的）,。據(jù)一份研究預(yù)測(cè)，在Yelp上,，有16%餐廳評(píng)價(jià)是虛假的,。虛假評(píng)論往往更趨于極端，口碑較差的餐廳通常更傾向于請(qǐng)“水軍”。IT研究機(jī)構(gòu)高德納公司2012年的一份研究估算稱,，有15%的網(wǎng)絡(luò)評(píng)價(jià)都是虛假的,。2013年，紐約州檢察長(zhǎng)“與19家企業(yè)達(dá)成一致,，后者同意停止撰寫不實(shí)評(píng)價(jià)?！?/p>

????數(shù)不清的網(wǎng)站時(shí)不時(shí)都會(huì)冒出招聘水軍的帖子（然后這些帖子又消失了）,，讓他們寫自己的好話，或是寫競(jìng)爭(zhēng)對(duì)手的壞話,。消費(fèi)者網(wǎng)購(gòu)前應(yīng)該充分獲得各方面的評(píng)價(jià)信息,，以做出明智的購(gòu)買決定。保障這些評(píng)價(jià)的真實(shí)性在經(jīng)濟(jì)上具有重要意義,。因此,，亞馬遜和Yelp都在積極構(gòu)建各種算法以篩除不實(shí)評(píng)價(jià)，同時(shí)也在針對(duì)虛假評(píng)價(jià)的發(fā)布者采取法律行動(dòng),。

????Teachers

????Then there are those ubiquitous teacher ratings, particularly of college professors. For decades, higher education institutions have used student surveys as part of the faculty evaluation process, and now most places mandate end-of-course student evaluations. If, like me, you believe that the fundamental job of a teacher is to teach—to impart knowledge that students learn and retain—as contrasted, for instance, with providing entertainment or becoming students’ best friends, then it seems reasonable to measure accuracy by examining the relationship between teacher ratings and what students learn through an objective measurement.

????The good news is that teacher ratings have been done for a long time and there are numerous studies of the relationship between student evaluations and learning. The bad news is that student course evaluations do not have any relationship with objective measures of what students have learned—a fact that has been known for more than four decades. For instance, one paper, published in 1972, studied 293 undergraduates in a calculus course and found that, “Instructors with the lowest subjective ratings received the highest objective scores.” The fact that student ratings do not offer any valuable insight on how well students learn has not affected the prevalence and use of the ratings.

????Restaurants

????Restaurant quality and the dining experience are both more subjective and also have fewer consequences than choosing the right doctor or getting a good teacher. Michelin has, since 1926, employed anonymous, knowledgeable, experienced experts to go to cities all over the world and find the very best places to eat. We can compare how Michelin rates restaurants with the same restaurants’ ratings made by the general public on sites such as TripAdvisor.

????I selected two cities, San Francisco, near where I live, and Barcelona, a place my wife and I recently visited. I looked at the 2015 Michelin lists of the places that earned stars (in San Francisco, I considered only establishments located in the city itself) and also ratings on TripAdvisor. Here’s what I found.

????Barcelona has 21 one- or two-star Michelin restaurants. Of the Michelin-rated establishments, presumably the very best in the city, only one is in TripAdvisor’s top 10, only 2 are in the top 50, and only 7 of the 21 ranked in TripAdvisor’s top 100. Nectari, with 1 Michelin star, ranks 2,262 on TripAdvisor, and Enoteca ranks 1,333.

????Diners/raters in San Francisco agree with Michelin only slightly more. Of San Francisco’s 24 Michelin-starred restaurants, one, Gary Danko, is in TripAdvisor’s top 10, but 6 are in the top 50. However, Coi, one of four places in the entire Bay Area that earned two Michelin stars, ranks just 562 on TripAdvisor.

????At least for these three domains, and quite possibly many others, ratings by consumers—of restaurants, academic instruction, or medical services—are quite uncorrelated with either expert opinion or objective measures of performance. This fact, of course, is precisely why companies in the reputation management space can be successful—reputations can be “managed” in the best and worst sense of that term, regardless of actual quality.

????Why ratings encourage the wrong behaviors

????Because ratings, and the reputations those ratings create, have economic consequences, there are, unsurprisingly, substantial incentives to game the system. One increasingly common way of gaming the system entails hiring people (or developing software, which is fortunately easier to detect and prevent) to post inauthentic reviews. One study estimated that 16% of the restaurant reviews on Yelp were fraudulent, that fraudulent reviews were more extreme, and that restaurants with weak reputations were more likely to commit review fraud. A 2012 study by IT research firm Gartner estimated that 15% of online reviews were fake. In 2013, New York State’s attorney general “announced a deal with 19 businesses that agreed to stop writing fake reviews.”

????Numerous websites pop up (and then disappear) offering to hire people to write positive reviews about you and negative reviews about your competitors. Online purchasing is supposed to give customers access to informative reviews before they make a purchase decision. Maintaining the integrity of these reviews is economically important. Not surprisingly, then, both Amazon.com and Yelp have been increasingly aggressive in their attempts to build algorithms that weed out fake reviews and also to initiate legal action against their perpetrators.