Publications

First Impressions

How Dependable Are Your First Impressions?

July 19, 2019 – Funing Yang

People are able to distinguish credible websites under time constraints with visual and content cues

The usage of first impressions serves as an important tool for us to make judgments instantly. But how accurate it can be is another question. In Jane Austen’s Pride and Prejudice, Elizabeth and Darcy’s prejudiced first impressions towards each other led them to a tangled journey of rediscovery. When lacking information and time, it is easy to fall into the trap of misunderstanding. In the 21st century, the information explosion on the web pushes people to assess the credibility of sources in a suppressed time. In our study, we’ve discovered that when evaluating the credibility of a website under a time constraint, people are able to quickly make correct decisions based on meaningful visual and content cues.

For our study, we invited 42 participants of diverse backgrounds ranging from 15 to 66 years old (mean age=28.3) for a credibility assessment activity as well as a series of interviews. The participants were first introduced to three hypothetical scenarios that featured evaluating the credibility of the website one at a time and were asked to describe how they would act under each. Then, participants were to describe how they defined credibility as well as their strategies for assessing it. The third component of the study involved a credibility rating task, where each participant was asked to rate the credibility of eight websites on a 1 to 7 scale, with 1 being not credible and 7 being very credible.

The credibility rating options as shown on participants’ screen.

Participants were randomly assigned in a different time exposure and advertisements groups (with or without advertisements), in which they had 6s, 12s, or 20s to explore the screenshots of a website and make their evaluation. In the end, the participants were asked to reflect upon the previous task and share any new insights regarding the strategies for evaluating credibility. 

“Fake”
“Real”

The study demonstrated the participants were able to distinguish between “real news” websites and “fake news” websites. In the credibility rating task, we’ve discovered that there was a significant difference in how the participants rated the two types of websites, with most participants rated the “real news” websites higher than the other. More specifically, 54% of the scores of “real news” websites lie between 5 and 7, and only 20% of their ratings were under the non-credible interval, which was from 1 to 3. To the contrary, 54% of the ratings of “fake news” websites lied in the non-credible interval (1 to 3) while 30% were on the credible side.

Another important aspect of our research is the various strategies that the participants employed to identify “real” or “fake” websites, referred to as credibility signals, that the participants mentioned during the interviews both before and after the credibility assessment task (pre-task and post-task). Our study deliberately did not use pre-existing definitions of credibility and did not suggest strategies for credibility assessment to the participants. Instead, we extracted the criteria for evaluating credibility from the participants’ justifications. After identifying a large group of credibility signals, we categorized them into signals related to textual content as well as those related to visual aspects. The table below demonstrates some of the most frequent signals as well as the quotations from our interviews.

Comparing the signals of the pre-task group and the post-task group, we’ve discovered a large increase in many signals that the participants described after the credibility assessment task. For instance, the visual signals were only mentioned 16 times in the pre-task interviews, but 127 times post-task. The increased frequency indicated that people were unaware of the potentially large effect that the signals have on their credibility assessment. In terms of comparing visual and content signals, we’ve discovered that contrary to participants’ self-reports, participants were using almost equal percentage of textual content and visual signals, while the participants may not be aware of their reliance on the visual aspects. 

Our study also demonstrated the impact of advertisements on the evaluation abilities of the participants. In the assessment task, the participants were randomly assigned the screenshots of the “real news” and “fake news” websites with or without ads. When there are no ads present, most people could make accurate distinctions between the “real” and “fake” news sites. However, the ads proved to be a large distraction.

Thus, what are the take-aways for our study?

The good news

In the majority of cases, people do possess the ability to make correct judgments under a time constraint.

However

The websites still present many distractions for us that could affect our impressions

Strategies for improved web literacy

Although ad-blockers save us from the trouble of receiving ads when visiting an unfamiliar website, it is wise to turn the ad-blocker off in order to observe the quantity and quality of the ads on the web-page, which serve as an important signal for a “fake news” website.

Also, just because a website uses a familiar structure that resembles some other well-known, established websites, it does not necessarily suggest the credibility of a website. Ask yourself what makes you trust the site. If the appearance turns out to be the only factor, then it is wise to be suspicious and find other reasons.

Be sure to give yourself some time to assess the credibility of a website, more time can improve your accuracy with the evaluation.