Appraisal & VAD Label Validation
GoEmotions · human validation study

Sign in

Your progress is saved under your username, so you can stop and resume anytime.

3–32 characters: lowercase letters, digits, and . _ -. Usernames are unique.

What you're doing

Each comment below was automatically scored by AI models on 14 emotion dimensions, each on a scale from -1 to +1. We converted every score into one of three plain-language labels. Your job is to judge, for each dimension, whether that label is a reasonable description of the comment — answering Yes or No. When you answer No, pick the label you think is correct (low / medium / high) and, if you like, add a short reason. You can also leave one overall note per comment.

Every comment is checked independently by three people. Judge based on the text as you read it; there are no trick questions and disagreement is expected and useful.

The label scale

The number line from -1 to +1 is split into three equal bands:

Value rangeLabelMeaning (roughly)
-1.00 … -0.33lowtoward the negative pole
-0.33 … +0.33mediumneutral / mixed / not clearly either way
+0.33 … +1.00hightoward the positive pole

Background: appraisal theory & VAD

Emotions can be described not just by a name (anger, joy…) but by how a person appraises a situation — component-process / appraisal theories (e.g. Scherer, and the OCC model) break an emotional reaction into evaluative components such as how pleasant it is, how much control one has, whether it is fair, whether it advances or blocks one's goals, and so on. The first three dimensions form the classic VAD core (Valence, Arousal, Dominance); the remaining eleven are finer appraisal dimensions. Below, each dimension says what the negative and positive ends mean.

The 14 dimensions

Note: these dimensions come from AI scoring and may be imperfect — that is exactly why your judgment matters. When a label clearly doesn't match the comment, say No.

Study progress

By emotion

EmotionCompleteTotalProgress

Admin access

Enter the admin key to view everyone's activity logs and export the data. (Set via the ADMIN_KEY environment variable.)