Verified run

zai-org/GLM-4-9B-0414

vllm-glm-4-9b-bf16-a100-p002-lexen-v1-20260614

Rank
145
Accuracy lexEN v1 · WordNet fine
80.46%
Correct
3911 / 4861
Actual run cost
$0.0242
Cost / M items
$4.98

Accuracy across label schemes

The headline accuracy is lexEN v1 · WordNet fine-grained. Each cell re-scores this run's predictions against a different gold label set and sense granularity; coarse counts a prediction correct when its WordNet sense maps to the same Glite or CSI concept as a gold sense. About these label schemes → About coarsening →

Score against WordNet fine-grainedGlite coarse-grainedCSI coarse-grained (Lacerra 2020)
lexEN v1 80.46% 3911 / 4861 91.30% 4438 / 4861 87.29% 4243 / 4861
Maru 2022 (ALLamended) 79.28% 3854 / 4861 90.74% 4411 / 4861 86.40% 4200 / 4861
Raganato 2017 (original) 73.65% 3580 / 4861 88.44% 4299 / 4861 82.99% 4034 / 4861

Model

Display
zai-org/GLM-4-9B-0414
Requested
hosted_vllm/zai-org/GLM-4-9B-0414
Resolved
zai-org/GLM-4-9B-0414
Vendor
Z.ai
API provider
n/a
Hosting
self_hosted
Source
open_source
License
MIT
Reasoning
n/a
Model URL
https://huggingface.co/zai-org/GLM-4-9B-0414
Endpoint
http://localhost:8000/v1
Resolved counts
zai-org/GLM-4-9B-0414: 4861
HF revision
645b8482494e31b6b752272bf7f7f273ef0f3caf
Quantization
bf16
Inference engine
vllm
Engine version
0.22.1
Container image
vllm/vllm-openai:v0.22.1
Serve command
exec vllm serve zai-org/GLM-4-9B-0414 --port 8000 --served-model-name zai-org/GLM-4-9B-0414 --max-model-len 8192 --gpu-memory-utilization 0.90 --trust-remote-code --revision 645b8482494e31b6b752272bf7f7f273ef0f3caf

Reproducibility

Created
2026-06-14T13:13:51.676910+00:00
Run schema
sensebench-run-v2
Commit
n/a
Dataset ID
lexen
Dataset version
lexen-v1
Dataset revision
n/a
Dataset items
4,861
Dataset hash
sha256:5fd4382b93f19087a1e31f6dd7d1db17c1eb17ff80fcbe1d3fdd55c0c3ecefe8
Prompt
p002
Prompt version
0.1.0
Repository files
Browse in GitHub

Runtime

Calls
4861
Total tokens
1,130,834
Tokens / item
232.6
Total cost
$0.0242
Cost / M items
$4.98
Cost source
machine_time_estimate
Elapsed
1.33m
Success items
4,861
Monosemous
0
No candidates
0
No valid vote
0
Invalid votes
0
Transport errors
0

Machine & Timing

GPU
NVIDIA A100-SXM4-80GB
VRAM / GPU
81,920 MiB
Driver
570.195.03
CUDA
13.0
CPU
AMD EPYC 7513 32-Core Processor (128 cores)
RAM
1,385.7 GiB
Platform
Linux-5.15.0-160-generic-x86_64-with-glibc2.35
Provider
vast.ai (40919271)
Hourly rate
$1.09/h
Benchmark time
1.33m
Machine-hours / 1M items
4.56 h/M
Concurrency
96

Benchmark time covers only the per-item evaluation loop on this machine, excluding model and asset loading.

Runner

GitHub
vassiliphilippov
Name
Vassili Philippov
Contact
n/a

Sampling

Temperature
0.0000
Top P
n/a
Max tokens
128
Seed
n/a
Extra
n/a

Policy

Votes / item
1
Reasks / invalid vote
1
Tie break
earliest_vote
Monosemous
short_circuit

Cost And Token Breakdown

Actual recorded run cost: $0.0242. Cost estimate source: machine_time_estimate. Cost is estimated from machine time multiplied by the recorded hourly rate.

Token Type Tokens Cost Price / 1M tokens
Input uncached 1,116,251 n/a n/a
Input cached n/a n/a n/a
Output 14,583 n/a n/a
Reasoning output n/a included in output cost n/a

Reasoning output tokens are shown separately when the provider reports them; recorded cost is stored in the output cost bucket.

Download Raw Run Files

Slice Breakdown

Group Bucket Accuracy Correct Items
Candidate Count 11+ 68.51% 544 794
Candidate Count 2 88.77% 759 855
Candidate Count 3-5 83.60% 1626 1945
Candidate Count 6-10 77.51% 982 1267
POS ADJ 84.37% 556 659
POS ADV 72.31% 141 195
POS NOUN 83.05% 2406 2897
POS VERB 72.79% 808 1110
Source Dataset semeval2013 82.89% 945 1140
Source Dataset semeval2015 82.14% 607 739
Source Dataset senseval2 79.62% 1297 1629
Source Dataset senseval3 78.49% 1062 1353

Worst-Bucket Examples

game Noun

Instance: semeval2013.d001.s004.t000 Dataset: semeval2013 Bucket: Candidate Count / 11+

Incorrect

The game started out in favor of the Azulgranas, who put themselves ahead on the scoreboard from the start.

Show raw prompt

user

Which numbered sense of "game" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: The <t>game</t> started out in favor of the Azulgranas, who put themselves ahead on the scoreboard from the start.

Senses:
1. definition=a contest with rules to determine a winner | examples=you need four people to play this game
2. definition=a single play of a sport or other contest | examples=the game lasted two hours
3. definition=an amusement or pastime | examples=they played word games
4. definition=animal hunted for food or sport
5. definition=(tennis) a division of play during which one player serves
6. definition=(games) the score at a particular point or the score needed to win | examples=the game is 6 all
7. definition=the flesh of wild animals that is used for food
8. definition=a secret scheme to do something (especially something underhand or illegal) | examples=they concocted a plot to discredit the governor
9. definition=the game equipment needed in order to play a particular game | examples=the child received several games for his birthday
10. definition=your occupation or line of work | examples=he's in the plumbing game
11. definition=frivolous or trifling behavior | examples=for actors, memorizing lines is no game

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    a contest with rules to determine a winner

    game%1:04:00:: · game.n.01

    • you need four people to play this game
    Selected
  2. 2

    a single play of a sport or other contest

    game%1:04:03:: · game.n.02

    • the game lasted two hours
    Gold lexEN Gold Maru 2022 Gold Raganato
  3. 3

    an amusement or pastime

    game%1:04:01:: · game.n.03

    • they played word games
    • he thought of his painting as a game that filled his empty time
  4. 4

    animal hunted for food or sport

    game%1:05:00:: · game.n.04

  5. 5

    (tennis) a division of play during which one player serves

    game%1:28:00:: · game.n.05

  6. 6

    (games) the score at a particular point or the score needed to win

    game%1:23:00:: · game.n.06

    • the game is 6 all
    • he is serving for the game
  7. 7

    the flesh of wild animals that is used for food

    game%1:13:00:: · game.n.07

  8. 8

    a secret scheme to do something (especially something underhand or illegal)

    game%1:09:00:: · plot.n.01

    • they concocted a plot to discredit the governor
    • I saw through his little game from the start

    Synonyms: plot, secret plan

  9. 9

    the game equipment needed in order to play a particular game

    game%1:06:00:: · game.n.09

    • the child received several games for his birthday
  10. 10

    your occupation or line of work

    game%1:04:02:: · game.n.10

    • he's in the plumbing game
    • she's in show biz

    Synonyms: biz

  11. 11

    frivolous or trifling behavior

    game%1:04:04:: · game.n.11

    • for actors, memorizing lines is no game
    • for him, life is all fun and games

game Noun

Instance: semeval2013.d001.s009.t000 Dataset: semeval2013 Bucket: Candidate Count / 11+

Incorrect

But the game, despite being headed for halftime, was not more decidedand the Maccabees tried to remind the Vitorians that not just anybody wins on their court, and with the 'hand of Elias' putting the pressure on, they managed to get involved in the game.

Show raw prompt

user

Which numbered sense of "game" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: But the <t>game</t>, despite being headed for halftime, was not more decidedand the Maccabees tried to remind the Vitorians that not just anybody wins on their court, and with the 'hand of Elias' putting the pressure on, they managed to get involved in the game.

Senses:
1. definition=a contest with rules to determine a winner | examples=you need four people to play this game
2. definition=a single play of a sport or other contest | examples=the game lasted two hours
3. definition=an amusement or pastime | examples=they played word games
4. definition=animal hunted for food or sport
5. definition=(tennis) a division of play during which one player serves
6. definition=(games) the score at a particular point or the score needed to win | examples=the game is 6 all
7. definition=the flesh of wild animals that is used for food
8. definition=a secret scheme to do something (especially something underhand or illegal) | examples=they concocted a plot to discredit the governor
9. definition=the game equipment needed in order to play a particular game | examples=the child received several games for his birthday
10. definition=your occupation or line of work | examples=he's in the plumbing game
11. definition=frivolous or trifling behavior | examples=for actors, memorizing lines is no game

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    a contest with rules to determine a winner

    game%1:04:00:: · game.n.01

    • you need four people to play this game
    Selected
  2. 2

    a single play of a sport or other contest

    game%1:04:03:: · game.n.02

    • the game lasted two hours
    Gold lexEN Gold Maru 2022 Gold Raganato
  3. 3

    an amusement or pastime

    game%1:04:01:: · game.n.03

    • they played word games
    • he thought of his painting as a game that filled his empty time
  4. 4

    animal hunted for food or sport

    game%1:05:00:: · game.n.04

  5. 5

    (tennis) a division of play during which one player serves

    game%1:28:00:: · game.n.05

  6. 6

    (games) the score at a particular point or the score needed to win

    game%1:23:00:: · game.n.06

    • the game is 6 all
    • he is serving for the game
  7. 7

    the flesh of wild animals that is used for food

    game%1:13:00:: · game.n.07

  8. 8

    a secret scheme to do something (especially something underhand or illegal)

    game%1:09:00:: · plot.n.01

    • they concocted a plot to discredit the governor
    • I saw through his little game from the start

    Synonyms: plot, secret plan

  9. 9

    the game equipment needed in order to play a particular game

    game%1:06:00:: · game.n.09

    • the child received several games for his birthday
  10. 10

    your occupation or line of work

    game%1:04:02:: · game.n.10

    • he's in the plumbing game
    • she's in show biz

    Synonyms: biz

  11. 11

    frivolous or trifling behavior

    game%1:04:04:: · game.n.11

    • for actors, memorizing lines is no game
    • for him, life is all fun and games

game Noun

Instance: semeval2013.d001.s009.t004 Dataset: semeval2013 Bucket: Candidate Count / 11+

Incorrect

But the game, despite being headed for halftime, was not more decidedand the Maccabees tried to remind the Vitorians that not just anybody wins on their court, and with the 'hand of Elias' putting the pressure on, they managed to get involved in the game.

Show raw prompt

user

Which numbered sense of "game" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: But the game, despite being headed for halftime, was not more decidedand the Maccabees tried to remind the Vitorians that not just anybody wins on their court, and with the 'hand of Elias' putting the pressure on, they managed to get involved in the <t>game</t>.

Senses:
1. definition=a contest with rules to determine a winner | examples=you need four people to play this game
2. definition=a single play of a sport or other contest | examples=the game lasted two hours
3. definition=an amusement or pastime | examples=they played word games
4. definition=animal hunted for food or sport
5. definition=(tennis) a division of play during which one player serves
6. definition=(games) the score at a particular point or the score needed to win | examples=the game is 6 all
7. definition=the flesh of wild animals that is used for food
8. definition=a secret scheme to do something (especially something underhand or illegal) | examples=they concocted a plot to discredit the governor
9. definition=the game equipment needed in order to play a particular game | examples=the child received several games for his birthday
10. definition=your occupation or line of work | examples=he's in the plumbing game
11. definition=frivolous or trifling behavior | examples=for actors, memorizing lines is no game

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    a contest with rules to determine a winner

    game%1:04:00:: · game.n.01

    • you need four people to play this game
    Selected
  2. 2

    a single play of a sport or other contest

    game%1:04:03:: · game.n.02

    • the game lasted two hours
    Gold lexEN Gold Maru 2022 Gold Raganato
  3. 3

    an amusement or pastime

    game%1:04:01:: · game.n.03

    • they played word games
    • he thought of his painting as a game that filled his empty time
  4. 4

    animal hunted for food or sport

    game%1:05:00:: · game.n.04

  5. 5

    (tennis) a division of play during which one player serves

    game%1:28:00:: · game.n.05

  6. 6

    (games) the score at a particular point or the score needed to win

    game%1:23:00:: · game.n.06

    • the game is 6 all
    • he is serving for the game
  7. 7

    the flesh of wild animals that is used for food

    game%1:13:00:: · game.n.07

  8. 8

    a secret scheme to do something (especially something underhand or illegal)

    game%1:09:00:: · plot.n.01

    • they concocted a plot to discredit the governor
    • I saw through his little game from the start

    Synonyms: plot, secret plan

  9. 9

    the game equipment needed in order to play a particular game

    game%1:06:00:: · game.n.09

    • the child received several games for his birthday
  10. 10

    your occupation or line of work

    game%1:04:02:: · game.n.10

    • he's in the plumbing game
    • she's in show biz

    Synonyms: biz

  11. 11

    frivolous or trifling behavior

    game%1:04:04:: · game.n.11

    • for actors, memorizing lines is no game
    • for him, life is all fun and games

game Noun

Instance: semeval2013.d001.s015.t003 Dataset: semeval2013 Bucket: Candidate Count / 11+

Incorrect

In spite of this, and with Splitter fouling out, Ivonic's team didn't rush, defended well, and, even though the Israelis came close at the last minute, the game was already won and victory escaped Tel Aviv for the first time in the season.

Show raw prompt

user

Which numbered sense of "game" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: In spite of this, and with Splitter fouling out, Ivonic's team didn't rush, defended well, and, even though the Israelis came close at the last minute, the <t>game</t> was already won and victory escaped Tel Aviv for the first time in the season.

Senses:
1. definition=a contest with rules to determine a winner | examples=you need four people to play this game
2. definition=a single play of a sport or other contest | examples=the game lasted two hours
3. definition=an amusement or pastime | examples=they played word games
4. definition=animal hunted for food or sport
5. definition=(tennis) a division of play during which one player serves
6. definition=(games) the score at a particular point or the score needed to win | examples=the game is 6 all
7. definition=the flesh of wild animals that is used for food
8. definition=a secret scheme to do something (especially something underhand or illegal) | examples=they concocted a plot to discredit the governor
9. definition=the game equipment needed in order to play a particular game | examples=the child received several games for his birthday
10. definition=your occupation or line of work | examples=he's in the plumbing game
11. definition=frivolous or trifling behavior | examples=for actors, memorizing lines is no game

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    a contest with rules to determine a winner

    game%1:04:00:: · game.n.01

    • you need four people to play this game
    Selected
  2. 2

    a single play of a sport or other contest

    game%1:04:03:: · game.n.02

    • the game lasted two hours
    Gold lexEN Gold Maru 2022 Gold Raganato
  3. 3

    an amusement or pastime

    game%1:04:01:: · game.n.03

    • they played word games
    • he thought of his painting as a game that filled his empty time
  4. 4

    animal hunted for food or sport

    game%1:05:00:: · game.n.04

  5. 5

    (tennis) a division of play during which one player serves

    game%1:28:00:: · game.n.05

  6. 6

    (games) the score at a particular point or the score needed to win

    game%1:23:00:: · game.n.06

    • the game is 6 all
    • he is serving for the game
  7. 7

    the flesh of wild animals that is used for food

    game%1:13:00:: · game.n.07

  8. 8

    a secret scheme to do something (especially something underhand or illegal)

    game%1:09:00:: · plot.n.01

    • they concocted a plot to discredit the governor
    • I saw through his little game from the start

    Synonyms: plot, secret plan

  9. 9

    the game equipment needed in order to play a particular game

    game%1:06:00:: · game.n.09

    • the child received several games for his birthday
  10. 10

    your occupation or line of work

    game%1:04:02:: · game.n.10

    • he's in the plumbing game
    • she's in show biz

    Synonyms: biz

  11. 11

    frivolous or trifling behavior

    game%1:04:04:: · game.n.11

    • for actors, memorizing lines is no game
    • for him, life is all fun and games

figure Noun

Instance: semeval2013.d002.s002.t001 Dataset: semeval2013 Bucket: Candidate Count / 11+

Incorrect

According to final closing figures, the Dow Jones Industrial Average has risen by 65.67 points to 10 471.50 points, whereas the NASDAQ, with a technology predominance, fell by 0.55 point to 2 190.31 points.

Show raw prompt

user

Which numbered sense of "figure" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: According to final closing <t>figures</t>, the Dow Jones Industrial Average has risen by 65.67 points to 10 471.50 points, whereas the NASDAQ, with a technology predominance, fell by 0.55 point to 2 190.31 points.

Senses:
1. definition=a diagram or picture illustrating textual material | examples=the area covered can be seen from Figure 2
2. definition=alternative names for the body of a human being | examples=Leonardo studied the human body
3. definition=one of the elements that collectively form a system of numeration | examples=0 and 1 are digits
4. definition=a model of a bodily form (especially of a person) | examples=he made a figure of Santa Claus
5. definition=a well-known or notable person | examples=they studied all the great names in the history of France
6. definition=a combination of points and lines and planes that form a visible palpable shape
7. definition=an amount of money expressed numerically | examples=a figure of $17 was suggested
8. definition=the impression produced by a person | examples=he cut a fine figure
9. definition=the property possessed by a sum or total or indefinite quantity of units or individuals | examples=he had a number of chores to do
10. definition=language used in a figurative or nonliteral sense
11. definition=a unitary percept having structure and coherence that is the object of attention and that stands out against a ground
12. definition=a decorative or artistic work | examples=the coach had a design on the doors
13. definition=a predetermined set of movements in dancing or skating | examples=she made the best score on compulsory figures

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    a diagram or picture illustrating textual material

    figure%1:10:00:: · figure.n.01

    • the area covered can be seen from Figure 2

    Synonyms: fig

  2. 2

    alternative names for the body of a human being

    figure%1:08:00:: · human_body.n.01

    • Leonardo studied the human body
    • he has a strong physique

    Synonyms: human body, physical body, material body, soma, build, physique, anatomy, shape, bod, chassis, frame, form, flesh

  3. 3

    one of the elements that collectively form a system of numeration

    figure%1:23:00:: · digit.n.01

    • 0 and 1 are digits

    Synonyms: digit

  4. 4

    a model of a bodily form (especially of a person)

    figure%1:06:00:: · figure.n.04

    • he made a figure of Santa Claus
  5. 5

    a well-known or notable person

    figure%1:18:00:: · name.n.04

    • they studied all the great names in the history of France
    • she is an important figure in modern music

    Synonyms: name, public figure

  6. 6

    a combination of points and lines and planes that form a visible palpable shape

    figure%1:25:00:: · figure.n.06

  7. 7

    an amount of money expressed numerically

    figure%1:21:00:: · figure.n.07

    • a figure of $17 was suggested
    Selected
  8. 8

    the impression produced by a person

    figure%1:07:00:: · figure.n.08

    • he cut a fine figure
    • a heroic figure
  9. 9

    the property possessed by a sum or total or indefinite quantity of units or individuals

    figure%1:07:01:: · number.n.01

    • he had a number of chores to do
    • the number of parameters is small

    Synonyms: number

    Gold lexEN Gold Maru 2022 Gold Raganato
  10. 10

    language used in a figurative or nonliteral sense

    figure%1:10:01:: · trope.n.01

    Synonyms: trope, figure of speech, image

  11. 11

    a unitary percept having structure and coherence that is the object of attention and that stands out against a ground

    figure%1:09:00:: · figure.n.11

  12. 12

    a decorative or artistic work

    figure%1:06:01:: · design.n.04

    • the coach had a design on the doors

    Synonyms: design, pattern

  13. 13

    a predetermined set of movements in dancing or skating

    figure%1:04:00:: · figure.n.13

    • she made the best score on compulsory figures

case Noun

Instance: semeval2013.d003.s011.t002 Dataset: semeval2013 Bucket: Candidate Count / 11+

Incorrect

U.S. firms were in some cases at a disadvantage because rivals, particularly the Chinese and other government controlled energy firms, have markedly lower labor costs and are more prone to take risks because they don't respond to shareholders.

Show raw prompt

user

Which numbered sense of "case" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: U.S. firms were in some <t>cases</t> at a disadvantage because rivals, particularly the Chinese and other government controlled energy firms, have markedly lower labor costs and are more prone to take risks because they don't respond to shareholders.

Senses:
1. definition=an occurrence of something | examples=it was a case of bad judgment
2. definition=a special set of circumstances | examples=in that event, the first possibility is excluded
3. definition=a comprehensive term for any proceeding in a court of law whereby an individual seeks a legal remedy | examples=the family brought suit against the landlord
4. definition=the actual state of things | examples=that was not the case
5. definition=a portable container for carrying several objects | examples=the musicians left their instrument cases backstage
6. definition=a person requiring professional services | examples=a typical case was the suburban housewife described by a marriage counselor
7. definition=a person who is subjected to experimental or other observational procedures; someone who is an object of investigation | examples=the subjects for this investigation were selected randomly
8. definition=a problem requiring investigation | examples=Perry Mason solved the case of the missing heir
9. definition=a statement of facts and reasons used to support an argument | examples=he stated his case clearly
10. definition=the quantity contained in a case
11. definition=nouns or pronouns or adjectives (often marked by inflection) related in some way to other words in a sentence
12. definition=a specific state of mind that is temporary | examples=a case of the jitters
13. definition=a person of a specified kind (usually with many eccentricities) | examples=a real character
14. definition=a specific size and style of type within a type family
15. definition=an enveloping structure or covering enclosing an animal or plant organ or part
16. definition=the housing or outer covering of something | examples=the clock has a walnut case
17. definition=the enclosing frame around a door or window opening | examples=the casings had rotted away and had to be replaced
18. definition=(printing) the receptacle in which a compositor has his type, which is divided into compartments for the different letters, spaces, or numbers | examples=for English, a compositor will ordinarily have two such cases, the upper case containing the capitals and the lower case containing the small letters
19. definition=bed linen consisting of a cover for a pillow | examples=the burglar carried his loot in a pillowcase
20. definition=a glass container used to store and display items in a shop or museum or home

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    an occurrence of something

    case%1:11:00:: · case.n.01

    • it was a case of bad judgment
    • another instance occurred yesterday

    Synonyms: instance, example

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    a special set of circumstances

    case%1:26:00:: · event.n.02

    • in that event, the first possibility is excluded
    • it may rain in which case the picnic will be canceled

    Synonyms: event

    Selected
  3. 3

    a comprehensive term for any proceeding in a court of law whereby an individual seeks a legal remedy

    case%1:04:00:: · lawsuit.n.01

    • the family brought suit against the landlord

    Synonyms: lawsuit, suit, cause, causa

  4. 4

    the actual state of things

    case%1:09:00:: · case.n.04

    • that was not the case
  5. 5

    a portable container for carrying several objects

    case%1:06:00:: · case.n.05

    • the musicians left their instrument cases backstage
  6. 6

    a person requiring professional services

    case%1:18:03:: · case.n.06

    • a typical case was the suburban housewife described by a marriage counselor
  7. 7

    a person who is subjected to experimental or other observational procedures; someone who is an object of investigation

    case%1:18:02:: · subject.n.06

    • the subjects for this investigation were selected randomly
    • the cases that we studied were drawn from two different communities

    Synonyms: subject, guinea pig

  8. 8

    a problem requiring investigation

    case%1:10:02:: · case.n.08

    • Perry Mason solved the case of the missing heir
  9. 9

    a statement of facts and reasons used to support an argument

    case%1:10:01:: · case.n.09

    • he stated his case clearly
  10. 10

    the quantity contained in a case

    case%1:23:00:: · case.n.10

    Synonyms: caseful

  11. 11

    nouns or pronouns or adjectives (often marked by inflection) related in some way to other words in a sentence

    case%1:10:00:: · case.n.11

    Synonyms: grammatical case

  12. 12

    a specific state of mind that is temporary

    case%1:26:02:: · case.n.12

    • a case of the jitters
  13. 13

    a person of a specified kind (usually with many eccentricities)

    case%1:18:04:: · character.n.05

    • a real character
    • a strange character

    Synonyms: character, eccentric, type

  14. 14

    a specific size and style of type within a type family

    case%1:10:03:: · font.n.01

    Synonyms: font, fount, typeface, face

  15. 15

    an enveloping structure or covering enclosing an animal or plant organ or part

    case%1:08:00:: · sheath.n.02

    Synonyms: sheath

  16. 16

    the housing or outer covering of something

    case%1:06:06:: · shell.n.08

    • the clock has a walnut case

    Synonyms: shell, casing

  17. 17

    the enclosing frame around a door or window opening

    case%1:06:07:: · casing.n.03

    • the casings had rotted away and had to be replaced

    Synonyms: casing

  18. 18

    (printing) the receptacle in which a compositor has his type, which is divided into compartments for the different letters, spaces, or numbers

    case%1:06:03:: · case.n.18

    • for English, a compositor will ordinarily have two such cases, the upper case containing the capitals and the lower case containing the small letters

    Synonyms: compositor's case, typesetter's case

  19. 19

    bed linen consisting of a cover for a pillow

    case%1:06:02:: · case.n.19

    • the burglar carried his loot in a pillowcase

    Synonyms: pillowcase, slip, pillow slip

  20. 20

    a glass container used to store and display items in a shop or museum or home

    case%1:06:01:: · case.n.20

    Synonyms: display case, showcase, vitrine

support Noun

Instance: semeval2013.d004.s000.t002 Dataset: semeval2013 Bucket: Candidate Count / 11+

Incorrect

US banks repay state support

Show raw prompt

user

Which numbered sense of "support" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: US banks repay state <t>support</t>

Senses:
1. definition=the activity of providing for or maintaining by supplying with money or necessities | examples=his support kept the family together
2. definition=aiding the cause or policy or interests of | examples=the president no longer has the support of his own party
3. definition=something providing immaterial assistance to a person or cause or interest | examples=the policy found little public support
4. definition=a military operation (often involving new supplies of men and materiel) to strengthen a military force or aid in the performance of its mission | examples=they called for artillery support
5. definition=documentary validation | examples=his documentation of the results was excellent
6. definition=the financial means whereby one lives | examples=each child was expected to pay for their keep
7. definition=supporting structure that holds up or provides a foundation | examples=the statue stood on a marble support
8. definition=the act of bearing the weight of or strengthening | examples=he leaned against the wall for support
9. definition=a musical part (vocal or instrumental) that supports or provides background for other musical parts
10. definition=any device that bears the weight of another thing | examples=there was no place to attach supports for a shelf
11. definition=financial resources provided to make some project possible | examples=the foundation provided support for the experiment

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    the activity of providing for or maintaining by supplying with money or necessities

    support%1:04:04:: · support.n.01

    • his support kept the family together
    • they gave him emotional support during difficult times
    Selected
  2. 2

    aiding the cause or policy or interests of

    support%1:04:00:: · support.n.02

    • the president no longer has the support of his own party
    • they developed a scheme of mutual support
  3. 3

    something providing immaterial assistance to a person or cause or interest

    support%1:09:00:: · support.n.03

    • the policy found little public support
    • his faith was all the support he needed
  4. 4

    a military operation (often involving new supplies of men and materiel) to strengthen a military force or aid in the performance of its mission

    support%1:04:03:: · support.n.04

    • they called for artillery support

    Synonyms: reinforcement, reenforcement

  5. 5

    documentary validation

    support%1:04:02:: · documentation.n.03

    • his documentation of the results was excellent
    • the strongest support for this view is the work of Jones

    Synonyms: documentation

  6. 6

    the financial means whereby one lives

    support%1:21:00:: · support.n.06

    • each child was expected to pay for their keep
    • he applied to the state for support

    Synonyms: keep, livelihood, living, bread and butter, sustenance

  7. 7

    supporting structure that holds up or provides a foundation

    support%1:06:01:: · support.n.07

    • the statue stood on a marble support
  8. 8

    the act of bearing the weight of or strengthening

    support%1:04:01:: · support.n.08

    • he leaned against the wall for support

    Synonyms: supporting

  9. 9

    a musical part (vocal or instrumental) that supports or provides background for other musical parts

    support%1:10:01:: · accompaniment.n.02

    Synonyms: accompaniment, musical accompaniment, backup

  10. 10

    any device that bears the weight of another thing

    support%1:06:00:: · support.n.10

    • there was no place to attach supports for a shelf
  11. 11

    financial resources provided to make some project possible

    support%1:21:01:: · support.n.11

    • the foundation provided support for the experiment

    Synonyms: financial support, funding, backing, financial backing

    Gold lexEN Gold Maru 2022 Gold Raganato

support Noun

Instance: semeval2013.d004.s002.t001 Dataset: semeval2013 Bucket: Candidate Count / 11+

Incorrect

On Wednesday Bank of America repaid the support from the bank protection fund TARP in the amount USD 45 billion (nearly EUR 30 billion) in one go.

Show raw prompt

user

Which numbered sense of "support" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: On Wednesday Bank of America repaid the <t>support</t> from the bank protection fund TARP in the amount USD 45 billion (nearly EUR 30 billion) in one go.

Senses:
1. definition=the activity of providing for or maintaining by supplying with money or necessities | examples=his support kept the family together
2. definition=aiding the cause or policy or interests of | examples=the president no longer has the support of his own party
3. definition=something providing immaterial assistance to a person or cause or interest | examples=the policy found little public support
4. definition=a military operation (often involving new supplies of men and materiel) to strengthen a military force or aid in the performance of its mission | examples=they called for artillery support
5. definition=documentary validation | examples=his documentation of the results was excellent
6. definition=the financial means whereby one lives | examples=each child was expected to pay for their keep
7. definition=supporting structure that holds up or provides a foundation | examples=the statue stood on a marble support
8. definition=the act of bearing the weight of or strengthening | examples=he leaned against the wall for support
9. definition=a musical part (vocal or instrumental) that supports or provides background for other musical parts
10. definition=any device that bears the weight of another thing | examples=there was no place to attach supports for a shelf
11. definition=financial resources provided to make some project possible | examples=the foundation provided support for the experiment

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    the activity of providing for or maintaining by supplying with money or necessities

    support%1:04:04:: · support.n.01

    • his support kept the family together
    • they gave him emotional support during difficult times
    Selected
  2. 2

    aiding the cause or policy or interests of

    support%1:04:00:: · support.n.02

    • the president no longer has the support of his own party
    • they developed a scheme of mutual support
  3. 3

    something providing immaterial assistance to a person or cause or interest

    support%1:09:00:: · support.n.03

    • the policy found little public support
    • his faith was all the support he needed
  4. 4

    a military operation (often involving new supplies of men and materiel) to strengthen a military force or aid in the performance of its mission

    support%1:04:03:: · support.n.04

    • they called for artillery support

    Synonyms: reinforcement, reenforcement

  5. 5

    documentary validation

    support%1:04:02:: · documentation.n.03

    • his documentation of the results was excellent
    • the strongest support for this view is the work of Jones

    Synonyms: documentation

  6. 6

    the financial means whereby one lives

    support%1:21:00:: · support.n.06

    • each child was expected to pay for their keep
    • he applied to the state for support

    Synonyms: keep, livelihood, living, bread and butter, sustenance

  7. 7

    supporting structure that holds up or provides a foundation

    support%1:06:01:: · support.n.07

    • the statue stood on a marble support
  8. 8

    the act of bearing the weight of or strengthening

    support%1:04:01:: · support.n.08

    • he leaned against the wall for support

    Synonyms: supporting

  9. 9

    a musical part (vocal or instrumental) that supports or provides background for other musical parts

    support%1:10:01:: · accompaniment.n.02

    Synonyms: accompaniment, musical accompaniment, backup

  10. 10

    any device that bears the weight of another thing

    support%1:06:00:: · support.n.10

    • there was no place to attach supports for a shelf
  11. 11

    financial resources provided to make some project possible

    support%1:21:01:: · support.n.11

    • the foundation provided support for the experiment

    Synonyms: financial support, funding, backing, financial backing

    Gold lexEN Gold Maru 2022 Gold Raganato

support Noun

Instance: semeval2013.d004.s003.t004 Dataset: semeval2013 Bucket: Candidate Count / 11+

Incorrect

And, according to reports from US broadcaster CNBC, Citigroup is also planning to repay the state support.

Show raw prompt

user

Which numbered sense of "support" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: And, according to reports from US broadcaster CNBC, Citigroup is also planning to repay the state <t>support</t>.

Senses:
1. definition=the activity of providing for or maintaining by supplying with money or necessities | examples=his support kept the family together
2. definition=aiding the cause or policy or interests of | examples=the president no longer has the support of his own party
3. definition=something providing immaterial assistance to a person or cause or interest | examples=the policy found little public support
4. definition=a military operation (often involving new supplies of men and materiel) to strengthen a military force or aid in the performance of its mission | examples=they called for artillery support
5. definition=documentary validation | examples=his documentation of the results was excellent
6. definition=the financial means whereby one lives | examples=each child was expected to pay for their keep
7. definition=supporting structure that holds up or provides a foundation | examples=the statue stood on a marble support
8. definition=the act of bearing the weight of or strengthening | examples=he leaned against the wall for support
9. definition=a musical part (vocal or instrumental) that supports or provides background for other musical parts
10. definition=any device that bears the weight of another thing | examples=there was no place to attach supports for a shelf
11. definition=financial resources provided to make some project possible | examples=the foundation provided support for the experiment

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    the activity of providing for or maintaining by supplying with money or necessities

    support%1:04:04:: · support.n.01

    • his support kept the family together
    • they gave him emotional support during difficult times
    Selected
  2. 2

    aiding the cause or policy or interests of

    support%1:04:00:: · support.n.02

    • the president no longer has the support of his own party
    • they developed a scheme of mutual support
  3. 3

    something providing immaterial assistance to a person or cause or interest

    support%1:09:00:: · support.n.03

    • the policy found little public support
    • his faith was all the support he needed
  4. 4

    a military operation (often involving new supplies of men and materiel) to strengthen a military force or aid in the performance of its mission

    support%1:04:03:: · support.n.04

    • they called for artillery support

    Synonyms: reinforcement, reenforcement

  5. 5

    documentary validation

    support%1:04:02:: · documentation.n.03

    • his documentation of the results was excellent
    • the strongest support for this view is the work of Jones

    Synonyms: documentation

  6. 6

    the financial means whereby one lives

    support%1:21:00:: · support.n.06

    • each child was expected to pay for their keep
    • he applied to the state for support

    Synonyms: keep, livelihood, living, bread and butter, sustenance

  7. 7

    supporting structure that holds up or provides a foundation

    support%1:06:01:: · support.n.07

    • the statue stood on a marble support
  8. 8

    the act of bearing the weight of or strengthening

    support%1:04:01:: · support.n.08

    • he leaned against the wall for support

    Synonyms: supporting

  9. 9

    a musical part (vocal or instrumental) that supports or provides background for other musical parts

    support%1:10:01:: · accompaniment.n.02

    Synonyms: accompaniment, musical accompaniment, backup

  10. 10

    any device that bears the weight of another thing

    support%1:06:00:: · support.n.10

    • there was no place to attach supports for a shelf
  11. 11

    financial resources provided to make some project possible

    support%1:21:01:: · support.n.11

    • the foundation provided support for the experiment

    Synonyms: financial support, funding, backing, financial backing

    Gold lexEN Gold Maru 2022 Gold Raganato

support Noun

Instance: semeval2013.d004.s012.t000 Dataset: semeval2013 Bucket: Candidate Count / 11+

Incorrect

Goldman Sachs, JP Morgan Chase and Morgan Stanley had already repaid their support in June.

Show raw prompt

user

Which numbered sense of "support" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Goldman Sachs, JP Morgan Chase and Morgan Stanley had already repaid their <t>support</t> in June.

Senses:
1. definition=the activity of providing for or maintaining by supplying with money or necessities | examples=his support kept the family together
2. definition=aiding the cause or policy or interests of | examples=the president no longer has the support of his own party
3. definition=something providing immaterial assistance to a person or cause or interest | examples=the policy found little public support
4. definition=a military operation (often involving new supplies of men and materiel) to strengthen a military force or aid in the performance of its mission | examples=they called for artillery support
5. definition=documentary validation | examples=his documentation of the results was excellent
6. definition=the financial means whereby one lives | examples=each child was expected to pay for their keep
7. definition=supporting structure that holds up or provides a foundation | examples=the statue stood on a marble support
8. definition=the act of bearing the weight of or strengthening | examples=he leaned against the wall for support
9. definition=a musical part (vocal or instrumental) that supports or provides background for other musical parts
10. definition=any device that bears the weight of another thing | examples=there was no place to attach supports for a shelf
11. definition=financial resources provided to make some project possible | examples=the foundation provided support for the experiment

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    the activity of providing for or maintaining by supplying with money or necessities

    support%1:04:04:: · support.n.01

    • his support kept the family together
    • they gave him emotional support during difficult times
    Selected
  2. 2

    aiding the cause or policy or interests of

    support%1:04:00:: · support.n.02

    • the president no longer has the support of his own party
    • they developed a scheme of mutual support
  3. 3

    something providing immaterial assistance to a person or cause or interest

    support%1:09:00:: · support.n.03

    • the policy found little public support
    • his faith was all the support he needed
  4. 4

    a military operation (often involving new supplies of men and materiel) to strengthen a military force or aid in the performance of its mission

    support%1:04:03:: · support.n.04

    • they called for artillery support

    Synonyms: reinforcement, reenforcement

  5. 5

    documentary validation

    support%1:04:02:: · documentation.n.03

    • his documentation of the results was excellent
    • the strongest support for this view is the work of Jones

    Synonyms: documentation

  6. 6

    the financial means whereby one lives

    support%1:21:00:: · support.n.06

    • each child was expected to pay for their keep
    • he applied to the state for support

    Synonyms: keep, livelihood, living, bread and butter, sustenance

  7. 7

    supporting structure that holds up or provides a foundation

    support%1:06:01:: · support.n.07

    • the statue stood on a marble support
  8. 8

    the act of bearing the weight of or strengthening

    support%1:04:01:: · support.n.08

    • he leaned against the wall for support

    Synonyms: supporting

  9. 9

    a musical part (vocal or instrumental) that supports or provides background for other musical parts

    support%1:10:01:: · accompaniment.n.02

    Synonyms: accompaniment, musical accompaniment, backup

  10. 10

    any device that bears the weight of another thing

    support%1:06:00:: · support.n.10

    • there was no place to attach supports for a shelf
  11. 11

    financial resources provided to make some project possible

    support%1:21:01:: · support.n.11

    • the foundation provided support for the experiment

    Synonyms: financial support, funding, backing, financial backing

    Gold lexEN Gold Maru 2022 Gold Raganato

house Noun

Instance: semeval2013.d005.s003.t007 Dataset: semeval2013 Bucket: Candidate Count / 11+

Incorrect

In the end, the defense says, there were more than 140,000 Twitter messages about the trial that dealt with rape and children tied to their beds in a house that was set ablaze.

Show raw prompt

user

Which numbered sense of "house" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: In the end, the defense says, there were more than 140,000 Twitter messages about the trial that dealt with rape and children tied to their beds in a <t>house</t> that was set ablaze.

Senses:
1. definition=a dwelling that serves as living quarters for one or more families | examples=he has a house on Cape Cod
2. definition=the members of a business organization that owns or operates one or more establishments | examples=he worked for a brokerage house
3. definition=the members of a religious community living together
4. definition=the audience gathered together in a theatre or cinema | examples=the house applauded
5. definition=an official assembly having legislative powers | examples=a bicameral legislature has two houses
6. definition=aristocratic family line | examples=the House of York
7. definition=play in which children take the roles of father or mother or children and pretend to interact like adults | examples=the children were playing house
8. definition=(astrology) one of 12 equal areas into which the zodiac is divided
9. definition=the management of a gambling house or casino | examples=the house gets a percentage of every bet
10. definition=a social unit living together | examples=he moved his family to Virginia
11. definition=a building where theatrical performances or motion-picture shows can be presented | examples=the house was full
12. definition=a building in which something is sheltered or located | examples=they had a large carriage house

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    a dwelling that serves as living quarters for one or more families

    house%1:06:00:: · house.n.01

    • he has a house on Cape Cod
    • she felt she had to get out of the house
    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    the members of a business organization that owns or operates one or more establishments

    house%1:14:01:: · firm.n.01

    • he worked for a brokerage house

    Synonyms: firm, business firm

  3. 3

    the members of a religious community living together

    house%1:14:04:: · house.n.03

  4. 4

    the audience gathered together in a theatre or cinema

    house%1:14:03:: · house.n.04

    • the house applauded
    • he counted the house
  5. 5

    an official assembly having legislative powers

    house%1:14:02:: · house.n.05

    • a bicameral legislature has two houses
  6. 6

    aristocratic family line

    house%1:14:00:: · house.n.06

    • the House of York
  7. 7

    play in which children take the roles of father or mother or children and pretend to interact like adults

    house%1:04:00:: · house.n.07

    • the children were playing house
  8. 8

    (astrology) one of 12 equal areas into which the zodiac is divided

    house%1:15:00:: · sign_of_the_zodiac.n.01

    Synonyms: sign of the zodiac, star sign, sign, mansion, planetary house

  9. 9

    the management of a gambling house or casino

    house%1:14:06:: · house.n.09

    • the house gets a percentage of every bet
  10. 10

    a social unit living together

    house%1:14:05:: · family.n.01

    • he moved his family to Virginia
    • It was a good Christian household

    Synonyms: family, household, home, menage

  11. 11

    a building where theatrical performances or motion-picture shows can be presented

    house%1:06:02:: · theater.n.01

    • the house was full

    Synonyms: theater, theatre

  12. 12

    a building in which something is sheltered or located

    house%1:06:01:: · house.n.12

    • they had a large carriage house
    Selected

issue Noun

Instance: semeval2013.d005.s010.t001 Dataset: semeval2013 Bucket: Candidate Count / 11+

Incorrect

But that is not likely to be the last word on the issue.

Show raw prompt

user

Which numbered sense of "issue" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: But that is not likely to be the last word on the <t>issue</t>.

Senses:
1. definition=an important question that is in dispute and must be settled | examples=the issue could be settled by requiring public education for everyone
2. definition=one of a series published periodically | examples=she found an old issue of the magazine in her dentist's waiting room
3. definition=some situation or event that is thought about | examples=he kept drifting off the topic
4. definition=the act of providing an item for general use or for official purposes (usually in quantity) | examples=a new issue of stamps
5. definition=supplies (as food or clothing or ammunition) issued by the government
6. definition=the income or profit arising from such transactions as the sale of land or other property | examples=the average return was about 5%
7. definition=a phenomenon that follows and is caused by some previous phenomenon | examples=the magnetic effect was greater when the rod was lengthwise
8. definition=the immediate descendants of a person | examples=she was the mother of many offspring
9. definition=the becoming visible | examples=not a day's difference between the emergence of the andrenas and the opening of the willow catkins
10. definition=an opening that permits escape or release | examples=he blocked the way out
11. definition=the act of issuing printed materials

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    an important question that is in dispute and must be settled

    issue%1:09:01:: · issue.n.01

    • the issue could be settled by requiring public education for everyone
    • politicians never discuss the real issues
    Gold lexEN Gold Raganato
  2. 2

    one of a series published periodically

    issue%1:10:00:: · issue.n.02

    • she found an old issue of the magazine in her dentist's waiting room

    Synonyms: number

    Selected
  3. 3

    some situation or event that is thought about

    issue%1:09:00:: · topic.n.02

    • he kept drifting off the topic
    • he had been thinking about the subject for several years

    Synonyms: topic, subject, matter

    Gold Maru 2022
  4. 4

    the act of providing an item for general use or for official purposes (usually in quantity)

    issue%1:04:01:: · issue.n.04

    • a new issue of stamps
    • the last issue of penicillin was over a month ago

    Synonyms: issuing, issuance

  5. 5

    supplies (as food or clothing or ammunition) issued by the government

    issue%1:21:01:: · issue.n.05

    Synonyms: military issue, government issue

  6. 6

    the income or profit arising from such transactions as the sale of land or other property

    issue%1:21:00:: · return.n.06

    • the average return was about 5%

    Synonyms: return, take, takings, proceeds, yield, payoff

  7. 7

    a phenomenon that follows and is caused by some previous phenomenon

    issue%1:19:00:: · consequence.n.01

    • the magnetic effect was greater when the rod was lengthwise
    • his decision had depressing consequences for business

    Synonyms: consequence, effect, outcome, result, event, upshot

  8. 8

    the immediate descendants of a person

    issue%1:18:00:: · offspring.n.01

    • she was the mother of many offspring
    • he died without issue

    Synonyms: offspring, progeny

  9. 9

    the becoming visible

    issue%1:11:00:: · emergence.n.02

    • not a day's difference between the emergence of the andrenas and the opening of the willow catkins

    Synonyms: emergence, egress

  10. 10

    an opening that permits escape or release

    issue%1:06:00:: · exit.n.01

    • he blocked the way out
    • the canyon had only one issue

    Synonyms: exit, outlet, way out

  11. 11

    the act of issuing printed materials

    issue%1:04:00:: · issue.n.11

    Synonyms: publication