Verified run

google/gemma-4-E4B-it

vllm-gemma-4-e4b-fp8-h100-p002-lexen-v1-20260616-thinking

Rank
129
Accuracy lexEN v1 · WordNet fine
83.30%
Correct
4049 / 4861
Actual run cost
$0.0953
Cost / M items
$19.61

Accuracy across label schemes

The headline accuracy is lexEN v1 · WordNet fine-grained. Each cell re-scores this run's predictions against a different gold label set and sense granularity; coarse counts a prediction correct when its WordNet sense maps to the same Glite or CSI concept as a gold sense. About these label schemes → About coarsening →

Score against WordNet fine-grainedGlite coarse-grainedCSI coarse-grained (Lacerra 2020)
lexEN v1 83.30% 4049 / 4861 92.72% 4507 / 4861 89.01% 4327 / 4861
Maru 2022 (ALLamended) 81.81% 3977 / 4861 92.31% 4487 / 4861 88.11% 4283 / 4861
Raganato 2017 (original) 76.57% 3722 / 4861 89.96% 4373 / 4861 84.47% 4106 / 4861

Model

Display
google/gemma-4-E4B-it
Requested
hosted_vllm/google/gemma-4-E4B-it
Resolved
google/gemma-4-E4B-it
Vendor
Google
API provider
n/a
Hosting
self_hosted
Source
open_source
License
gemma
Reasoning
reasoning
Model URL
https://huggingface.co/google/gemma-4-E4B-it
Endpoint
http://localhost:8000/v1
Resolved counts
google/gemma-4-E4B-it: 4861
HF revision
fee6332c1abaafb77f6f9624236c63aa2f1d0187
Quantization
fp8
Inference engine
vllm
Engine version
0.22.1
Container image
vllm/vllm-openai:v0.22.1
Serve command
exec vllm serve google/gemma-4-E4B-it --port 8000 --served-model-name google/gemma-4-E4B-it --gpu-memory-utilization 0.90 --max-num-seqs 256 --no-enable-prefix-caching --language-model-only --max-model-len 16384 --reasoning-parser gemma4 --default-chat-template-kwargs \{\"enable_thinking\":\ true\} --quantization fp8 --dtype bfloat16 --revision fee6332c1abaafb77f6f9624236c63aa2f1d0187

Reproducibility

Created
2026-06-16T12:07:56.371195+00:00
Run schema
sensebench-run-v2
Commit
n/a
Dataset ID
lexen
Dataset version
lexen-v1
Dataset revision
n/a
Dataset items
4,861
Dataset hash
sha256:5fd4382b93f19087a1e31f6dd7d1db17c1eb17ff80fcbe1d3fdd55c0c3ecefe8
Prompt
p002
Prompt version
0.1.0
Repository files
Browse in GitHub

Runtime

Calls
4861
Total tokens
3,490,390
Tokens / item
718.0
Total cost
$0.0953
Cost / M items
$19.61
Cost source
machine_time_estimate
Elapsed
3.57m
Success items
4,861
Monosemous
0
No candidates
0
No valid vote
0
Invalid votes
0
Transport errors
0

Machine & Timing

GPU
NVIDIA H100 80GB HBM3
VRAM / GPU
81,559 MiB
Driver
570.124.06
CUDA
13.0
CPU
AMD Ryzen 9 7950X 16-Core Processor (32 cores)
RAM
187.8 GiB
Platform
Linux-6.8.0-57-generic-x86_64-with-glibc2.35
Provider
vast.ai (41180527)
Hourly rate
$1.60/h
Benchmark time
3.57m
Machine-hours / 1M items
12.2 h/M
Concurrency
96

Benchmark time covers only the per-item evaluation loop on this machine, excluding model and asset loading.

Runner

GitHub
vassiliphilippov
Name
Vassili Philippov
Contact
n/a

Sampling

Temperature
0.0000
Top P
n/a
Max tokens
4,096
Seed
n/a
Extra
n/a

Policy

Votes / item
1
Reasks / invalid vote
1
Tie break
earliest_vote
Monosemous
short_circuit

Cost And Token Breakdown

Actual recorded run cost: $0.0953. Cost estimate source: machine_time_estimate. Cost is estimated from machine time multiplied by the recorded hourly rate.

Token Type Tokens Cost Price / 1M tokens
Input uncached 1,197,352 n/a n/a
Input cached n/a n/a n/a
Output 2,293,038 n/a n/a
Reasoning output n/a included in output cost n/a

Reasoning output tokens are shown separately when the provider reports them; recorded cost is stored in the output cost bucket.

Download Raw Run Files

Slice Breakdown

Group Bucket Accuracy Correct Items
Candidate Count 11+ 75.57% 600 794
Candidate Count 2 89.47% 765 855
Candidate Count 3-5 85.96% 1672 1945
Candidate Count 6-10 79.87% 1012 1267
POS ADJ 88.32% 582 659
POS ADV 74.36% 145 195
POS NOUN 84.95% 2461 2897
POS VERB 77.57% 861 1110
Source Dataset semeval2013 86.67% 988 1140
Source Dataset semeval2015 83.49% 617 739
Source Dataset senseval2 82.01% 1336 1629
Source Dataset senseval3 81.89% 1108 1353

Worst-Bucket Examples

easily Adv

Instance: semeval2015.d000.s009.t012 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

Alimta is used together with cisplatin (another anticancer medicine) when the cancer is unresectable (cannot be removed by surgery alone) and malignant (has spread, or is likely to spread easily, to other parts of the body), in patients who have not received chemotherapy (medicines for cancer) before advanced or metastatic non-small cell lung cancer that is not affecting the squamous cells.

Show raw prompt

user

Which numbered sense of "easily" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Alimta is used together with cisplatin (another anticancer medicine) when the cancer is unresectable (cannot be removed by surgery alone) and malignant (has spread, or is likely to spread <t>easily</t>, to other parts of the body), in patients who have not received chemotherapy (medicines for cancer) before advanced or metastatic non-small cell lung cancer that is not affecting the squamous cells.

Senses:
1. definition=with ease (`easy' is sometimes used informally for `easily') | examples=she was easily excited
2. definition=without question | examples=easily the best book she's written
3. definition=indicating high probability; in all likelihood | examples=I might well do it

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    with ease (`easy' is sometimes used informally for `easily')

    easily%4:02:01:: · easily.r.01

    • she was easily excited
    • was easily confused

    Synonyms: easy

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    without question

    easily%4:02:06:: · easily.r.02

    • easily the best book she's written
  3. 3

    indicating high probability; in all likelihood

    easily%4:02:05:: · well.r.03

    • I might well do it
    • a mistake that could easily have ended in disaster

    Synonyms: well

    Selected

only Adv

Instance: semeval2015.d000.s013.t000 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

Alimta should only be given under the supervision of a doctor who is qualified in the use of anticancer chemotherapy.

Show raw prompt

user

Which numbered sense of "only" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Alimta should <t>only</t> be given under the supervision of a doctor who is qualified in the use of anticancer chemotherapy.

Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=without any others being included or involved | examples=was entirely to blame
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    only%4:02:02:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, just, but

  2. 2

    without any others being included or involved

    only%4:02:01:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, alone

    Selected
  3. 3

    with nevertheless the final result

    only%4:02:00:: · only.r.03

    • He arrived only to find his wife dead
    • We won only to lose again in the next round
  4. 4

    in the final outcome

    only%4:02:04:: · only.r.04

    • These news will only make you more upset
  5. 5

    except that

    only%4:02:06:: · only.r.05

    • It was the same story; only this time she came out better
  6. 6

    never except when

    only%4:02:03:: · only.r.06

    • call me only if your cold gets worse

    Synonyms: only if, only when

    Gold lexEN Gold Maru 2022 Gold Raganato
  7. 7

    as recently as

    only%4:02:05:: · only.r.07

    • I spoke to him only an hour ago

actually Adv

Instance: semeval2015.d001.s002.t000 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

It is actually not necessary to know MathML to use kalgebra.

Show raw prompt

user

Which numbered sense of "actually" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: It is <t>actually</t> not necessary to know MathML to use kalgebra.

Senses:
1. definition=in actual fact | examples=to be nominally but not actually independent
2. definition=used to imply that one would expect the fact to be the opposite of that stated; surprisingly | examples=you may actually be doing the right thing by walking out
3. definition=at the present moment | examples=the transmission screen shows the picture that is actually on the air
4. definition=as a sentence modifier to add slight emphasis | examples=actually, we all help clear up after a meal

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    in actual fact

    actually%4:02:01:: · actually.r.01

    • to be nominally but not actually independent
    • no one actually saw the shark

    Synonyms: really

    Selected
  2. 2

    used to imply that one would expect the fact to be the opposite of that stated; surprisingly

    actually%4:02:00:: · actually.r.02

    • you may actually be doing the right thing by walking out
    • she actually spoke Latin

    Synonyms: in reality

    Gold lexEN Gold Maru 2022 Gold Raganato
  3. 3

    at the present moment

    actually%4:02:03:: · actually.r.03

    • the transmission screen shows the picture that is actually on the air
  4. 4

    as a sentence modifier to add slight emphasis

    actually%4:02:02:: · actually.r.04

    • actually, we all help clear up after a meal
    • actually, I haven't seen the film
    Gold lexEN Gold Maru 2022 Gold Raganato

now Adv

Instance: semeval2015.d001.s004.t002 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

Although it was initially MathML oriented it can now be used by everyone with little mathematical knowledge.

Show raw prompt

user

Which numbered sense of "now" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Although it was initially MathML oriented it can <t>now</t> be used by everyone with little mathematical knowledge.

Senses:
1. definition=in the historical present; at this point in the narration of a series of past events | examples=President Kennedy now calls in the National Guard
2. definition=in these times; - Nancy Mitford | examples=it is solely by their language that the upper classes nowadays are distinguished
3. definition=used to preface a command or reproof or request | examples=now hear this!
4. definition=at the present moment | examples=goods now on sale
5. definition=without delay or hesitation; with no time intervening | examples=he answered immediately
6. definition=(prefatory or transitional) indicates a change of subject or activity | examples=Now the next problem is...
7. definition=in the immediate past | examples=told me just now

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    in the historical present; at this point in the narration of a series of past events

    now%4:02:05:: · now.r.01

    • President Kennedy now calls in the National Guard
    • Washington now decides to cross the Delaware
  2. 2

    in these times; - Nancy Mitford

    now%4:02:00:: · nowadays.r.01

    • it is solely by their language that the upper classes nowadays are distinguished
    • we now rarely see horse-drawn vehicles on city streets

    Synonyms: nowadays, today

    Gold lexEN Gold Maru 2022
  3. 3

    used to preface a command or reproof or request

    now%4:02:02:: · now.r.03

    • now hear this!
    • now pay attention
  4. 4

    at the present moment

    now%4:02:04:: · now.r.04

    • goods now on sale
    • the now-aging dictator

    Synonyms: at present

    Gold lexEN Gold Maru 2022 Gold Raganato
  5. 5

    without delay or hesitation; with no time intervening

    now%4:02:01:: · immediately.r.01

    • he answered immediately
    • found an answer straightaway

    Synonyms: immediately, instantly, straightaway, straight off, directly, right away, at once, forthwith, like a shot

  6. 6

    (prefatory or transitional) indicates a change of subject or activity

    now%4:02:07:: · now.r.06

    • Now the next problem is...
    Selected
  7. 7

    in the immediate past

    now%4:02:06:: · now.r.07

    • told me just now

here Adv

Instance: semeval2015.d001.s005.t000 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

Here's a screenshot of kalgebra main window

Show raw prompt

user

Which numbered sense of "here" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: <t>Here</t>'s a screenshot of kalgebra main window

Senses:
1. definition=in or at this place; where the speaker or writer is | examples=I work here
2. definition=in this circumstance or respect or on this point or detail | examples=what do we have here?
3. definition=to this place (especially toward the speaker) | examples=come here, please
4. definition=at this time; now | examples=we'll adjourn here for lunch and discuss the remaining issues this afternoon

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    in or at this place; where the speaker or writer is

    here%4:02:00:: · here.r.01

    • I work here
    • turn here
    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    in this circumstance or respect or on this point or detail

    here%4:02:02:: · here.r.02

    • what do we have here?
    • here I must disagree
    Selected
  3. 3

    to this place (especially toward the speaker)

    here%4:02:01:: · here.r.03

    • come here, please

    Synonyms: hither

  4. 4

    at this time; now

    here%4:02:04:: · here.r.04

    • we'll adjourn here for lunch and discuss the remaining issues this afternoon

too Adv

Instance: semeval2015.d001.s015.t004 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

Powers are one way to make roots too.

Show raw prompt

user

Which numbered sense of "too" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Powers are one way to make roots <t>too</t>.

Senses:
1. definition=to a degree exceeding normal or proper limits | examples=too big
2. definition=in addition | examples=he has a Mercedes, too

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    to a degree exceeding normal or proper limits

    too%4:02:00:: · excessively.r.01

    • too big

    Synonyms: excessively, overly, to a fault

    Selected
  2. 2

    in addition

    too%4:02:01:: · besides.r.02

    • he has a Mercedes, too

    Synonyms: besides, also, likewise, as well

    Gold lexEN Gold Maru 2022 Gold Raganato

well Adv

Instance: semeval2015.d003.s021.t001 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

Cerenia was generally well tolerated when administered daily at dosages up to 5 times the recommended doses for an extended period of time (up to 3 times the recommended maximum duration of treatment).

Show raw prompt

user

Which numbered sense of "well" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Cerenia was generally <t>well</t> tolerated when administered daily at dosages up to 5 times the recommended doses for an extended period of time (up to 3 times the recommended maximum duration of treatment).

Senses:
1. definition=(often used as a combining form) in a good or proper or satisfactory manner or to a high standard (`good' is a nonstandard dialectal variant for `well') | examples=the children behaved well
2. definition=thoroughly or completely; fully; often used as a combining form; ; ; ; ; , | examples=The problem is well understood
3. definition=indicating high probability; in all likelihood | examples=I might well do it
4. definition=(used for emphasis or as an intensifier) entirely or fully | examples=a book well worth reading
5. definition=to a suitable or appropriate extent or degree | examples=the project was well underway
6. definition=favorably; with approval | examples=their neighbors spoke well of them
7. definition=to a great extent or degree | examples=I'm afraid the film was well over budget
8. definition=with great or especially intimate knowledge | examples=we knew them well
9. definition=with prudence or propriety | examples=You would do well to say nothing more
10. definition=with skill or in a pleasing manner | examples=she dances well
11. definition=in a manner affording benefit or advantage | examples=she married well
12. definition=in financial comfort | examples=They live well
13. definition=without unusual distress or resentment; with good humor | examples=took the joke well

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    (often used as a combining form) in a good or proper or satisfactory manner or to a high standard (`good' is a nonstandard dialectal variant for `well')

    well%4:02:00:: · well.r.01

    • the children behaved well
    • a task well done

    Synonyms: good

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    thoroughly or completely; fully; often used as a combining form; ; ; ; ; ,

    well%4:02:05:: · well.r.02

    • The problem is well understood
    • she was well informed
  3. 3

    indicating high probability; in all likelihood

    well%4:02:03:: · well.r.03

    • I might well do it
    • a mistake that could easily have ended in disaster

    Synonyms: easily

  4. 4

    (used for emphasis or as an intensifier) entirely or fully

    well%4:02:15:: · well.r.04

    • a book well worth reading
    • was well aware of the difficulties ahead
  5. 5

    to a suitable or appropriate extent or degree

    well%4:02:08:: · well.r.05

    • the project was well underway
    • the fetus has well developed organs
  6. 6

    favorably; with approval

    well%4:02:07:: · well.r.06

    • their neighbors spoke well of them
    • he thought well of the book
  7. 7

    to a great extent or degree

    well%4:02:11:: · well.r.07

    • I'm afraid the film was well over budget
    • painting the room white made it seem considerably (or substantially) larger

    Synonyms: considerably, substantially

  8. 8

    with great or especially intimate knowledge

    well%4:02:14:: · well.r.08

    • we knew them well

    Synonyms: intimately

  9. 9

    with prudence or propriety

    well%4:02:13:: · well.r.09

    • You would do well to say nothing more
    • could not well refuse
  10. 10

    with skill or in a pleasing manner

    well%4:02:12:: · well.r.10

    • she dances well
    • he writes well
  11. 11

    in a manner affording benefit or advantage

    well%4:02:10:: · well.r.11

    • she married well
    • The children were settled advantageously in Seattle

    Synonyms: advantageously

  12. 12

    in financial comfort

    well%4:02:09:: · well.r.12

    • They live well
    • she has been able to live comfortably since her husband died

    Synonyms: comfortably

  13. 13

    without unusual distress or resentment; with good humor

    well%4:02:01:: · well.r.13

    • took the joke well
    • took the tragic news well
    Selected

now Adv

Instance: senseval2.d000.s008.t000 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Now, only one local ringer remains: 64-year-old Derek Hammond.

Show raw prompt

user

Which numbered sense of "now" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: <t>Now</t>, only one local ringer remains: 64-year-old Derek Hammond.

Senses:
1. definition=in the historical present; at this point in the narration of a series of past events | examples=President Kennedy now calls in the National Guard
2. definition=in these times; - Nancy Mitford | examples=it is solely by their language that the upper classes nowadays are distinguished
3. definition=used to preface a command or reproof or request | examples=now hear this!
4. definition=at the present moment | examples=goods now on sale
5. definition=without delay or hesitation; with no time intervening | examples=he answered immediately
6. definition=(prefatory or transitional) indicates a change of subject or activity | examples=Now the next problem is...
7. definition=in the immediate past | examples=told me just now

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    in the historical present; at this point in the narration of a series of past events

    now%4:02:05:: · now.r.01

    • President Kennedy now calls in the National Guard
    • Washington now decides to cross the Delaware
  2. 2

    in these times; - Nancy Mitford

    now%4:02:00:: · nowadays.r.01

    • it is solely by their language that the upper classes nowadays are distinguished
    • we now rarely see horse-drawn vehicles on city streets

    Synonyms: nowadays, today

    Gold Raganato
  3. 3

    used to preface a command or reproof or request

    now%4:02:02:: · now.r.03

    • now hear this!
    • now pay attention
  4. 4

    at the present moment

    now%4:02:04:: · now.r.04

    • goods now on sale
    • the now-aging dictator

    Synonyms: at present

    Gold lexEN Gold Maru 2022
  5. 5

    without delay or hesitation; with no time intervening

    now%4:02:01:: · immediately.r.01

    • he answered immediately
    • found an answer straightaway

    Synonyms: immediately, instantly, straightaway, straight off, directly, right away, at once, forthwith, like a shot

  6. 6

    (prefatory or transitional) indicates a change of subject or activity

    now%4:02:07:: · now.r.06

    • Now the next problem is...
    Selected
  7. 7

    in the immediate past

    now%4:02:06:: · now.r.07

    • told me just now

only Adv

Instance: senseval2.d000.s008.t001 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Now, only one local ringer remains: 64-year-old Derek Hammond.

Show raw prompt

user

Which numbered sense of "only" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Now, <t>only</t> one local ringer remains: 64-year-old Derek Hammond.

Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=without any others being included or involved | examples=was entirely to blame
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    only%4:02:02:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, just, but

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    without any others being included or involved

    only%4:02:01:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, alone

    Selected
  3. 3

    with nevertheless the final result

    only%4:02:00:: · only.r.03

    • He arrived only to find his wife dead
    • We won only to lose again in the next round
  4. 4

    in the final outcome

    only%4:02:04:: · only.r.04

    • These news will only make you more upset
  5. 5

    except that

    only%4:02:06:: · only.r.05

    • It was the same story; only this time she came out better
  6. 6

    never except when

    only%4:02:03:: · only.r.06

    • call me only if your cold gets worse

    Synonyms: only if, only when

  7. 7

    as recently as

    only%4:02:05:: · only.r.07

    • I spoke to him only an hour ago

just Adv

Instance: senseval2.d000.s012.t005 Dataset: senseval2 Bucket: POS / ADV

Incorrect

We've tried to train the youngsters, but they have their discos and their dances, and they just drift away. "

Show raw prompt

user

Which numbered sense of "just" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: We've tried to train the youngsters, but they have their discos and their dances, and they <t>just</t> drift away. "

Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=indicating exactness or preciseness | examples=he was doing precisely (or exactly) what she had told him to do
3. definition=only a moment ago | examples=he has just arrived
4. definition=absolutely | examples=I just can't take it anymore
5. definition=only a very short time before; ; ; ; ; - W.B.Yeats | examples=they could barely hear the speaker
6. definition=exactly at this moment or the moment described | examples=we've just finished painting the walls, so don't touch them

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    just%4:02:00:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, only, but

    Selected
  2. 2

    indicating exactness or preciseness

    just%4:02:05:: · precisely.r.01

    • he was doing precisely (or exactly) what she had told him to do
    • it was just as he said--the jewel was gone

    Synonyms: precisely, exactly

  3. 3

    only a moment ago

    just%4:02:01:: · just.r.03

    • he has just arrived
    • the sun just now came out

    Synonyms: just now

  4. 4

    absolutely

    just%4:02:03:: · just.r.04

    • I just can't take it anymore
    • he was just grand as Romeo

    Synonyms: simply

    Gold lexEN Gold Maru 2022 Gold Raganato
  5. 5

    only a very short time before; ; ; ; ; - W.B.Yeats

    just%4:02:04:: · barely.r.01

    • they could barely hear the speaker
    • we hardly knew them

    Synonyms: barely, hardly, scarcely, scarce

  6. 6

    exactly at this moment or the moment described

    just%4:02:06:: · just.r.06

    • we've just finished painting the walls, so don't touch them

away Adv

Instance: senseval2.d000.s012.t007 Dataset: senseval2 Bucket: POS / ADV

Incorrect

We've tried to train the youngsters, but they have their discos and their dances, and they just drift away. "

Show raw prompt

user

Which numbered sense of "away" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: We've tried to train the youngsters, but they have their discos and their dances, and they just drift <t>away</t>. "

Senses:
1. definition=from a particular thing or place or position (`forth' is obsolete) | examples=ran away from the lion
2. definition=from one's possession | examples=he gave out money to the poor
3. definition=out of the way (especially away from one's thoughts) | examples=brush the objections aside
4. definition=out of existence; ; - H.E.Scudder | examples=the music faded away
5. definition=at a distance in space or time | examples=the boat was 5 miles off (or away)
6. definition=indicating continuing action; continuously or steadily | examples=he worked away at the project for more than a year
7. definition=so as to be removed or gotten rid of | examples=cleared the mess away
8. definition=freely or at will | examples=fire away!
9. definition=in or into a proper place (especially for storage or safekeeping) | examples=put the toys away
10. definition=in a different direction | examples=turn aside
11. definition=in reserve; not for immediate use | examples=started setting aside money to buy a car

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    from a particular thing or place or position (`forth' is obsolete)

    away%4:02:00:: · away.r.01

    • ran away from the lion
    • wanted to get away from there

    Synonyms: off, forth

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    from one's possession

    away%4:02:01:: · away.r.02

    • he gave out money to the poor
    • gave away the tickets

    Synonyms: out

  3. 3

    out of the way (especially away from one's thoughts)

    away%4:02:03:: · aside.r.02

    • brush the objections aside
    • pushed all doubts away

    Synonyms: aside

    Selected
  4. 4

    out of existence; ; - H.E.Scudder

    away%4:02:04:: · away.r.04

    • the music faded away
    • tried to explain away the affair of the letter
  5. 5

    at a distance in space or time

    away%4:02:08:: · off.r.02

    • the boat was 5 miles off (or away)
    • the party is still 2 weeks off (or away)

    Synonyms: off

  6. 6

    indicating continuing action; continuously or steadily

    away%4:02:05:: · away.r.06

    • he worked away at the project for more than a year
    • the child kept hammering away as if his life depended on it
  7. 7

    so as to be removed or gotten rid of

    away%4:02:09:: · away.r.07

    • cleared the mess away
    • the rotted wood had to be cut away
  8. 8

    freely or at will

    away%4:02:10:: · away.r.08

    • fire away!
  9. 9

    in or into a proper place (especially for storage or safekeeping)

    away%4:02:07:: · away.r.09

    • put the toys away
    • her jewels are locked away in a safe
  10. 10

    in a different direction

    away%4:02:06:: · away.r.10

    • turn aside
    • turn away one's face

    Synonyms: aside

  11. 11

    in reserve; not for immediate use

    away%4:02:02:: · aside.r.06

    • started setting aside money to buy a car
    • put something by for her old age

    Synonyms: aside, by

only Adv

Instance: senseval2.d000.s021.t003 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Each variation, or change, can occur only once, the rules state.

Show raw prompt

user

Which numbered sense of "only" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Each variation, or change, can occur <t>only</t> once, the rules state.

Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=without any others being included or involved | examples=was entirely to blame
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    only%4:02:02:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, just, but

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    without any others being included or involved

    only%4:02:01:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, alone

    Selected
  3. 3

    with nevertheless the final result

    only%4:02:00:: · only.r.03

    • He arrived only to find his wife dead
    • We won only to lose again in the next round
  4. 4

    in the final outcome

    only%4:02:04:: · only.r.04

    • These news will only make you more upset
  5. 5

    except that

    only%4:02:06:: · only.r.05

    • It was the same story; only this time she came out better
  6. 6

    never except when

    only%4:02:03:: · only.r.06

    • call me only if your cold gets worse

    Synonyms: only if, only when

  7. 7

    as recently as

    only%4:02:05:: · only.r.07

    • I spoke to him only an hour ago