Verified run

google/gemma-4-31B-it

vllm-gemma-4-31b-fp8-h100-p003-lexen-v1-20260616

Rank
43
Accuracy lexEN v1 · WordNet fine
92.29%
Correct
4486 / 4861
Actual run cost
$0.0990
Cost / M items
$20.36

Accuracy across label schemes

The headline accuracy is lexEN v1 · WordNet fine-grained. Each cell re-scores this run's predictions against a different gold label set and sense granularity; coarse counts a prediction correct when its WordNet sense maps to the same Glite or CSI concept as a gold sense. About these label schemes → About coarsening →

Score against WordNet fine-grainedGlite coarse-grainedCSI coarse-grained (Lacerra 2020)
lexEN v1 92.29% 4486 / 4861 97.20% 4725 / 4861 95.33% 4634 / 4861
Maru 2022 (ALLamended) 89.38% 4345 / 4861 96.28% 4680 / 4861 93.44% 4542 / 4861
Raganato 2017 (original) 82.99% 4034 / 4861 93.48% 4544 / 4861 89.22% 4337 / 4861

Model

Display
google/gemma-4-31B-it
Requested
hosted_vllm/google/gemma-4-31B-it
Resolved
google/gemma-4-31B-it
Vendor
Google
API provider
n/a
Hosting
self_hosted
Source
open_source
License
gemma
Reasoning
no reasoning
Model URL
https://huggingface.co/google/gemma-4-31B-it
Endpoint
http://localhost:8000/v1
Resolved counts
google/gemma-4-31B-it: 4869
HF revision
0cae789c220fdf68c179ff9b18203021568e61cb
Quantization
fp8
Inference engine
vllm
Engine version
0.22.1
Container image
vllm/vllm-openai:v0.22.1
Serve command
exec vllm serve RedHatAI/gemma-4-31B-it-FP8-Dynamic --port 8000 --served-model-name google/gemma-4-31B-it --max-model-len 8192 --gpu-memory-utilization 0.90 --language-model-only --revision 0cae789c220fdf68c179ff9b18203021568e61cb

Reproducibility

Created
2026-06-16T03:16:10.855287+00:00
Run schema
sensebench-run-v2
Commit
n/a
Dataset ID
lexen
Dataset version
lexen-v1
Dataset revision
n/a
Dataset items
4,861
Dataset hash
sha256:5fd4382b93f19087a1e31f6dd7d1db17c1eb17ff80fcbe1d3fdd55c0c3ecefe8
Prompt
p003
Prompt version
0.1.0
Repository files
Browse in GitHub

Runtime

Calls
4869
Total tokens
2,164,148
Tokens / item
445.2
Total cost
$0.0990
Cost / M items
$20.36
Cost source
machine_time_estimate
Elapsed
2.72m
Success items
4,861
Monosemous
0
No candidates
0
No valid vote
0
Invalid votes
0
Transport errors
0

Machine & Timing

GPU
NVIDIA H100 80GB HBM3
VRAM / GPU
81,559 MiB
Driver
595.71.05
CUDA
13.2
CPU
AMD EPYC 7R13 Processor (192 cores)
RAM
1,999.9 GiB
Platform
Linux-6.17.0-1017-aws-x86_64-with-glibc2.35
Provider
vast.ai (41133741)
Hourly rate
$2.18/h
Benchmark time
2.72m
Machine-hours / 1M items
9.34 h/M
Concurrency
128

Benchmark time covers only the per-item evaluation loop on this machine, excluding model and asset loading.

Runner

GitHub
vassiliphilippov
Name
Vassili Philippov
Contact
n/a

Sampling

Temperature
0.0000
Top P
n/a
Max tokens
2,048
Seed
n/a
Extra
n/a

Policy

Votes / item
1
Reasks / invalid vote
1
Tie break
earliest_vote
Monosemous
short_circuit

Cost And Token Breakdown

Actual recorded run cost: $0.0990. Cost estimate source: machine_time_estimate. Cost is estimated from machine time multiplied by the recorded hourly rate.

Token Type Tokens Cost Price / 1M tokens
Input uncached 2,153,921 n/a n/a
Input cached n/a n/a n/a
Output 10,227 n/a n/a
Reasoning output n/a included in output cost n/a

Reasoning output tokens are shown separately when the provider reports them; recorded cost is stored in the output cost bucket.

Download Raw Run Files

Slice Breakdown

Group Bucket Accuracy Correct Items
Candidate Count 11+ 89.29% 709 794
Candidate Count 2 96.02% 821 855
Candidate Count 3-5 92.85% 1806 1945
Candidate Count 6-10 90.77% 1150 1267
POS ADJ 93.78% 618 659
POS ADV 87.18% 170 195
POS NOUN 93.03% 2695 2897
POS VERB 90.36% 1003 1110
Source Dataset semeval2013 94.04% 1072 1140
Source Dataset semeval2015 92.29% 682 739
Source Dataset senseval2 91.65% 1493 1629
Source Dataset senseval3 91.57% 1239 1353

Worst-Bucket Examples

only Adv

Instance: semeval2015.d000.s011.t001 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

It contains the active substance pemetrexed.

What is Alimta used for?

Alimta is used to treat two types of cancer: pleural mesothelioma (cancer of the lining of the lungs that is usually caused by exposure to asbestos).

Alimta is used together with cisplatin (another anticancer medicine) when the cancer is unresectable (cannot be removed by surgery alone) and malignant (has spread, or is likely to spread easily, to other parts of the body), in patients who have not received chemotherapy (medicines for cancer) before advanced or metastatic non-small cell lung cancer that is not affecting the squamous cells.

Alimta is used in combination with cisplatin in patients who have not been treated before, and on its own in patients who have received another type of chemotherapy in the past.

The medicine can only be obtained with a prescription.

How is Alimta used?

Show raw prompt

user

Which numbered sense of "only" (marked with <t></t>) is used in the context below? Return only the number of the correct sense.

Context: It contains the active substance pemetrexed. What is Alimta used for? Alimta is used to treat two types of cancer: pleural mesothelioma (cancer of the lining of the lungs that is usually caused by exposure to asbestos). Alimta is used together with cisplatin (another anticancer medicine) when the cancer is unresectable (cannot be removed by surgery alone) and malignant (has spread, or is likely to spread easily, to other parts of the body), in patients who have not received chemotherapy (medicines for cancer) before advanced or metastatic non-small cell lung cancer that is not affecting the squamous cells. Alimta is used in combination with cisplatin in patients who have not been treated before, and on its own in patients who have received another type of chemotherapy in the past. The medicine can <t>only</t> be obtained with a prescription. How is Alimta used?

Senses:
1. definition=and nothing more | synonyms=merely, simply, just, but | examples=I was merely asking; it is simply a matter of time
2. definition=without any others being included or involved | synonyms=entirely, exclusively, solely, alone | examples=was entirely to blame; a school devoted entirely to the needs of problem children
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead; We won only to lose again in the next round
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | synonyms=only if, only when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago

Prediction matches gold: lexEN ✗ Maru 2022 ✓ Raganato ✓

WordNet 3.0 senses

  1. 1

    and nothing more

    only%4:02:02:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, just, but

  2. 2

    without any others being included or involved

    only%4:02:01:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, alone

    Gold lexEN
  3. 3

    with nevertheless the final result

    only%4:02:00:: · only.r.03

    • He arrived only to find his wife dead
    • We won only to lose again in the next round
  4. 4

    in the final outcome

    only%4:02:04:: · only.r.04

    • These news will only make you more upset
  5. 5

    except that

    only%4:02:06:: · only.r.05

    • It was the same story; only this time she came out better
  6. 6

    never except when

    only%4:02:03:: · only.r.06

    • call me only if your cold gets worse

    Synonyms: only if, only when

    Gold Maru 2022 Gold Raganato Selected
  7. 7

    as recently as

    only%4:02:05:: · only.r.07

    • I spoke to him only an hour ago

locally Adv

Instance: semeval2015.d000.s027.t010 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

The active substance in Alimta, pemetrexed, is a cytotoxic medicine (a medicine that kills cells that are dividing, such as cancer cells), which belongs to the group antimetabolites.

In the body, pemetrexed is converted into an active form that blocks the activity of the enzymes that are involved in producing nucleotides (the building blocks of DNA and RNA, the genetic material of cells).

How has Alimta been studied?

For the treatment of malignant pleural mesothelioma, Alimta has been studied in one main study in 456 patients who had not received chemotherapy for their disease before.

The effects of Alimta in combination with cisplatin were compared with those of cisplatin alone.

For the treatment of non-small cell lung cancer, the effects of Alimta were compared with those of docetaxel (another anticancer medicine) in one study involving 571 patients with locally advanced or metastatic disease who had received chemotherapy in the past.

Alimta was also compared with gemcitabine (another anticancer medicine), both in combination with cisplatin, in a study involving 1,725 patients who had not received chemotherapy for lung cancer in the past.

Show raw prompt

user

Which numbered sense of "locally" (marked with <t></t>) is used in the context below? Return only the number of the correct sense.

Context: The active substance in Alimta, pemetrexed, is a cytotoxic medicine (a medicine that kills cells that are dividing, such as cancer cells), which belongs to the group antimetabolites. In the body, pemetrexed is converted into an active form that blocks the activity of the enzymes that are involved in producing nucleotides (the building blocks of DNA and RNA, the genetic material of cells). How has Alimta been studied? For the treatment of malignant pleural mesothelioma, Alimta has been studied in one main study in 456 patients who had not received chemotherapy for their disease before. The effects of Alimta in combination with cisplatin were compared with those of cisplatin alone. For the treatment of non-small cell lung cancer, the effects of Alimta were compared with those of docetaxel (another anticancer medicine) in one study involving 571 patients with <t>locally</t> advanced or metastatic disease who had received chemotherapy in the past. Alimta was also compared with gemcitabine (another anticancer medicine), both in combination with cisplatin, in a study involving 1,725 patients who had not received chemotherapy for lung cancer in the past.

Senses:
1. definition=by a particular locality | examples=it was locally decided
2. definition=to a restricted area of the body | synonyms=topically | examples=apply this medicine topically

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    by a particular locality

    locally%4:02:00:: · locally.r.01

    • it was locally decided
    Selected
  2. 2

    to a restricted area of the body

    locally%4:02:01:: · locally.r.02

    • apply this medicine topically

    Synonyms: topically

    Gold lexEN Gold Maru 2022 Gold Raganato

here Adv

Instance: semeval2015.d001.s005.t000 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

kalgebra is a mathematical calculator based on content markup MathML language.

Nowadays it is capable to make simple MathML operations (arithmetic and logical) and to representate 2D and 3D graphs.

It is actually not necessary to know MathML to use kalgebra.

kalgebra is a MathML based graph calculator.

Although it was initially MathML oriented it can now be used by everyone with little mathematical knowledge.

Here's a screenshot of kalgebra main window

kalgebra main window

Show raw prompt

user

Which numbered sense of "here" (marked with <t></t>) is used in the context below? Return only the number of the correct sense.

Context: kalgebra is a mathematical calculator based on content markup MathML language. Nowadays it is capable to make simple MathML operations (arithmetic and logical) and to representate 2D and 3D graphs. It is actually not necessary to know MathML to use kalgebra. kalgebra is a MathML based graph calculator. Although it was initially MathML oriented it can now be used by everyone with little mathematical knowledge. <t>Here</t>'s a screenshot of kalgebra main window kalgebra main window

Senses:
1. definition=in or at this place; where the speaker or writer is | examples=I work here; turn here
2. definition=in this circumstance or respect or on this point or detail | examples=what do we have here?; here I must disagree
3. definition=to this place (especially toward the speaker) | synonyms=hither | examples=come here, please
4. definition=at this time; now | examples=we'll adjourn here for lunch and discuss the remaining issues this afternoon

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    in or at this place; where the speaker or writer is

    here%4:02:00:: · here.r.01

    • I work here
    • turn here
    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    in this circumstance or respect or on this point or detail

    here%4:02:02:: · here.r.02

    • what do we have here?
    • here I must disagree
    Selected
  3. 3

    to this place (especially toward the speaker)

    here%4:02:01:: · here.r.03

    • come here, please

    Synonyms: hither

  4. 4

    at this time; now

    here%4:02:04:: · here.r.04

    • we'll adjourn here for lunch and discuss the remaining issues this afternoon

only Adv

Instance: semeval2015.d003.s019.t003 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

A large number of studies with Cerenia were carried out either in laboratory dogs or in animal patients at veterinary practises in several European countries and in the USA.

What benefit has Cerenia shown during the studies?

The results of the studies showed that Cerenia was more effective than the placebo: less vomiting was seen in dogs that received the medicine than in dogs that received a placebo, both in the treatment or in the prevention of vomiting.

The treatment of a vomiting dog should be started with the injection, since vomiting animals might not be able to keep a tablet down (as they might throw the intact tablet up again).

For follow-up treatment, the animal owner can administer the tablets to the dog.

Also, the treatment of vomiting should only be together with other supportive measure or other veterinary therapy while addressing the underlying causes of the vomiting.

What is the risk associated with Cerenia?

Show raw prompt

user

Which numbered sense of "only" (marked with <t></t>) is used in the context below? Return only the number of the correct sense.

Context: A large number of studies with Cerenia were carried out either in laboratory dogs or in animal patients at veterinary practises in several European countries and in the USA. What benefit has Cerenia shown during the studies? The results of the studies showed that Cerenia was more effective than the placebo: less vomiting was seen in dogs that received the medicine than in dogs that received a placebo, both in the treatment or in the prevention of vomiting. The treatment of a vomiting dog should be started with the injection, since vomiting animals might not be able to keep a tablet down (as they might throw the intact tablet up again). For follow-up treatment, the animal owner can administer the tablets to the dog. Also, the treatment of vomiting should <t>only</t> be together with other supportive measure or other veterinary therapy while addressing the underlying causes of the vomiting. What is the risk associated with Cerenia?

Senses:
1. definition=and nothing more | synonyms=merely, simply, just, but | examples=I was merely asking; it is simply a matter of time
2. definition=without any others being included or involved | synonyms=entirely, exclusively, solely, alone | examples=was entirely to blame; a school devoted entirely to the needs of problem children
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead; We won only to lose again in the next round
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | synonyms=only if, only when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    only%4:02:02:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, just, but

  2. 2

    without any others being included or involved

    only%4:02:01:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, alone

    Gold lexEN Gold Maru 2022 Gold Raganato
  3. 3

    with nevertheless the final result

    only%4:02:00:: · only.r.03

    • He arrived only to find his wife dead
    • We won only to lose again in the next round
  4. 4

    in the final outcome

    only%4:02:04:: · only.r.04

    • These news will only make you more upset
  5. 5

    except that

    only%4:02:06:: · only.r.05

    • It was the same story; only this time she came out better
  6. 6

    never except when

    only%4:02:03:: · only.r.06

    • call me only if your cold gets worse

    Synonyms: only if, only when

    Selected
  7. 7

    as recently as

    only%4:02:05:: · only.r.07

    • I spoke to him only an hour ago

always Adv

Instance: senseval2.d000.s004.t007 Dataset: senseval2 Bucket: POS / ADV

Incorrect

The art of change-ringing is peculiar to the English, and, like most English peculiarities, unintelligible to the rest of the world.

Dorothy L. Sayers, "The Nine Tailors "

ASLACTON, England --

Of all scenes that evoke rural England, this is one of the loveliest: An ancient stone church stands amid the fields, the sound of bells cascading from its tower, calling the faithful to evensong.

The parishioners of St. Michael and All Angels stop to chat at the church door, as members here always have.

In the tower, five men and women pull rhythmically on ropes attached to the same five bells that first sounded here in 1614.

Show raw prompt

user

Which numbered sense of "always" (marked with <t></t>) is used in the context below? Return only the number of the correct sense.

Context: The art of change-ringing is peculiar to the English, and, like most English peculiarities, unintelligible to the rest of the world. Dorothy L. Sayers, "The Nine Tailors "ASLACTON, England -- Of all scenes that evoke rural England, this is one of the loveliest: An ancient stone church stands amid the fields, the sound of bells cascading from its tower, calling the faithful to evensong. The parishioners of St. Michael and All Angels stop to chat at the church door, as members here <t>always</t> have. In the tower, five men and women pull rhythmically on ropes attached to the same five bells that first sounded here in 1614.

Senses:
1. definition=at all times; all the time and on every occasion | synonyms=ever, e'er | examples=I will always be there to help you; always arrives on time
2. definition=without variation or change, in every case | synonyms=constantly, invariably | examples=constantly kind and gracious; he always arrives on time
3. definition=without interruption | synonyms=constantly, forever, perpetually, incessantly | examples=the world is constantly changing
4. definition=at any time or in any event | examples=you can always resign if you don't like it; you could always take a day off
5. definition=forever; throughout all time | examples=we will always be friends; I shall treasure it always

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    at all times; all the time and on every occasion

    always%4:02:00:: · always.r.01

    • I will always be there to help you
    • always arrives on time

    Synonyms: ever, e'er

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    without variation or change, in every case

    always%4:02:04:: · constantly.r.01

    • constantly kind and gracious
    • he always arrives on time

    Synonyms: constantly, invariably

    Selected
  3. 3

    without interruption

    always%4:02:03:: · constantly.r.02

    • the world is constantly changing

    Synonyms: constantly, forever, perpetually, incessantly

  4. 4

    at any time or in any event

    always%4:02:02:: · always.r.04

    • you can always resign if you don't like it
    • you could always take a day off
  5. 5

    forever; throughout all time

    always%4:02:01:: · always.r.05

    • we will always be friends
    • I shall treasure it always

now Adv

Instance: senseval2.d000.s008.t000 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Of all scenes that evoke rural England, this is one of the loveliest: An ancient stone church stands amid the fields, the sound of bells cascading from its tower, calling the faithful to evensong.

The parishioners of St. Michael and All Angels stop to chat at the church door, as members here always have.

In the tower, five men and women pull rhythmically on ropes attached to the same five bells that first sounded here in 1614.

But there is also a discordant, modern note in Aslacton, though it can not be heard by the church-goers enjoying the peal of bells this cool autumn evening.

Like most of the other 6,000 churches in Britain with sets of bells, St. Michael once had its own "band "of ringers, who would herald every Sunday morning and evening service.

Now, only one local ringer remains: 64-year-old Derek Hammond.

The others here today live elsewhere.

Show raw prompt

user

Which numbered sense of "now" (marked with <t></t>) is used in the context below? Return only the number of the correct sense.

Context: Of all scenes that evoke rural England, this is one of the loveliest: An ancient stone church stands amid the fields, the sound of bells cascading from its tower, calling the faithful to evensong. The parishioners of St. Michael and All Angels stop to chat at the church door, as members here always have. In the tower, five men and women pull rhythmically on ropes attached to the same five bells that first sounded here in 1614. But there is also a discordant, modern note in Aslacton, though it can not be heard by the church-goers enjoying the peal of bells this cool autumn evening. Like most of the other 6,000 churches in Britain with sets of bells, St. Michael once had its own "band "of ringers, who would herald every Sunday morning and evening service. <t>Now</t>, only one local ringer remains: 64-year-old Derek Hammond. The others here today live elsewhere.

Senses:
1. definition=in the historical present; at this point in the narration of a series of past events | examples=President Kennedy now calls in the National Guard; Washington now decides to cross the Delaware
2. definition=in these times; - Nancy Mitford | synonyms=nowadays, today | examples=it is solely by their language that the upper classes nowadays are distinguished; we now rarely see horse-drawn vehicles on city streets
3. definition=used to preface a command or reproof or request | examples=now hear this!; now pay attention
4. definition=at the present moment | synonyms=at present | examples=goods now on sale; the now-aging dictator
5. definition=without delay or hesitation; with no time intervening | synonyms=immediately, instantly, straightaway, straight off, directly, right away | examples=he answered immediately; found an answer straightaway
6. definition=(prefatory or transitional) indicates a change of subject or activity | examples=Now the next problem is...
7. definition=in the immediate past | examples=told me just now

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✓

WordNet 3.0 senses

  1. 1

    in the historical present; at this point in the narration of a series of past events

    now%4:02:05:: · now.r.01

    • President Kennedy now calls in the National Guard
    • Washington now decides to cross the Delaware
  2. 2

    in these times; - Nancy Mitford

    now%4:02:00:: · nowadays.r.01

    • it is solely by their language that the upper classes nowadays are distinguished
    • we now rarely see horse-drawn vehicles on city streets

    Synonyms: nowadays, today

    Gold Raganato Selected
  3. 3

    used to preface a command or reproof or request

    now%4:02:02:: · now.r.03

    • now hear this!
    • now pay attention
  4. 4

    at the present moment

    now%4:02:04:: · now.r.04

    • goods now on sale
    • the now-aging dictator

    Synonyms: at present

    Gold lexEN Gold Maru 2022
  5. 5

    without delay or hesitation; with no time intervening

    now%4:02:01:: · immediately.r.01

    • he answered immediately
    • found an answer straightaway

    Synonyms: immediately, instantly, straightaway, straight off, directly, right away, at once, forthwith, like a shot

  6. 6

    (prefatory or transitional) indicates a change of subject or activity

    now%4:02:07:: · now.r.06

    • Now the next problem is...
  7. 7

    in the immediate past

    now%4:02:06:: · now.r.07

    • told me just now

just Adv

Instance: senseval2.d000.s012.t005 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Like most of the other 6,000 churches in Britain with sets of bells, St. Michael once had its own "band "of ringers, who would herald every Sunday morning and evening service.

Now, only one local ringer remains: 64-year-old Derek Hammond.

The others here today live elsewhere.

They belong to a group of 15 ringers -- including two octogenarians and four youngsters in training -- who drive every Sunday from church to church in a sometimes-exhausting effort to keep the bells sounding in the many belfries of East Anglia.

"To ring for even one service at this tower, we have to scrape, "says Mr. Hammond, a retired water-authority worker. "

We've tried to train the youngsters, but they have their discos and their dances, and they just drift away. "

Mr. Hammond worries that old age and the flightiness of youth will diminish the ranks of the East Anglian group that keeps the Aslacton bells pealing.

Show raw prompt

user

Which numbered sense of "just" (marked with <t></t>) is used in the context below? Return only the number of the correct sense.

Context: Like most of the other 6,000 churches in Britain with sets of bells, St. Michael once had its own "band "of ringers, who would herald every Sunday morning and evening service. Now, only one local ringer remains: 64-year-old Derek Hammond. The others here today live elsewhere. They belong to a group of 15 ringers -- including two octogenarians and four youngsters in training -- who drive every Sunday from church to church in a sometimes-exhausting effort to keep the bells sounding in the many belfries of East Anglia. "To ring for even one service at this tower, we have to scrape, "says Mr. Hammond, a retired water-authority worker. "We've tried to train the youngsters, but they have their discos and their dances, and they <t>just</t> drift away. "Mr. Hammond worries that old age and the flightiness of youth will diminish the ranks of the East Anglian group that keeps the Aslacton bells pealing.

Senses:
1. definition=and nothing more | synonyms=merely, simply, only, but | examples=I was merely asking; it is simply a matter of time
2. definition=indicating exactness or preciseness | synonyms=precisely, exactly | examples=he was doing precisely (or exactly) what she had told him to do; it was just as he said--the jewel was gone
3. definition=only a moment ago | synonyms=just now | examples=he has just arrived; the sun just now came out
4. definition=absolutely | synonyms=simply | examples=I just can't take it anymore; he was just grand as Romeo
5. definition=only a very short time before; ; ; ; ; - W.B.Yeats | synonyms=barely, hardly, scarcely, scarce | examples=they could barely hear the speaker; we hardly knew them
6. definition=exactly at this moment or the moment described | examples=we've just finished painting the walls, so don't touch them

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    just%4:02:00:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, only, but

    Selected
  2. 2

    indicating exactness or preciseness

    just%4:02:05:: · precisely.r.01

    • he was doing precisely (or exactly) what she had told him to do
    • it was just as he said--the jewel was gone

    Synonyms: precisely, exactly

  3. 3

    only a moment ago

    just%4:02:01:: · just.r.03

    • he has just arrived
    • the sun just now came out

    Synonyms: just now

  4. 4

    absolutely

    just%4:02:03:: · just.r.04

    • I just can't take it anymore
    • he was just grand as Romeo

    Synonyms: simply

    Gold lexEN Gold Maru 2022 Gold Raganato
  5. 5

    only a very short time before; ; ; ; ; - W.B.Yeats

    just%4:02:04:: · barely.r.01

    • they could barely hear the speaker
    • we hardly knew them

    Synonyms: barely, hardly, scarcely, scarce

  6. 6

    exactly at this moment or the moment described

    just%4:02:06:: · just.r.06

    • we've just finished painting the walls, so don't touch them

only Adv

Instance: senseval2.d000.s017.t013 Dataset: senseval2 Bucket: POS / ADV

Incorrect

We've tried to train the youngsters, but they have their discos and their dances, and they just drift away. "

Mr. Hammond worries that old age and the flightiness of youth will diminish the ranks of the East Anglian group that keeps the Aslacton bells pealing.

History, after all, is not on his side.

According to a nationwide survey taken a year ago, nearly a third of England's church bells are no longer rung on Sundays because there is no one to ring them.

It is easy to see why the ancient art is on the ropes.

The less complicated version of playing tunes on bells, as do the carillons of continental Europe, is considered by the English to be childish, fit only for foreigners.

Change-ringing, a mind-boggling exercise the English invented 380 years ago, requires physical dexterity -- some bells weigh more than a ton -- combined with intense mental concentration.

Show raw prompt

user

Which numbered sense of "only" (marked with <t></t>) is used in the context below? Return only the number of the correct sense.

Context: We've tried to train the youngsters, but they have their discos and their dances, and they just drift away. "Mr. Hammond worries that old age and the flightiness of youth will diminish the ranks of the East Anglian group that keeps the Aslacton bells pealing. History, after all, is not on his side. According to a nationwide survey taken a year ago, nearly a third of England's church bells are no longer rung on Sundays because there is no one to ring them. It is easy to see why the ancient art is on the ropes. The less complicated version of playing tunes on bells, as do the carillons of continental Europe, is considered by the English to be childish, fit <t>only</t> for foreigners. Change-ringing, a mind-boggling exercise the English invented 380 years ago, requires physical dexterity -- some bells weigh more than a ton -- combined with intense mental concentration.

Senses:
1. definition=and nothing more | synonyms=merely, simply, just, but | examples=I was merely asking; it is simply a matter of time
2. definition=without any others being included or involved | synonyms=entirely, exclusively, solely, alone | examples=was entirely to blame; a school devoted entirely to the needs of problem children
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead; We won only to lose again in the next round
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | synonyms=only if, only when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    only%4:02:02:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, just, but

    Selected
  2. 2

    without any others being included or involved

    only%4:02:01:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, alone

    Gold lexEN Gold Maru 2022 Gold Raganato
  3. 3

    with nevertheless the final result

    only%4:02:00:: · only.r.03

    • He arrived only to find his wife dead
    • We won only to lose again in the next round
  4. 4

    in the final outcome

    only%4:02:04:: · only.r.04

    • These news will only make you more upset
  5. 5

    except that

    only%4:02:06:: · only.r.05

    • It was the same story; only this time she came out better
  6. 6

    never except when

    only%4:02:03:: · only.r.06

    • call me only if your cold gets worse

    Synonyms: only if, only when

  7. 7

    as recently as

    only%4:02:05:: · only.r.07

    • I spoke to him only an hour ago

only Adv

Instance: senseval2.d000.s021.t003 Dataset: senseval2 Bucket: POS / ADV

Incorrect

It is easy to see why the ancient art is on the ropes.

The less complicated version of playing tunes on bells, as do the carillons of continental Europe, is considered by the English to be childish, fit only for foreigners.

Change-ringing, a mind-boggling exercise the English invented 380 years ago, requires physical dexterity -- some bells weigh more than a ton -- combined with intense mental concentration.

Proper English bells are started off in "rounds, "from the highest-pitched bell to the lowest -- a simple descending scale using, in larger churches, as many as 12 bells.

Then, at a signal, the ringers begin varying the order in which the bells sound without altering the steady rhythm of the striking.

Each variation, or change, can occur only once, the rules state.

Ringers memorize patterns of changes, known as "methods, "which have odd-sounding names like Kent Treble Bob Major or Grandsire Caters.

Show raw prompt

user

Which numbered sense of "only" (marked with <t></t>) is used in the context below? Return only the number of the correct sense.

Context: It is easy to see why the ancient art is on the ropes. The less complicated version of playing tunes on bells, as do the carillons of continental Europe, is considered by the English to be childish, fit only for foreigners. Change-ringing, a mind-boggling exercise the English invented 380 years ago, requires physical dexterity -- some bells weigh more than a ton -- combined with intense mental concentration. Proper English bells are started off in "rounds, "from the highest-pitched bell to the lowest -- a simple descending scale using, in larger churches, as many as 12 bells. Then, at a signal, the ringers begin varying the order in which the bells sound without altering the steady rhythm of the striking. Each variation, or change, can occur <t>only</t> once, the rules state. Ringers memorize patterns of changes, known as "methods, "which have odd-sounding names like Kent Treble Bob Major or Grandsire Caters.

Senses:
1. definition=and nothing more | synonyms=merely, simply, just, but | examples=I was merely asking; it is simply a matter of time
2. definition=without any others being included or involved | synonyms=entirely, exclusively, solely, alone | examples=was entirely to blame; a school devoted entirely to the needs of problem children
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead; We won only to lose again in the next round
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | synonyms=only if, only when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    only%4:02:02:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, just, but

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    without any others being included or involved

    only%4:02:01:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, alone

    Selected
  3. 3

    with nevertheless the final result

    only%4:02:00:: · only.r.03

    • He arrived only to find his wife dead
    • We won only to lose again in the next round
  4. 4

    in the final outcome

    only%4:02:04:: · only.r.04

    • These news will only make you more upset
  5. 5

    except that

    only%4:02:06:: · only.r.05

    • It was the same story; only this time she came out better
  6. 6

    never except when

    only%4:02:03:: · only.r.06

    • call me only if your cold gets worse

    Synonyms: only if, only when

  7. 7

    as recently as

    only%4:02:05:: · only.r.07

    • I spoke to him only an hour ago

however Adv

Instance: senseval2.d000.s033.t004 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Far above in the belfry, the huge bronze bells, mounted on wheels, swing madly through a full 360 degrees, starting and ending, surprisingly, in the inverted, or mouth-up position.

Skilled ringers use their wrists to advance or retard the next swing, so that one bell can swap places with another in the following change.

In a well-known detective-story involving church bells, English novelist Dorothy L. Sayers described ringing as a "passion that finds its satisfaction in mathematical completeness and mechanical perfection. "

Ringers, she added, are "filled with the solemn intoxication that comes of intricate ritual faultlessly performed. ""

Ringing does become a bit of an obsession, "admits Stephanie Pattenden, master of the band at St. Mary Abbot and one of England's best female ringers.

It is a passion that usually stays in the tower, however.

More often than not, ringers think of the church as something stuck on the bottom of the belfry.

Show raw prompt

user

Which numbered sense of "however" (marked with <t></t>) is used in the context below? Return only the number of the correct sense.

Context: Far above in the belfry, the huge bronze bells, mounted on wheels, swing madly through a full 360 degrees, starting and ending, surprisingly, in the inverted, or mouth-up position. Skilled ringers use their wrists to advance or retard the next swing, so that one bell can swap places with another in the following change. In a well-known detective-story involving church bells, English novelist Dorothy L. Sayers described ringing as a "passion that finds its satisfaction in mathematical completeness and mechanical perfection. "Ringers, she added, are "filled with the solemn intoxication that comes of intricate ritual faultlessly performed. ""Ringing does become a bit of an obsession, "admits Stephanie Pattenden, master of the band at St. Mary Abbot and one of England's best female ringers. It is a passion that usually stays in the tower, <t>however</t>. More often than not, ringers think of the church as something stuck on the bottom of the belfry.

Senses:
1. definition=despite anything to the contrary (usually following a concession) | synonyms=nevertheless, withal, still, yet, all the same, even so | examples=although I'm a little afraid, however I'd like to try it; while we disliked each other, nevertheless we agreed
2. definition=by contrast; on the other hand | examples=the first part was easy; the second, however, took hours
3. definition=to whatever degree or extent | examples=The results, however general, are important; they have begun, however reluctantly, to acknowledge the legitimacy of some of the opposition's concerns
4. definition=in whatever way or manner | examples=Victory, however it was brought about, was sweet; however he did it, it was very clever

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    despite anything to the contrary (usually following a concession)

    however%4:02:00:: · however.r.01

    • although I'm a little afraid, however I'd like to try it
    • while we disliked each other, nevertheless we agreed

    Synonyms: nevertheless, withal, still, yet, all the same, even so, nonetheless, notwithstanding

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    by contrast; on the other hand

    however%4:02:04:: · however.r.02

    • the first part was easy; the second, however, took hours
    Selected
  3. 3

    to whatever degree or extent

    however%4:02:02:: · however.r.03

    • The results, however general, are important
    • they have begun, however reluctantly, to acknowledge the legitimacy of some of the opposition's concerns
  4. 4

    in whatever way or manner

    however%4:02:01:: · however.r.04

    • Victory, however it was brought about, was sweet
    • however he did it, it was very clever

more Adv

Instance: senseval2.d000.s034.t000 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Skilled ringers use their wrists to advance or retard the next swing, so that one bell can swap places with another in the following change.

In a well-known detective-story involving church bells, English novelist Dorothy L. Sayers described ringing as a "passion that finds its satisfaction in mathematical completeness and mechanical perfection. "

Ringers, she added, are "filled with the solemn intoxication that comes of intricate ritual faultlessly performed. ""

Ringing does become a bit of an obsession, "admits Stephanie Pattenden, master of the band at St. Mary Abbot and one of England's best female ringers.

It is a passion that usually stays in the tower, however.

More often than not, ringers think of the church as something stuck on the bottom of the belfry.

When their changes are completed, and after they have worked up a sweat, ringers often skip off to the local pub, leaving worship for others below.

Show raw prompt

user

Which numbered sense of "more" (marked with <t></t>) is used in the context below? Return only the number of the correct sense.

Context: Skilled ringers use their wrists to advance or retard the next swing, so that one bell can swap places with another in the following change. In a well-known detective-story involving church bells, English novelist Dorothy L. Sayers described ringing as a "passion that finds its satisfaction in mathematical completeness and mechanical perfection. "Ringers, she added, are "filled with the solemn intoxication that comes of intricate ritual faultlessly performed. ""Ringing does become a bit of an obsession, "admits Stephanie Pattenden, master of the band at St. Mary Abbot and one of England's best female ringers. It is a passion that usually stays in the tower, however. <t>More</t> often than not, ringers think of the church as something stuck on the bottom of the belfry. When their changes are completed, and after they have worked up a sweat, ringers often skip off to the local pub, leaving worship for others below.

Senses:
1. definition=used to form the comparative of some adjectives and adverbs | synonyms=to a greater extent | examples=more interesting; more beautiful
2. definition=comparative of much; to a greater degree or extent | examples=he works more now; they eat more than they should

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    used to form the comparative of some adjectives and adverbs

    more%4:02:00:: · more.r.01

    • more interesting
    • more beautiful

    Synonyms: to a greater extent

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    comparative of much; to a greater degree or extent

    more%4:02:01:: · more.r.02

    • he works more now
    • they eat more than they should
    Selected

below Adv

Instance: senseval2.d000.s035.t011 Dataset: senseval2 Bucket: POS / ADV

Incorrect

In a well-known detective-story involving church bells, English novelist Dorothy L. Sayers described ringing as a "passion that finds its satisfaction in mathematical completeness and mechanical perfection. "

Ringers, she added, are "filled with the solemn intoxication that comes of intricate ritual faultlessly performed. ""

Ringing does become a bit of an obsession, "admits Stephanie Pattenden, master of the band at St. Mary Abbot and one of England's best female ringers.

It is a passion that usually stays in the tower, however.

More often than not, ringers think of the church as something stuck on the bottom of the belfry.

When their changes are completed, and after they have worked up a sweat, ringers often skip off to the local pub, leaving worship for others below.

This does not sit well with some clerics.

Show raw prompt

user

Which numbered sense of "below" (marked with <t></t>) is used in the context below? Return only the number of the correct sense.

Context: In a well-known detective-story involving church bells, English novelist Dorothy L. Sayers described ringing as a "passion that finds its satisfaction in mathematical completeness and mechanical perfection. "Ringers, she added, are "filled with the solemn intoxication that comes of intricate ritual faultlessly performed. ""Ringing does become a bit of an obsession, "admits Stephanie Pattenden, master of the band at St. Mary Abbot and one of England's best female ringers. It is a passion that usually stays in the tower, however. More often than not, ringers think of the church as something stuck on the bottom of the belfry. When their changes are completed, and after they have worked up a sweat, ringers often skip off to the local pub, leaving worship for others <t>below</t>. This does not sit well with some clerics.

Senses:
1. definition=in or to a place that is lower | synonyms=at a lower place, to a lower place, beneath
2. definition=at a later place | examples=see below
3. definition=(in writing) see below | synonyms=infra | examples=vide infra
4. definition=on a floor below | synonyms=downstairs, down the stairs, on a lower floor | examples=the tenants live downstairs
5. definition=further down | synonyms=under | examples=see under for further discussion

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    in or to a place that is lower

    below%4:02:01:: · below.r.01

    Synonyms: at a lower place, to a lower place, beneath

    Selected
  2. 2

    at a later place

    below%4:02:00:: · below.r.02

    • see below
  3. 3

    (in writing) see below

    below%4:02:02:: · below.r.03

    • vide infra

    Synonyms: infra

  4. 4

    on a floor below

    below%4:02:06:: · downstairs.r.01

    • the tenants live downstairs

    Synonyms: downstairs, down the stairs, on a lower floor

    Gold lexEN Gold Maru 2022 Gold Raganato
  5. 5

    further down

    below%4:02:03:: · under.r.08

    • see under for further discussion

    Synonyms: under