Verified run

claude-opus-4-8

claude-opus-4.8-p001-lexen-v1-20260617

Rank
21
Accuracy lexEN v1 · WordNet fine
93.85%
Correct
4562 / 4861
Actual run cost
$23.50
Cost / M items
$4,834

Accuracy across label schemes

The headline accuracy is lexEN v1 · WordNet fine-grained. Each cell re-scores this run's predictions against a different gold label set and sense granularity; coarse counts a prediction correct when its WordNet sense maps to the same Glite or CSI concept as a gold sense. About these label schemes → About coarsening →

Score against WordNet fine-grainedGlite coarse-grainedCSI coarse-grained (Lacerra 2020)
lexEN v1 93.85% 4562 / 4861 98.21% 4774 / 4861 96.17% 4675 / 4861
Maru 2022 (ALLamended) 90.58% 4403 / 4861 97.20% 4725 / 4861 94.30% 4584 / 4861
Raganato 2017 (original) 84.53% 4109 / 4861 94.47% 4592 / 4861 90.35% 4392 / 4861

Model

Display
claude-opus-4-8
Requested
claude-opus-4-8
Resolved
claude-opus-4-8
Vendor
Anthropic
API provider
Anthropic
Hosting
cloud_api
Source
proprietary
License
n/a
Reasoning
n/a
Model URL
n/a
Endpoint
n/a
Resolved counts
claude-opus-4-8: 4861

Reproducibility

Created
2026-06-17T11:56:28.079841+00:00
Run schema
sensebench-run-v2
Commit
85eaf1d73ac85fdd11bde5db99f17959abea2966
Dataset ID
lexen
Dataset version
lexen-v1
Dataset revision
n/a
Dataset items
4,861
Dataset hash
sha256:5fd4382b93f19087a1e31f6dd7d1db17c1eb17ff80fcbe1d3fdd55c0c3ecefe8
Prompt
p001
Prompt version
0.1.0
Repository files
Browse in GitHub

Runtime

Calls
4861
Total tokens
4,465,940
Tokens / item
918.7
Total cost
$23.50
Cost / M items
$4,834
Cost source
litellm_estimate
Elapsed
18.90m
Success items
4,861
Monosemous
0
No candidates
0
No valid vote
0
Invalid votes
0
Transport errors
0

Runner

GitHub
vassiliphilippov
Name
Vassili Philippov
Contact
n/a

Sampling

Temperature
n/a
Top P
n/a
Max tokens
8,192
Seed
n/a
Extra
n/a

Policy

Votes / item
1
Reasks / invalid vote
1
Tie break
earliest_vote
Monosemous
short_circuit

Cost And Token Breakdown

Actual recorded run cost: $23.50. Cost estimate source: litellm_estimate.

Token Type Tokens Cost Price / 1M tokens
Input uncached 4,407,608 $22.04 $5.00
Input cached 0 $0.00 $0.500
Output 58,332 $1.46 $25.00
Reasoning output 0 included in output cost n/a

Reasoning output tokens are shown separately when the provider reports them; recorded cost is stored in the output cost bucket.

Download Raw Run Files

Slice Breakdown

Group Bucket Accuracy Correct Items
Candidate Count 11+ 92.44% 734 794
Candidate Count 2 97.43% 833 855
Candidate Count 3-5 93.16% 1812 1945
Candidate Count 6-10 93.37% 1183 1267
POS ADJ 94.08% 620 659
POS ADV 91.79% 179 195
POS NOUN 94.37% 2734 2897
POS VERB 92.70% 1029 1110
Source Dataset semeval2013 95.44% 1088 1140
Source Dataset semeval2015 93.64% 692 739
Source Dataset senseval2 93.98% 1531 1629
Source Dataset senseval3 92.46% 1251 1353

Worst-Bucket Examples

only Adv

Instance: semeval2015.d000.s013.t000 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

Alimta is used to treat two types of cancer: pleural mesothelioma (cancer of the lining of the lungs that is usually caused by exposure to asbestos).

Alimta is used together with cisplatin (another anticancer medicine) when the cancer is unresectable (cannot be removed by surgery alone) and malignant (has spread, or is likely to spread easily, to other parts of the body), in patients who have not received chemotherapy (medicines for cancer) before advanced or metastatic non-small cell lung cancer that is not affecting the squamous cells.

Alimta is used in combination with cisplatin in patients who have not been treated before, and on its own in patients who have received another type of chemotherapy in the past.

The medicine can only be obtained with a prescription.

How is Alimta used?

Alimta should only be given under the supervision of a doctor who is qualified in the use of anticancer chemotherapy.

The recommended dose of Alimta is 500 mg per square metre of body surface area (calculated using the patient s height and weight).

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: only
Target surface form: only
Context:
Alimta is used to treat two types of cancer: pleural mesothelioma (cancer of the lining of the lungs that is usually caused by exposure to asbestos). Alimta is used together with cisplatin (another anticancer medicine) when the cancer is unresectable (cannot be removed by surgery alone) and malignant (has spread, or is likely to spread easily, to other parts of the body), in patients who have not received chemotherapy (medicines for cancer) before advanced or metastatic non-small cell lung cancer that is not affecting the squamous cells. Alimta is used in combination with cisplatin in patients who have not been treated before, and on its own in patients who have received another type of chemotherapy in the past. The medicine can only be obtained with a prescription. How is Alimta used? Alimta should <t>only</t> be given under the supervision of a doctor who is qualified in the use of anticancer chemotherapy. The recommended dose of Alimta is 500 mg per square metre of body surface area (calculated using the patient s height and weight).

Candidate senses:
1. sense_key=only%4:02:02:: | definition=and nothing more | synonyms=merely, simply, just, but | examples=I was merely asking; it is simply a matter of time
2. sense_key=only%4:02:01:: | definition=without any others being included or involved | synonyms=entirely, exclusively, solely, alone | examples=was entirely to blame; a school devoted entirely to the needs of problem children
3. sense_key=only%4:02:00:: | definition=with nevertheless the final result | examples=He arrived only to find his wife dead; We won only to lose again in the next round
4. sense_key=only%4:02:04:: | definition=in the final outcome | examples=These news will only make you more upset
5. sense_key=only%4:02:06:: | definition=except that | examples=It was the same story; only this time she came out better
6. sense_key=only%4:02:03:: | definition=never except when | synonyms=only if, only when | examples=call me only if your cold gets worse
7. sense_key=only%4:02:05:: | definition=as recently as | examples=I spoke to him only an hour ago

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    only%4:02:02:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, just, but

  2. 2

    without any others being included or involved

    only%4:02:01:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, alone

    Selected
  3. 3

    with nevertheless the final result

    only%4:02:00:: · only.r.03

    • He arrived only to find his wife dead
    • We won only to lose again in the next round
  4. 4

    in the final outcome

    only%4:02:04:: · only.r.04

    • These news will only make you more upset
  5. 5

    except that

    only%4:02:06:: · only.r.05

    • It was the same story; only this time she came out better
  6. 6

    never except when

    only%4:02:03:: · only.r.06

    • call me only if your cold gets worse

    Synonyms: only if, only when

    Gold lexEN Gold Maru 2022 Gold Raganato
  7. 7

    as recently as

    only%4:02:05:: · only.r.07

    • I spoke to him only an hour ago

actually Adv

Instance: semeval2015.d001.s002.t000 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

kalgebra is a mathematical calculator based on content markup MathML language.

Nowadays it is capable to make simple MathML operations (arithmetic and logical) and to representate 2D and 3D graphs.

It is actually not necessary to know MathML to use kalgebra.

kalgebra is a MathML based graph calculator.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: actually
Target surface form: actually
Context:
kalgebra is a mathematical calculator based on content markup MathML language. Nowadays it is capable to make simple MathML operations (arithmetic and logical) and to representate 2D and 3D graphs. It is <t>actually</t> not necessary to know MathML to use kalgebra. kalgebra is a MathML based graph calculator.

Candidate senses:
1. sense_key=actually%4:02:01:: | definition=in actual fact | synonyms=really | examples=to be nominally but not actually independent; no one actually saw the shark
2. sense_key=actually%4:02:00:: | definition=used to imply that one would expect the fact to be the opposite of that stated; surprisingly | synonyms=in reality | examples=you may actually be doing the right thing by walking out; she actually spoke Latin
3. sense_key=actually%4:02:03:: | definition=at the present moment | examples=the transmission screen shows the picture that is actually on the air
4. sense_key=actually%4:02:02:: | definition=as a sentence modifier to add slight emphasis | examples=actually, we all help clear up after a meal; actually, I haven't seen the film

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    in actual fact

    actually%4:02:01:: · actually.r.01

    • to be nominally but not actually independent
    • no one actually saw the shark

    Synonyms: really

    Selected
  2. 2

    used to imply that one would expect the fact to be the opposite of that stated; surprisingly

    actually%4:02:00:: · actually.r.02

    • you may actually be doing the right thing by walking out
    • she actually spoke Latin

    Synonyms: in reality

    Gold lexEN Gold Maru 2022 Gold Raganato
  3. 3

    at the present moment

    actually%4:02:03:: · actually.r.03

    • the transmission screen shows the picture that is actually on the air
  4. 4

    as a sentence modifier to add slight emphasis

    actually%4:02:02:: · actually.r.04

    • actually, we all help clear up after a meal
    • actually, I haven't seen the film
    Gold lexEN Gold Maru 2022 Gold Raganato

here Adv

Instance: semeval2015.d001.s005.t000 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

kalgebra is a mathematical calculator based on content markup MathML language.

Nowadays it is capable to make simple MathML operations (arithmetic and logical) and to representate 2D and 3D graphs.

It is actually not necessary to know MathML to use kalgebra.

kalgebra is a MathML based graph calculator.

Although it was initially MathML oriented it can now be used by everyone with little mathematical knowledge.

Here's a screenshot of kalgebra main window

kalgebra main window

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: here
Target surface form: Here
Context:
kalgebra is a mathematical calculator based on content markup MathML language. Nowadays it is capable to make simple MathML operations (arithmetic and logical) and to representate 2D and 3D graphs. It is actually not necessary to know MathML to use kalgebra. kalgebra is a MathML based graph calculator. Although it was initially MathML oriented it can now be used by everyone with little mathematical knowledge. <t>Here</t>'s a screenshot of kalgebra main window kalgebra main window

Candidate senses:
1. sense_key=here%4:02:00:: | definition=in or at this place; where the speaker or writer is | examples=I work here; turn here
2. sense_key=here%4:02:02:: | definition=in this circumstance or respect or on this point or detail | examples=what do we have here?; here I must disagree
3. sense_key=here%4:02:01:: | definition=to this place (especially toward the speaker) | synonyms=hither | examples=come here, please
4. sense_key=here%4:02:04:: | definition=at this time; now | examples=we'll adjourn here for lunch and discuss the remaining issues this afternoon

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    in or at this place; where the speaker or writer is

    here%4:02:00:: · here.r.01

    • I work here
    • turn here
    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    in this circumstance or respect or on this point or detail

    here%4:02:02:: · here.r.02

    • what do we have here?
    • here I must disagree
    Selected
  3. 3

    to this place (especially toward the speaker)

    here%4:02:01:: · here.r.03

    • come here, please

    Synonyms: hither

  4. 4

    at this time; now

    here%4:02:04:: · here.r.04

    • we'll adjourn here for lunch and discuss the remaining issues this afternoon

only Adv

Instance: senseval2.d000.s008.t001 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Of all scenes that evoke rural England, this is one of the loveliest: An ancient stone church stands amid the fields, the sound of bells cascading from its tower, calling the faithful to evensong.

The parishioners of St. Michael and All Angels stop to chat at the church door, as members here always have.

In the tower, five men and women pull rhythmically on ropes attached to the same five bells that first sounded here in 1614.

But there is also a discordant, modern note in Aslacton, though it can not be heard by the church-goers enjoying the peal of bells this cool autumn evening.

Like most of the other 6,000 churches in Britain with sets of bells, St. Michael once had its own "band "of ringers, who would herald every Sunday morning and evening service.

Now, only one local ringer remains: 64-year-old Derek Hammond.

The others here today live elsewhere.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: only
Target surface form: only
Context:
Of all scenes that evoke rural England, this is one of the loveliest: An ancient stone church stands amid the fields, the sound of bells cascading from its tower, calling the faithful to evensong. The parishioners of St. Michael and All Angels stop to chat at the church door, as members here always have. In the tower, five men and women pull rhythmically on ropes attached to the same five bells that first sounded here in 1614. But there is also a discordant, modern note in Aslacton, though it can not be heard by the church-goers enjoying the peal of bells this cool autumn evening. Like most of the other 6,000 churches in Britain with sets of bells, St. Michael once had its own "band "of ringers, who would herald every Sunday morning and evening service. Now, <t>only</t> one local ringer remains: 64-year-old Derek Hammond. The others here today live elsewhere.

Candidate senses:
1. sense_key=only%4:02:02:: | definition=and nothing more | synonyms=merely, simply, just, but | examples=I was merely asking; it is simply a matter of time
2. sense_key=only%4:02:01:: | definition=without any others being included or involved | synonyms=entirely, exclusively, solely, alone | examples=was entirely to blame; a school devoted entirely to the needs of problem children
3. sense_key=only%4:02:00:: | definition=with nevertheless the final result | examples=He arrived only to find his wife dead; We won only to lose again in the next round
4. sense_key=only%4:02:04:: | definition=in the final outcome | examples=These news will only make you more upset
5. sense_key=only%4:02:06:: | definition=except that | examples=It was the same story; only this time she came out better
6. sense_key=only%4:02:03:: | definition=never except when | synonyms=only if, only when | examples=call me only if your cold gets worse
7. sense_key=only%4:02:05:: | definition=as recently as | examples=I spoke to him only an hour ago

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    only%4:02:02:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, just, but

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    without any others being included or involved

    only%4:02:01:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, alone

    Selected
  3. 3

    with nevertheless the final result

    only%4:02:00:: · only.r.03

    • He arrived only to find his wife dead
    • We won only to lose again in the next round
  4. 4

    in the final outcome

    only%4:02:04:: · only.r.04

    • These news will only make you more upset
  5. 5

    except that

    only%4:02:06:: · only.r.05

    • It was the same story; only this time she came out better
  6. 6

    never except when

    only%4:02:03:: · only.r.06

    • call me only if your cold gets worse

    Synonyms: only if, only when

  7. 7

    as recently as

    only%4:02:05:: · only.r.07

    • I spoke to him only an hour ago

just Adv

Instance: senseval2.d000.s012.t005 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Like most of the other 6,000 churches in Britain with sets of bells, St. Michael once had its own "band "of ringers, who would herald every Sunday morning and evening service.

Now, only one local ringer remains: 64-year-old Derek Hammond.

The others here today live elsewhere.

They belong to a group of 15 ringers -- including two octogenarians and four youngsters in training -- who drive every Sunday from church to church in a sometimes-exhausting effort to keep the bells sounding in the many belfries of East Anglia.

"To ring for even one service at this tower, we have to scrape, "says Mr. Hammond, a retired water-authority worker. "

We've tried to train the youngsters, but they have their discos and their dances, and they just drift away. "

Mr. Hammond worries that old age and the flightiness of youth will diminish the ranks of the East Anglian group that keeps the Aslacton bells pealing.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: just
Target surface form: just
Context:
Like most of the other 6,000 churches in Britain with sets of bells, St. Michael once had its own "band "of ringers, who would herald every Sunday morning and evening service. Now, only one local ringer remains: 64-year-old Derek Hammond. The others here today live elsewhere. They belong to a group of 15 ringers -- including two octogenarians and four youngsters in training -- who drive every Sunday from church to church in a sometimes-exhausting effort to keep the bells sounding in the many belfries of East Anglia. "To ring for even one service at this tower, we have to scrape, "says Mr. Hammond, a retired water-authority worker. "We've tried to train the youngsters, but they have their discos and their dances, and they <t>just</t> drift away. "Mr. Hammond worries that old age and the flightiness of youth will diminish the ranks of the East Anglian group that keeps the Aslacton bells pealing.

Candidate senses:
1. sense_key=just%4:02:00:: | definition=and nothing more | synonyms=merely, simply, only, but | examples=I was merely asking; it is simply a matter of time
2. sense_key=just%4:02:05:: | definition=indicating exactness or preciseness | synonyms=precisely, exactly | examples=he was doing precisely (or exactly) what she had told him to do; it was just as he said--the jewel was gone
3. sense_key=just%4:02:01:: | definition=only a moment ago | synonyms=just now | examples=he has just arrived; the sun just now came out
4. sense_key=just%4:02:03:: | definition=absolutely | synonyms=simply | examples=I just can't take it anymore; he was just grand as Romeo
5. sense_key=just%4:02:04:: | definition=only a very short time before; ; ; ; ; - W.B.Yeats | synonyms=barely, hardly, scarcely, scarce | examples=they could barely hear the speaker; we hardly knew them
6. sense_key=just%4:02:06:: | definition=exactly at this moment or the moment described | examples=we've just finished painting the walls, so don't touch them

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    just%4:02:00:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, only, but

    Selected
  2. 2

    indicating exactness or preciseness

    just%4:02:05:: · precisely.r.01

    • he was doing precisely (or exactly) what she had told him to do
    • it was just as he said--the jewel was gone

    Synonyms: precisely, exactly

  3. 3

    only a moment ago

    just%4:02:01:: · just.r.03

    • he has just arrived
    • the sun just now came out

    Synonyms: just now

  4. 4

    absolutely

    just%4:02:03:: · just.r.04

    • I just can't take it anymore
    • he was just grand as Romeo

    Synonyms: simply

    Gold lexEN Gold Maru 2022 Gold Raganato
  5. 5

    only a very short time before; ; ; ; ; - W.B.Yeats

    just%4:02:04:: · barely.r.01

    • they could barely hear the speaker
    • we hardly knew them

    Synonyms: barely, hardly, scarcely, scarce

  6. 6

    exactly at this moment or the moment described

    just%4:02:06:: · just.r.06

    • we've just finished painting the walls, so don't touch them

straight Adv

Instance: senseval2.d000.s027.t004 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Ringers memorize patterns of changes, known as "methods, "which have odd-sounding names like Kent Treble Bob Major or Grandsire Caters.

A series of 5,000 or so changes is a "peal "and takes about three hours.

A look at a Thursday night practice at St. Mary Abbot church in the Kensington district of London gives an idea of the work involved.

Ten shirt-sleeved ringers stand in a circle, one foot ahead of the other in a prize-fighter's stance, each pulling a rope that disappears through a small hole in the high ceiling of the ringing chamber.

No one speaks, and the snaking of the ropes seems to make as much sound as the bells themselves, muffled by the ceiling.

Totally absorbed, the ringers stare straight ahead, using peripheral vision (they call it "rope-sight ") to watch the other ropes and thus time their pulls.

Far above in the belfry, the huge bronze bells, mounted on wheels, swing madly through a full 360 degrees, starting and ending, surprisingly, in the inverted, or mouth-up position.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: straight
Target surface form: straight
Context:
Ringers memorize patterns of changes, known as "methods, "which have odd-sounding names like Kent Treble Bob Major or Grandsire Caters. A series of 5,000 or so changes is a "peal "and takes about three hours. A look at a Thursday night practice at St. Mary Abbot church in the Kensington district of London gives an idea of the work involved. Ten shirt-sleeved ringers stand in a circle, one foot ahead of the other in a prize-fighter's stance, each pulling a rope that disappears through a small hole in the high ceiling of the ringing chamber. No one speaks, and the snaking of the ropes seems to make as much sound as the bells themselves, muffled by the ceiling. Totally absorbed, the ringers stare <t>straight</t> ahead, using peripheral vision (they call it "rope-sight ") to watch the other ropes and thus time their pulls. Far above in the belfry, the huge bronze bells, mounted on wheels, swing madly through a full 360 degrees, starting and ending, surprisingly, in the inverted, or mouth-up position.

Candidate senses:
1. sense_key=straight%4:02:05:: | definition=without deviation | synonyms=directly, direct | examples=the path leads directly to the lake; went direct to the office
2. sense_key=straight%4:02:04:: | definition=in a forthright manner; candidly or frankly | synonyms=directly, flat | examples=he didn't answer directly; told me straight out
3. sense_key=straight%4:02:00:: | definition=in a straight line; in a direct course | examples=the road runs straight

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✓ Raganato ✗

WordNet 3.0 senses

  1. 1

    without deviation

    straight%4:02:05:: · directly.r.01

    • the path leads directly to the lake
    • went direct to the office

    Synonyms: directly, direct

    Gold lexEN Gold Raganato
  2. 2

    in a forthright manner; candidly or frankly

    straight%4:02:04:: · directly.r.04

    • he didn't answer directly
    • told me straight out

    Synonyms: directly, flat

  3. 3

    in a straight line; in a direct course

    straight%4:02:00:: · straight.r.03

    • the road runs straight
    Gold Maru 2022 Selected

more Adv

Instance: senseval2.d000.s034.t000 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Skilled ringers use their wrists to advance or retard the next swing, so that one bell can swap places with another in the following change.

In a well-known detective-story involving church bells, English novelist Dorothy L. Sayers described ringing as a "passion that finds its satisfaction in mathematical completeness and mechanical perfection. "

Ringers, she added, are "filled with the solemn intoxication that comes of intricate ritual faultlessly performed. ""

Ringing does become a bit of an obsession, "admits Stephanie Pattenden, master of the band at St. Mary Abbot and one of England's best female ringers.

It is a passion that usually stays in the tower, however.

More often than not, ringers think of the church as something stuck on the bottom of the belfry.

When their changes are completed, and after they have worked up a sweat, ringers often skip off to the local pub, leaving worship for others below.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: more
Target surface form: More
Context:
Skilled ringers use their wrists to advance or retard the next swing, so that one bell can swap places with another in the following change. In a well-known detective-story involving church bells, English novelist Dorothy L. Sayers described ringing as a "passion that finds its satisfaction in mathematical completeness and mechanical perfection. "Ringers, she added, are "filled with the solemn intoxication that comes of intricate ritual faultlessly performed. ""Ringing does become a bit of an obsession, "admits Stephanie Pattenden, master of the band at St. Mary Abbot and one of England's best female ringers. It is a passion that usually stays in the tower, however. <t>More</t> often than not, ringers think of the church as something stuck on the bottom of the belfry. When their changes are completed, and after they have worked up a sweat, ringers often skip off to the local pub, leaving worship for others below.

Candidate senses:
1. sense_key=more%4:02:00:: | definition=used to form the comparative of some adjectives and adverbs | synonyms=to a greater extent | examples=more interesting; more beautiful
2. sense_key=more%4:02:01:: | definition=comparative of much; to a greater degree or extent | examples=he works more now; they eat more than they should

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    used to form the comparative of some adjectives and adverbs

    more%4:02:00:: · more.r.01

    • more interesting
    • more beautiful

    Synonyms: to a greater extent

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    comparative of much; to a greater degree or extent

    more%4:02:01:: · more.r.02

    • he works more now
    • they eat more than they should
    Selected

below Adv

Instance: senseval2.d000.s035.t011 Dataset: senseval2 Bucket: POS / ADV

Incorrect

In a well-known detective-story involving church bells, English novelist Dorothy L. Sayers described ringing as a "passion that finds its satisfaction in mathematical completeness and mechanical perfection. "

Ringers, she added, are "filled with the solemn intoxication that comes of intricate ritual faultlessly performed. ""

Ringing does become a bit of an obsession, "admits Stephanie Pattenden, master of the band at St. Mary Abbot and one of England's best female ringers.

It is a passion that usually stays in the tower, however.

More often than not, ringers think of the church as something stuck on the bottom of the belfry.

When their changes are completed, and after they have worked up a sweat, ringers often skip off to the local pub, leaving worship for others below.

This does not sit well with some clerics.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: below
Target surface form: below
Context:
In a well-known detective-story involving church bells, English novelist Dorothy L. Sayers described ringing as a "passion that finds its satisfaction in mathematical completeness and mechanical perfection. "Ringers, she added, are "filled with the solemn intoxication that comes of intricate ritual faultlessly performed. ""Ringing does become a bit of an obsession, "admits Stephanie Pattenden, master of the band at St. Mary Abbot and one of England's best female ringers. It is a passion that usually stays in the tower, however. More often than not, ringers think of the church as something stuck on the bottom of the belfry. When their changes are completed, and after they have worked up a sweat, ringers often skip off to the local pub, leaving worship for others <t>below</t>. This does not sit well with some clerics.

Candidate senses:
1. sense_key=below%4:02:01:: | definition=in or to a place that is lower | synonyms=at a lower place, to a lower place, beneath
2. sense_key=below%4:02:00:: | definition=at a later place | examples=see below
3. sense_key=below%4:02:02:: | definition=(in writing) see below | synonyms=infra | examples=vide infra
4. sense_key=below%4:02:06:: | definition=on a floor below | synonyms=downstairs, down the stairs, on a lower floor | examples=the tenants live downstairs
5. sense_key=below%4:02:03:: | definition=further down | synonyms=under | examples=see under for further discussion

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    in or to a place that is lower

    below%4:02:01:: · below.r.01

    Synonyms: at a lower place, to a lower place, beneath

    Selected
  2. 2

    at a later place

    below%4:02:00:: · below.r.02

    • see below
  3. 3

    (in writing) see below

    below%4:02:02:: · below.r.03

    • vide infra

    Synonyms: infra

  4. 4

    on a floor below

    below%4:02:06:: · downstairs.r.01

    • the tenants live downstairs

    Synonyms: downstairs, down the stairs, on a lower floor

    Gold lexEN Gold Maru 2022 Gold Raganato
  5. 5

    further down

    below%4:02:03:: · under.r.08

    • see under for further discussion

    Synonyms: under

right Adv

Instance: senseval2.d000.s056.t000 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Says Mr. Baldwin, "We recognize that we may no longer have as high a priority in church life and experience. "

Mr. Baldwin is also attacking the greater problem: lack of ringers.

One survey says that of the 100,000 trained bellringers in England today, only 40,000 of them still ring.

Also, ringers do not always live where the bells need to be rung -- like in small, rural parishes and inner-city churches.

But the council's program to attract and train ringers is only partly successful, says Mr. Baldwin.

"Right now, we're lucky if after five years we keep one new ringer out of 10, "he adds.

One bright sign is that a growing number of women have entered the once male-dominated field; more than a third of the ringers today are women.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: right
Target surface form: Right
Context:
Says Mr. Baldwin, "We recognize that we may no longer have as high a priority in church life and experience. "Mr. Baldwin is also attacking the greater problem: lack of ringers. One survey says that of the 100,000 trained bellringers in England today, only 40,000 of them still ring. Also, ringers do not always live where the bells need to be rung -- like in small, rural parishes and inner-city churches. But the council's program to attract and train ringers is only partly successful, says Mr. Baldwin. "<t>Right</t> now, we're lucky if after five years we keep one new ringer out of 10, "he adds. One bright sign is that a growing number of women have entered the once male-dominated field; more than a third of the ringers today are women.

Candidate senses:
1. sense_key=right%4:02:04:: | definition=precisely, exactly | examples=stand right here!
2. sense_key=right%4:02:08:: | definition=immediately | examples=she called right after dinner
3. sense_key=right%4:02:09:: | definition=exactly | synonyms=flop | examples=he fell flop on his face
4. sense_key=right%4:02:03:: | definition=toward or on the right; also used figuratively | examples=he looked right and left; the party has moved right
5. sense_key=right%4:02:01:: | definition=in the right manner | synonyms=properly, decently, decent, in good order, the right way | examples=please do your job properly!; can't you carry me decent?
6. sense_key=right%4:02:00:: | definition=an interjection expressing agreement | synonyms=right on
7. sense_key=right%4:02:07:: | definition=completely | examples=she felt right at home; he fell right into the trap
8. sense_key=right%4:02:06:: | definition=(Southern regional intensive) very; to a great degree | synonyms=mighty, mightily, powerful | examples=the baby is mighty cute; he's mighty tired
9. sense_key=right%4:02:05:: | definition=in accordance with moral or social standards | synonyms=justly | examples=that serves him right; do right by him
10. sense_key=right%4:02:02:: | definition=in an accurate manner | synonyms=correctly, aright | examples=the flower had been correctly depicted by his son; he guessed right

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    precisely, exactly

    right%4:02:04:: · right.r.01

    • stand right here!
    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    immediately

    right%4:02:08:: · right.r.02

    • she called right after dinner
    Selected
  3. 3

    exactly

    right%4:02:09:: · right.r.03

    • he fell flop on his face

    Synonyms: flop

  4. 4

    toward or on the right; also used figuratively

    right%4:02:03:: · right.r.04

    • he looked right and left
    • the party has moved right
  5. 5

    in the right manner

    right%4:02:01:: · properly.r.01

    • please do your job properly!
    • can't you carry me decent?

    Synonyms: properly, decently, decent, in good order, the right way

  6. 6

    an interjection expressing agreement

    right%4:02:00:: · right.r.06

    Synonyms: right on

  7. 7

    completely

    right%4:02:07:: · right.r.07

    • she felt right at home
    • he fell right into the trap
  8. 8

    (Southern regional intensive) very; to a great degree

    right%4:02:06:: · mighty.r.01

    • the baby is mighty cute
    • he's mighty tired

    Synonyms: mighty, mightily, powerful

  9. 9

    in accordance with moral or social standards

    right%4:02:05:: · justly.r.02

    • that serves him right
    • do right by him

    Synonyms: justly

  10. 10

    in an accurate manner

    right%4:02:02:: · correctly.r.01

    • the flower had been correctly depicted by his son
    • he guessed right

    Synonyms: correctly, aright

simply Adv

Instance: senseval2.d001.s008.t005 Dataset: senseval2 Bucket: POS / ADV

Incorrect

That is for the future.

Already, scientists are developing tests based on the newly identified genes that, for the first time, can predict whether an otherwise healthy individual is likely to get cancer.

"It's a super-exciting set of discoveries, "says Bert Vogelstein, a Johns Hopkins University researcher who has just found a gene pivotal to the triggering of colon cancer.

"Only a decade ago cancer was a black box about which we knew nothing at the molecular level.

Today, we know that the accumulation of several of these altered genes can initiate a cancer and, then, propel it into a deadly state. "

Scientists call the new class of genes tumor-suppressors, or simply anti-cancer genes.

When functioning normally, they make proteins that hold a cell's growth in check.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: simply
Target surface form: simply
Context:
That is for the future. Already, scientists are developing tests based on the newly identified genes that, for the first time, can predict whether an otherwise healthy individual is likely to get cancer. "It's a super-exciting set of discoveries, "says Bert Vogelstein, a Johns Hopkins University researcher who has just found a gene pivotal to the triggering of colon cancer. "Only a decade ago cancer was a black box about which we knew nothing at the molecular level. Today, we know that the accumulation of several of these altered genes can initiate a cancer and, then, propel it into a deadly state. "Scientists call the new class of genes tumor-suppressors, or <t>simply</t> anti-cancer genes. When functioning normally, they make proteins that hold a cell's growth in check.

Candidate senses:
1. sense_key=simply%4:02:00:: | definition=and nothing more | synonyms=merely, just, only, but | examples=I was merely asking; it is simply a matter of time
2. sense_key=simply%4:02:03:: | definition=absolutely | synonyms=just | examples=I just can't take it anymore; he was just grand as Romeo
3. sense_key=simply%4:02:02:: | definition=absolutely; altogether; really | examples=we are simply broke
4. sense_key=simply%4:02:01:: | definition=in a simple manner; without extravagance or embellishment | synonyms=plainly | examples=she was dressed plainly; they lived very simply

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✓

WordNet 3.0 senses

  1. 1

    and nothing more

    simply%4:02:00:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, just, only, but

    Gold Raganato Selected
  2. 2

    absolutely

    simply%4:02:03:: · just.r.04

    • I just can't take it anymore
    • he was just grand as Romeo

    Synonyms: just

  3. 3

    absolutely; altogether; really

    simply%4:02:02:: · simply.r.03

    • we are simply broke
  4. 4

    in a simple manner; without extravagance or embellishment

    simply%4:02:01:: · plainly.r.02

    • she was dressed plainly
    • they lived very simply

    Synonyms: plainly

    Gold lexEN Gold Maru 2022 Gold Raganato

there Adv

Instance: senseval2.d001.s027.t010 Dataset: senseval2 Bucket: POS / ADV

Incorrect

A test using new "genetic probes "showed that little Will Quinlan had not inherited a damaged retinoblastoma supressor gene and, therefore, faced no more risk than other children of developing the rare cancer.

"It made our New Year, "says Mr. Quinlan.

This test was the first to predict reliably whether an individual could expect to develop cancer.

Equally important, the initial discovery of the gene that controls retinal cell growth, made by a Boston doctor named Thaddeus Dryja, has opened a field of cancer study, which in recent months has exploded.

"It turns out that studying a tragic but uncommon tumor made possible some fundamental insights about the most basic workings of cancer, "says Samuel Broder, director of the National Cancer Institute.

"All this may not be obvious to the public, which is concerned about advances in treatment, but I am convinced this basic research will begin showing results there soon. "

To date, scientists have fingered two of these cancer-suppressors.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: there
Target surface form: there
Context:
A test using new "genetic probes "showed that little Will Quinlan had not inherited a damaged retinoblastoma supressor gene and, therefore, faced no more risk than other children of developing the rare cancer. "It made our New Year, "says Mr. Quinlan. This test was the first to predict reliably whether an individual could expect to develop cancer. Equally important, the initial discovery of the gene that controls retinal cell growth, made by a Boston doctor named Thaddeus Dryja, has opened a field of cancer study, which in recent months has exploded. "It turns out that studying a tragic but uncommon tumor made possible some fundamental insights about the most basic workings of cancer, "says Samuel Broder, director of the National Cancer Institute. "All this may not be obvious to the public, which is concerned about advances in treatment, but I am convinced this basic research will begin showing results <t>there</t> soon. "To date, scientists have fingered two of these cancer-suppressors.

Candidate senses:
1. sense_key=there%4:02:00:: | definition=in or at that place | synonyms=at that place, in that location | examples=they have lived there for years; it's not there
2. sense_key=there%4:02:02:: | definition=in that matter | synonyms=in that respect, on that point | examples=I agree with you there
3. sense_key=there%4:02:01:: | definition=to or toward that place; away from the speaker | synonyms=thither | examples=go there around noon!

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✓

WordNet 3.0 senses

  1. 1

    in or at that place

    there%4:02:00:: · there.r.01

    • they have lived there for years
    • it's not there

    Synonyms: at that place, in that location

    Gold Raganato Selected
  2. 2

    in that matter

    there%4:02:02:: · there.r.02

    • I agree with you there

    Synonyms: in that respect, on that point

    Gold lexEN Gold Maru 2022
  3. 3

    to or toward that place; away from the speaker

    there%4:02:01:: · there.r.03

    • go there around noon!

    Synonyms: thither

finally Adv

Instance: senseval2.d001.s036.t010 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Soon after that report, two other research teams uncovered evidence that the same damaged p53 gene is present in tissue from lung and breast cancers.

Colon, lung and breast cancers are the most common and lethal forms of the disease, collectively killing almost 200,000 Americans a year.

Right now about a dozen laboratories, in the U.S., Canada and Britain, are racing to unmask other suspected tumor-suppressing genes.

They have about seven candidates.

Researchers say the inactivation of tumor-suppressor genes, alone or in combination, appears crucial to the development of such scourges as cancer of the brain, the skin, kidney, prostate, and cervix.

There is evidence that if people inherit defective versions of these genes, they are especially prone to cancer, perhaps explaining, finally, why some cancers seem to haunt certain families.

The story of tumor-suppressor genes goes back to the 1970s, when a pediatrician named Alfred G. Knudson Jr. proposed that retinoblastoma stemmed from two separate genetic defects.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: finally
Target surface form: finally
Context:
Soon after that report, two other research teams uncovered evidence that the same damaged p53 gene is present in tissue from lung and breast cancers. Colon, lung and breast cancers are the most common and lethal forms of the disease, collectively killing almost 200,000 Americans a year. Right now about a dozen laboratories, in the U.S., Canada and Britain, are racing to unmask other suspected tumor-suppressing genes. They have about seven candidates. Researchers say the inactivation of tumor-suppressor genes, alone or in combination, appears crucial to the development of such scourges as cancer of the brain, the skin, kidney, prostate, and cervix. There is evidence that if people inherit defective versions of these genes, they are especially prone to cancer, perhaps explaining, <t>finally</t>, why some cancers seem to haunt certain families. The story of tumor-suppressor genes goes back to the 1970s, when a pediatrician named Alfred G. Knudson Jr. proposed that retinoblastoma stemmed from two separate genetic defects.

Candidate senses:
1. sense_key=finally%4:02:02:: | definition=after an unspecified period of time or an especially long delay | synonyms=eventually
2. sense_key=finally%4:02:00:: | definition=as the end result of a succession or process | synonyms=ultimately, in the end, at last, at long last | examples=ultimately he had to give in; at long last the winter was over
3. sense_key=finally%4:02:03:: | definition=the item at the end | synonyms=last, lastly, in conclusion | examples=last, I'll discuss family values

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    after an unspecified period of time or an especially long delay

    finally%4:02:02:: · finally.r.01

    Synonyms: eventually

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    as the end result of a succession or process

    finally%4:02:00:: · ultimately.r.01

    • ultimately he had to give in
    • at long last the winter was over

    Synonyms: ultimately, in the end, at last, at long last

    Selected
  3. 3

    the item at the end

    finally%4:02:03:: · last.r.02

    • last, I'll discuss family values

    Synonyms: last, lastly, in conclusion