Verified run

gemini/gemini-4-argon

gemini-4-argon-high-reasoning-p001-lexen-v1-20261013

Rank
10
Accuracy lexEN v1 · WordNet fine
95.12%
Correct
4624 / 4861
Actual run cost
$37.42
Cost / M items
$7,699

Accuracy across label schemes

The headline accuracy is lexEN v1 · WordNet fine-grained. Each cell re-scores this run's predictions against a different gold label set and sense granularity; coarse counts a prediction correct when its WordNet sense maps to the same Glite or CSI concept as a gold sense. About these label schemes → About coarsening →

Score against WordNet fine-grainedGlite coarse-grainedCSI coarse-grained (Lacerra 2020)
lexEN v1 95.12% 4624 / 4861 98.42% 4784 / 4861 97.08% 4719 / 4861
Maru 2022 (ALLamended) 92.47% 4495 / 4861 97.61% 4745 / 4861 95.52% 4643 / 4861
Raganato 2017 (original) 86.32% 4196 / 4861 95.06% 4621 / 4861 91.50% 4448 / 4861

Model

Display
gemini/gemini-4-argon
Requested
gemini/gemini-4-argon
Resolved
gemini-4-argon
Vendor
Google
API provider
Google
Hosting
cloud_api
Source
proprietary
License
n/a
Reasoning
high
Model URL
n/a
Endpoint
n/a
Resolved counts
gemini-4-argon: 4861

Reproducibility

Created
2026-10-13T18:00:12.937587+00:00
Run schema
sensebench-run-v2
Commit
56adc7fc800950485db00c464c9b65d18715e975
Dataset ID
lexen
Dataset version
lexen-v1
Dataset revision
n/a
Dataset items
4,861
Dataset hash
sha256:5fd4382b93f19087a1e31f6dd7d1db17c1eb17ff80fcbe1d3fdd55c0c3ecefe8
Prompt
p001
Prompt version
0.1.0
Repository files
Browse in GitHub

Runtime

Calls
4861
Total tokens
9,376,938
Tokens / item
1,929.0
Total cost
$37.42
Cost / M items
$7,699
Cost source
litellm_estimate
Elapsed
27.18m
Success items
4,861
Monosemous
0
No candidates
0
No valid vote
0
Invalid votes
0
Transport errors
0

Runner

GitHub
emilkayumov
Name
Emil Kaiumov
Contact
n/a

Sampling

Temperature
n/a
Top P
n/a
Max tokens
32,768
Seed
n/a
Extra
n/a

Policy

Votes / item
1
Reasks / invalid vote
1
Tie break
earliest_vote
Monosemous
short_circuit

Cost And Token Breakdown

Actual recorded run cost: $37.42. Cost estimate source: litellm_estimate.

Token Type Tokens Cost Price / 1M tokens
Input uncached 3,251,214 $6.50 $2.00
Input cached n/a $0.00 $0.100
Output 3,092,074 $30.92 $10.00
Reasoning output 3,033,650 included in output cost n/a

Reasoning output tokens are shown separately when the provider reports them; recorded cost is stored in the output cost bucket.

Download Raw Run Files

Slice Breakdown

Group Bucket Accuracy Correct Items
Candidate Count 11+ 91.81% 729 794
Candidate Count 2 98.01% 838 855
Candidate Count 3-5 95.84% 1864 1945
Candidate Count 6-10 94.16% 1193 1267
POS ADJ 95.90% 632 659
POS ADV 91.79% 179 195
POS NOUN 96.13% 2785 2897
POS VERB 92.61% 1028 1110
Source Dataset semeval2013 97.02% 1106 1140
Source Dataset semeval2015 95.67% 707 739
Source Dataset senseval2 94.23% 1535 1629
Source Dataset senseval3 94.31% 1276 1353

Worst-Bucket Examples

only Adv

Instance: semeval2015.d000.s013.t000 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

Alimta is used to treat two types of cancer: pleural mesothelioma (cancer of the lining of the lungs that is usually caused by exposure to asbestos).

Alimta is used together with cisplatin (another anticancer medicine) when the cancer is unresectable (cannot be removed by surgery alone) and malignant (has spread, or is likely to spread easily, to other parts of the body), in patients who have not received chemotherapy (medicines for cancer) before advanced or metastatic non-small cell lung cancer that is not affecting the squamous cells.

Alimta is used in combination with cisplatin in patients who have not been treated before, and on its own in patients who have received another type of chemotherapy in the past.

The medicine can only be obtained with a prescription.

How is Alimta used?

Alimta should only be given under the supervision of a doctor who is qualified in the use of anticancer chemotherapy.

The recommended dose of Alimta is 500 mg per square metre of body surface area (calculated using the patient s height and weight).

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: only
Target surface form: only
Context:
Alimta is used to treat two types of cancer: pleural mesothelioma (cancer of the lining of the lungs that is usually caused by exposure to asbestos). Alimta is used together with cisplatin (another anticancer medicine) when the cancer is unresectable (cannot be removed by surgery alone) and malignant (has spread, or is likely to spread easily, to other parts of the body), in patients who have not received chemotherapy (medicines for cancer) before advanced or metastatic non-small cell lung cancer that is not affecting the squamous cells. Alimta is used in combination with cisplatin in patients who have not been treated before, and on its own in patients who have received another type of chemotherapy in the past. The medicine can only be obtained with a prescription. How is Alimta used? Alimta should <t>only</t> be given under the supervision of a doctor who is qualified in the use of anticancer chemotherapy. The recommended dose of Alimta is 500 mg per square metre of body surface area (calculated using the patient s height and weight).

Candidate senses:
1. sense_key=only%4:02:02:: | definition=and nothing more | synonyms=merely, simply, just, but | examples=I was merely asking; it is simply a matter of time
2. sense_key=only%4:02:01:: | definition=without any others being included or involved | synonyms=entirely, exclusively, solely, alone | examples=was entirely to blame; a school devoted entirely to the needs of problem children
3. sense_key=only%4:02:00:: | definition=with nevertheless the final result | examples=He arrived only to find his wife dead; We won only to lose again in the next round
4. sense_key=only%4:02:04:: | definition=in the final outcome | examples=These news will only make you more upset
5. sense_key=only%4:02:06:: | definition=except that | examples=It was the same story; only this time she came out better
6. sense_key=only%4:02:03:: | definition=never except when | synonyms=only if, only when | examples=call me only if your cold gets worse
7. sense_key=only%4:02:05:: | definition=as recently as | examples=I spoke to him only an hour ago

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    only%4:02:02:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, just, but

  2. 2

    without any others being included or involved

    only%4:02:01:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, alone

    Selected
  3. 3

    with nevertheless the final result

    only%4:02:00:: · only.r.03

    • He arrived only to find his wife dead
    • We won only to lose again in the next round
  4. 4

    in the final outcome

    only%4:02:04:: · only.r.04

    • These news will only make you more upset
  5. 5

    except that

    only%4:02:06:: · only.r.05

    • It was the same story; only this time she came out better
  6. 6

    never except when

    only%4:02:03:: · only.r.06

    • call me only if your cold gets worse

    Synonyms: only if, only when

    Gold lexEN Gold Maru 2022 Gold Raganato
  7. 7

    as recently as

    only%4:02:05:: · only.r.07

    • I spoke to him only an hour ago

however Adv

Instance: semeval2015.d000.s036.t000 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

Alimta increased the survival time of patients with malignant pleural mesothelioma.

Patients receiving Alimta and cisplatin survived for an average of 12.1 months, compared with 9.3 months in those receiving cisplatin alone.

In the treatment of patients with non-small cell lung cancer, Alimta was as effective as the comparators.

In patients who had received chemotherapy in the past, the average survival time was 8.3 months with Alimta, compared with 7.9 months with docetaxel.

In patients who had not received chemotherapy in the past, the average survival time was 10.3 months in both groups.

However, in both studies, patients whose cancer was not affecting squamous cells had longer survival times if they received Alimta than if they received the comparator.

In contrast, patients whose cancer was affecting squamous cells had shorter survival times if they received Alimta.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: however
Target surface form: However
Context:
Alimta increased the survival time of patients with malignant pleural mesothelioma. Patients receiving Alimta and cisplatin survived for an average of 12.1 months, compared with 9.3 months in those receiving cisplatin alone. In the treatment of patients with non-small cell lung cancer, Alimta was as effective as the comparators. In patients who had received chemotherapy in the past, the average survival time was 8.3 months with Alimta, compared with 7.9 months with docetaxel. In patients who had not received chemotherapy in the past, the average survival time was 10.3 months in both groups. <t>However</t>, in both studies, patients whose cancer was not affecting squamous cells had longer survival times if they received Alimta than if they received the comparator. In contrast, patients whose cancer was affecting squamous cells had shorter survival times if they received Alimta.

Candidate senses:
1. sense_key=however%4:02:00:: | definition=despite anything to the contrary (usually following a concession) | synonyms=nevertheless, withal, still, yet, all the same, even so | examples=although I'm a little afraid, however I'd like to try it; while we disliked each other, nevertheless we agreed
2. sense_key=however%4:02:04:: | definition=by contrast; on the other hand | examples=the first part was easy; the second, however, took hours
3. sense_key=however%4:02:02:: | definition=to whatever degree or extent | examples=The results, however general, are important; they have begun, however reluctantly, to acknowledge the legitimacy of some of the opposition's concerns
4. sense_key=however%4:02:01:: | definition=in whatever way or manner | examples=Victory, however it was brought about, was sweet; however he did it, it was very clever

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✓ Raganato ✗

WordNet 3.0 senses

  1. 1

    despite anything to the contrary (usually following a concession)

    however%4:02:00:: · however.r.01

    • although I'm a little afraid, however I'd like to try it
    • while we disliked each other, nevertheless we agreed

    Synonyms: nevertheless, withal, still, yet, all the same, even so, nonetheless, notwithstanding

    Gold Maru 2022 Selected
  2. 2

    by contrast; on the other hand

    however%4:02:04:: · however.r.02

    • the first part was easy; the second, however, took hours
    Gold lexEN Gold Raganato
  3. 3

    to whatever degree or extent

    however%4:02:02:: · however.r.03

    • The results, however general, are important
    • they have begun, however reluctantly, to acknowledge the legitimacy of some of the opposition's concerns
  4. 4

    in whatever way or manner

    however%4:02:01:: · however.r.04

    • Victory, however it was brought about, was sweet
    • however he did it, it was very clever

just Adv

Instance: senseval2.d000.s012.t005 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Like most of the other 6,000 churches in Britain with sets of bells, St. Michael once had its own "band "of ringers, who would herald every Sunday morning and evening service.

Now, only one local ringer remains: 64-year-old Derek Hammond.

The others here today live elsewhere.

They belong to a group of 15 ringers -- including two octogenarians and four youngsters in training -- who drive every Sunday from church to church in a sometimes-exhausting effort to keep the bells sounding in the many belfries of East Anglia.

"To ring for even one service at this tower, we have to scrape, "says Mr. Hammond, a retired water-authority worker. "

We've tried to train the youngsters, but they have their discos and their dances, and they just drift away. "

Mr. Hammond worries that old age and the flightiness of youth will diminish the ranks of the East Anglian group that keeps the Aslacton bells pealing.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: just
Target surface form: just
Context:
Like most of the other 6,000 churches in Britain with sets of bells, St. Michael once had its own "band "of ringers, who would herald every Sunday morning and evening service. Now, only one local ringer remains: 64-year-old Derek Hammond. The others here today live elsewhere. They belong to a group of 15 ringers -- including two octogenarians and four youngsters in training -- who drive every Sunday from church to church in a sometimes-exhausting effort to keep the bells sounding in the many belfries of East Anglia. "To ring for even one service at this tower, we have to scrape, "says Mr. Hammond, a retired water-authority worker. "We've tried to train the youngsters, but they have their discos and their dances, and they <t>just</t> drift away. "Mr. Hammond worries that old age and the flightiness of youth will diminish the ranks of the East Anglian group that keeps the Aslacton bells pealing.

Candidate senses:
1. sense_key=just%4:02:00:: | definition=and nothing more | synonyms=merely, simply, only, but | examples=I was merely asking; it is simply a matter of time
2. sense_key=just%4:02:05:: | definition=indicating exactness or preciseness | synonyms=precisely, exactly | examples=he was doing precisely (or exactly) what she had told him to do; it was just as he said--the jewel was gone
3. sense_key=just%4:02:01:: | definition=only a moment ago | synonyms=just now | examples=he has just arrived; the sun just now came out
4. sense_key=just%4:02:03:: | definition=absolutely | synonyms=simply | examples=I just can't take it anymore; he was just grand as Romeo
5. sense_key=just%4:02:04:: | definition=only a very short time before; ; ; ; ; - W.B.Yeats | synonyms=barely, hardly, scarcely, scarce | examples=they could barely hear the speaker; we hardly knew them
6. sense_key=just%4:02:06:: | definition=exactly at this moment or the moment described | examples=we've just finished painting the walls, so don't touch them

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    just%4:02:00:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, only, but

    Selected
  2. 2

    indicating exactness or preciseness

    just%4:02:05:: · precisely.r.01

    • he was doing precisely (or exactly) what she had told him to do
    • it was just as he said--the jewel was gone

    Synonyms: precisely, exactly

  3. 3

    only a moment ago

    just%4:02:01:: · just.r.03

    • he has just arrived
    • the sun just now came out

    Synonyms: just now

  4. 4

    absolutely

    just%4:02:03:: · just.r.04

    • I just can't take it anymore
    • he was just grand as Romeo

    Synonyms: simply

    Gold lexEN Gold Maru 2022 Gold Raganato
  5. 5

    only a very short time before; ; ; ; ; - W.B.Yeats

    just%4:02:04:: · barely.r.01

    • they could barely hear the speaker
    • we hardly knew them

    Synonyms: barely, hardly, scarcely, scarce

  6. 6

    exactly at this moment or the moment described

    just%4:02:06:: · just.r.06

    • we've just finished painting the walls, so don't touch them

below Adv

Instance: senseval2.d000.s035.t011 Dataset: senseval2 Bucket: POS / ADV

Incorrect

In a well-known detective-story involving church bells, English novelist Dorothy L. Sayers described ringing as a "passion that finds its satisfaction in mathematical completeness and mechanical perfection. "

Ringers, she added, are "filled with the solemn intoxication that comes of intricate ritual faultlessly performed. ""

Ringing does become a bit of an obsession, "admits Stephanie Pattenden, master of the band at St. Mary Abbot and one of England's best female ringers.

It is a passion that usually stays in the tower, however.

More often than not, ringers think of the church as something stuck on the bottom of the belfry.

When their changes are completed, and after they have worked up a sweat, ringers often skip off to the local pub, leaving worship for others below.

This does not sit well with some clerics.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: below
Target surface form: below
Context:
In a well-known detective-story involving church bells, English novelist Dorothy L. Sayers described ringing as a "passion that finds its satisfaction in mathematical completeness and mechanical perfection. "Ringers, she added, are "filled with the solemn intoxication that comes of intricate ritual faultlessly performed. ""Ringing does become a bit of an obsession, "admits Stephanie Pattenden, master of the band at St. Mary Abbot and one of England's best female ringers. It is a passion that usually stays in the tower, however. More often than not, ringers think of the church as something stuck on the bottom of the belfry. When their changes are completed, and after they have worked up a sweat, ringers often skip off to the local pub, leaving worship for others <t>below</t>. This does not sit well with some clerics.

Candidate senses:
1. sense_key=below%4:02:01:: | definition=in or to a place that is lower | synonyms=at a lower place, to a lower place, beneath
2. sense_key=below%4:02:00:: | definition=at a later place | examples=see below
3. sense_key=below%4:02:02:: | definition=(in writing) see below | synonyms=infra | examples=vide infra
4. sense_key=below%4:02:06:: | definition=on a floor below | synonyms=downstairs, down the stairs, on a lower floor | examples=the tenants live downstairs
5. sense_key=below%4:02:03:: | definition=further down | synonyms=under | examples=see under for further discussion

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    in or to a place that is lower

    below%4:02:01:: · below.r.01

    Synonyms: at a lower place, to a lower place, beneath

    Selected
  2. 2

    at a later place

    below%4:02:00:: · below.r.02

    • see below
  3. 3

    (in writing) see below

    below%4:02:02:: · below.r.03

    • vide infra

    Synonyms: infra

  4. 4

    on a floor below

    below%4:02:06:: · downstairs.r.01

    • the tenants live downstairs

    Synonyms: downstairs, down the stairs, on a lower floor

    Gold lexEN Gold Maru 2022 Gold Raganato
  5. 5

    further down

    below%4:02:03:: · under.r.08

    • see under for further discussion

    Synonyms: under

promptly Adv

Instance: senseval2.d000.s038.t012 Dataset: senseval2 Bucket: POS / ADV

Incorrect

It is a passion that usually stays in the tower, however.

More often than not, ringers think of the church as something stuck on the bottom of the belfry.

When their changes are completed, and after they have worked up a sweat, ringers often skip off to the local pub, leaving worship for others below.

This does not sit well with some clerics.

With membership of the Church of England steadily dwindling, strong-willed vicars are pressing equally strong-willed and often non-religious ringers to attend services.

Two years ago, the Rev. Jeremy Hummerstone, vicar of Great Torrington, Devon, got so fed up with ringers who didn't attend service he sacked the entire band; the ringers promptly set up a picket line in protest.

"They were a self-perpetuating club that treated the tower as sort of a separate premises, "the Vicar Hummerstone says.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: promptly
Target surface form: promptly
Context:
It is a passion that usually stays in the tower, however. More often than not, ringers think of the church as something stuck on the bottom of the belfry. When their changes are completed, and after they have worked up a sweat, ringers often skip off to the local pub, leaving worship for others below. This does not sit well with some clerics. With membership of the Church of England steadily dwindling, strong-willed vicars are pressing equally strong-willed and often non-religious ringers to attend services. Two years ago, the Rev. Jeremy Hummerstone, vicar of Great Torrington, Devon, got so fed up with ringers who didn't attend service he sacked the entire band; the ringers <t>promptly</t> set up a picket line in protest. "They were a self-perpetuating club that treated the tower as sort of a separate premises, "the Vicar Hummerstone says.

Candidate senses:
1. sense_key=promptly%4:02:02:: | definition=with little or no delay | synonyms=quickly, quick | examples=the rescue squad arrived promptly; come here, quick!
2. sense_key=promptly%4:02:00:: | definition=in a punctual manner | synonyms=readily, pronto | examples=he did his homework promptly
3. sense_key=promptly%4:02:01:: | definition=at once (usually modifies an undesirable occurrence) | synonyms=right away | examples=he promptly forgot the address

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    with little or no delay

    promptly%4:02:02:: · promptly.r.01

    • the rescue squad arrived promptly
    • come here, quick!

    Synonyms: quickly, quick

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    in a punctual manner

    promptly%4:02:00:: · promptly.r.02

    • he did his homework promptly

    Synonyms: readily, pronto

  3. 3

    at once (usually modifies an undesirable occurrence)

    promptly%4:02:01:: · promptly.r.03

    • he promptly forgot the address

    Synonyms: right away

    Selected

always Adv

Instance: senseval2.d000.s054.t002 Dataset: senseval2 Bucket: POS / ADV

Incorrect

The Central Council of Church Bell Ringers, a sort of parliament of ringing groups, aims to improve relations with vicars, says John C. Baldwin, president.

It hopes to speak to students at theological colleges about the joys of bell ringing and will shortly publish a booklet for every vicar in the country entitled, "The Bells in Your Care. "

Says Mr. Baldwin, "We recognize that we may no longer have as high a priority in church life and experience. "

Mr. Baldwin is also attacking the greater problem: lack of ringers.

One survey says that of the 100,000 trained bellringers in England today, only 40,000 of them still ring.

Also, ringers do not always live where the bells need to be rung -- like in small, rural parishes and inner-city churches.

But the council's program to attract and train ringers is only partly successful, says Mr. Baldwin.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: always
Target surface form: always
Context:
The Central Council of Church Bell Ringers, a sort of parliament of ringing groups, aims to improve relations with vicars, says John C. Baldwin, president. It hopes to speak to students at theological colleges about the joys of bell ringing and will shortly publish a booklet for every vicar in the country entitled, "The Bells in Your Care. "Says Mr. Baldwin, "We recognize that we may no longer have as high a priority in church life and experience. "Mr. Baldwin is also attacking the greater problem: lack of ringers. One survey says that of the 100,000 trained bellringers in England today, only 40,000 of them still ring. Also, ringers do not <t>always</t> live where the bells need to be rung -- like in small, rural parishes and inner-city churches. But the council's program to attract and train ringers is only partly successful, says Mr. Baldwin.

Candidate senses:
1. sense_key=always%4:02:00:: | definition=at all times; all the time and on every occasion | synonyms=ever, e'er | examples=I will always be there to help you; always arrives on time
2. sense_key=always%4:02:04:: | definition=without variation or change, in every case | synonyms=constantly, invariably | examples=constantly kind and gracious; he always arrives on time
3. sense_key=always%4:02:03:: | definition=without interruption | synonyms=constantly, forever, perpetually, incessantly | examples=the world is constantly changing
4. sense_key=always%4:02:02:: | definition=at any time or in any event | examples=you can always resign if you don't like it; you could always take a day off
5. sense_key=always%4:02:01:: | definition=forever; throughout all time | examples=we will always be friends; I shall treasure it always

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✓ Raganato ✓

WordNet 3.0 senses

  1. 1

    at all times; all the time and on every occasion

    always%4:02:00:: · always.r.01

    • I will always be there to help you
    • always arrives on time

    Synonyms: ever, e'er

    Gold Maru 2022 Gold Raganato Selected
  2. 2

    without variation or change, in every case

    always%4:02:04:: · constantly.r.01

    • constantly kind and gracious
    • he always arrives on time

    Synonyms: constantly, invariably

    Gold lexEN
  3. 3

    without interruption

    always%4:02:03:: · constantly.r.02

    • the world is constantly changing

    Synonyms: constantly, forever, perpetually, incessantly

  4. 4

    at any time or in any event

    always%4:02:02:: · always.r.04

    • you can always resign if you don't like it
    • you could always take a day off
  5. 5

    forever; throughout all time

    always%4:02:01:: · always.r.05

    • we will always be friends
    • I shall treasure it always

simply Adv

Instance: senseval2.d001.s008.t005 Dataset: senseval2 Bucket: POS / ADV

Incorrect

That is for the future.

Already, scientists are developing tests based on the newly identified genes that, for the first time, can predict whether an otherwise healthy individual is likely to get cancer.

"It's a super-exciting set of discoveries, "says Bert Vogelstein, a Johns Hopkins University researcher who has just found a gene pivotal to the triggering of colon cancer.

"Only a decade ago cancer was a black box about which we knew nothing at the molecular level.

Today, we know that the accumulation of several of these altered genes can initiate a cancer and, then, propel it into a deadly state. "

Scientists call the new class of genes tumor-suppressors, or simply anti-cancer genes.

When functioning normally, they make proteins that hold a cell's growth in check.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: simply
Target surface form: simply
Context:
That is for the future. Already, scientists are developing tests based on the newly identified genes that, for the first time, can predict whether an otherwise healthy individual is likely to get cancer. "It's a super-exciting set of discoveries, "says Bert Vogelstein, a Johns Hopkins University researcher who has just found a gene pivotal to the triggering of colon cancer. "Only a decade ago cancer was a black box about which we knew nothing at the molecular level. Today, we know that the accumulation of several of these altered genes can initiate a cancer and, then, propel it into a deadly state. "Scientists call the new class of genes tumor-suppressors, or <t>simply</t> anti-cancer genes. When functioning normally, they make proteins that hold a cell's growth in check.

Candidate senses:
1. sense_key=simply%4:02:00:: | definition=and nothing more | synonyms=merely, just, only, but | examples=I was merely asking; it is simply a matter of time
2. sense_key=simply%4:02:03:: | definition=absolutely | synonyms=just | examples=I just can't take it anymore; he was just grand as Romeo
3. sense_key=simply%4:02:02:: | definition=absolutely; altogether; really | examples=we are simply broke
4. sense_key=simply%4:02:01:: | definition=in a simple manner; without extravagance or embellishment | synonyms=plainly | examples=she was dressed plainly; they lived very simply

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✓

WordNet 3.0 senses

  1. 1

    and nothing more

    simply%4:02:00:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, just, only, but

    Gold Raganato Selected
  2. 2

    absolutely

    simply%4:02:03:: · just.r.04

    • I just can't take it anymore
    • he was just grand as Romeo

    Synonyms: just

  3. 3

    absolutely; altogether; really

    simply%4:02:02:: · simply.r.03

    • we are simply broke
  4. 4

    in a simple manner; without extravagance or embellishment

    simply%4:02:01:: · plainly.r.02

    • she was dressed plainly
    • they lived very simply

    Synonyms: plainly

    Gold lexEN Gold Maru 2022 Gold Raganato

alone Adv

Instance: senseval2.d001.s035.t003 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Then last spring, researchers reported finding a gene called p53 which, if impaired, turns healthy colon cells cancerous.

Soon after that report, two other research teams uncovered evidence that the same damaged p53 gene is present in tissue from lung and breast cancers.

Colon, lung and breast cancers are the most common and lethal forms of the disease, collectively killing almost 200,000 Americans a year.

Right now about a dozen laboratories, in the U.S., Canada and Britain, are racing to unmask other suspected tumor-suppressing genes.

They have about seven candidates.

Researchers say the inactivation of tumor-suppressor genes, alone or in combination, appears crucial to the development of such scourges as cancer of the brain, the skin, kidney, prostate, and cervix.

There is evidence that if people inherit defective versions of these genes, they are especially prone to cancer, perhaps explaining, finally, why some cancers seem to haunt certain families.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: alone
Target surface form: alone
Context:
Then last spring, researchers reported finding a gene called p53 which, if impaired, turns healthy colon cells cancerous. Soon after that report, two other research teams uncovered evidence that the same damaged p53 gene is present in tissue from lung and breast cancers. Colon, lung and breast cancers are the most common and lethal forms of the disease, collectively killing almost 200,000 Americans a year. Right now about a dozen laboratories, in the U.S., Canada and Britain, are racing to unmask other suspected tumor-suppressing genes. They have about seven candidates. Researchers say the inactivation of tumor-suppressor genes, <t>alone</t> or in combination, appears crucial to the development of such scourges as cancer of the brain, the skin, kidney, prostate, and cervix. There is evidence that if people inherit defective versions of these genes, they are especially prone to cancer, perhaps explaining, finally, why some cancers seem to haunt certain families.

Candidate senses:
1. sense_key=alone%4:02:00:: | definition=without any others being included or involved | synonyms=entirely, exclusively, solely, only | examples=was entirely to blame; a school devoted entirely to the needs of problem children
2. sense_key=alone%4:02:01:: | definition=without anybody else or anything else | synonyms=solo, unaccompanied | examples=the child stayed home alone; the pillar stood alone, supporting nothing

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✓

WordNet 3.0 senses

  1. 1

    without any others being included or involved

    alone%4:02:00:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, only

    Gold Raganato Selected
  2. 2

    without anybody else or anything else

    alone%4:02:01:: · alone.r.02

    • the child stayed home alone
    • the pillar stood alone, supporting nothing

    Synonyms: solo, unaccompanied

    Gold lexEN Gold Maru 2022

finally Adv

Instance: senseval2.d001.s036.t010 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Soon after that report, two other research teams uncovered evidence that the same damaged p53 gene is present in tissue from lung and breast cancers.

Colon, lung and breast cancers are the most common and lethal forms of the disease, collectively killing almost 200,000 Americans a year.

Right now about a dozen laboratories, in the U.S., Canada and Britain, are racing to unmask other suspected tumor-suppressing genes.

They have about seven candidates.

Researchers say the inactivation of tumor-suppressor genes, alone or in combination, appears crucial to the development of such scourges as cancer of the brain, the skin, kidney, prostate, and cervix.

There is evidence that if people inherit defective versions of these genes, they are especially prone to cancer, perhaps explaining, finally, why some cancers seem to haunt certain families.

The story of tumor-suppressor genes goes back to the 1970s, when a pediatrician named Alfred G. Knudson Jr. proposed that retinoblastoma stemmed from two separate genetic defects.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: finally
Target surface form: finally
Context:
Soon after that report, two other research teams uncovered evidence that the same damaged p53 gene is present in tissue from lung and breast cancers. Colon, lung and breast cancers are the most common and lethal forms of the disease, collectively killing almost 200,000 Americans a year. Right now about a dozen laboratories, in the U.S., Canada and Britain, are racing to unmask other suspected tumor-suppressing genes. They have about seven candidates. Researchers say the inactivation of tumor-suppressor genes, alone or in combination, appears crucial to the development of such scourges as cancer of the brain, the skin, kidney, prostate, and cervix. There is evidence that if people inherit defective versions of these genes, they are especially prone to cancer, perhaps explaining, <t>finally</t>, why some cancers seem to haunt certain families. The story of tumor-suppressor genes goes back to the 1970s, when a pediatrician named Alfred G. Knudson Jr. proposed that retinoblastoma stemmed from two separate genetic defects.

Candidate senses:
1. sense_key=finally%4:02:02:: | definition=after an unspecified period of time or an especially long delay | synonyms=eventually
2. sense_key=finally%4:02:00:: | definition=as the end result of a succession or process | synonyms=ultimately, in the end, at last, at long last | examples=ultimately he had to give in; at long last the winter was over
3. sense_key=finally%4:02:03:: | definition=the item at the end | synonyms=last, lastly, in conclusion | examples=last, I'll discuss family values

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    after an unspecified period of time or an especially long delay

    finally%4:02:02:: · finally.r.01

    Synonyms: eventually

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    as the end result of a succession or process

    finally%4:02:00:: · ultimately.r.01

    • ultimately he had to give in
    • at long last the winter was over

    Synonyms: ultimately, in the end, at last, at long last

    Selected
  3. 3

    the item at the end

    finally%4:02:03:: · last.r.02

    • last, I'll discuss family values

    Synonyms: last, lastly, in conclusion

often Adv

Instance: senseval2.d001.s060.t005 Dataset: senseval2 Bucket: POS / ADV

Incorrect

But in Baltimore, Dr. Vogelstein, a young molecular biologist at Johns Hopkins Medical School, believed Dr. Knudson was right, and set out to repeat the Cavenee experiment in cells from other cancers.

His was one of two research teams in 1984 to report dual chromosome losses for a rare childhood cancer of the kidney called Wilm's tumor.

Dr. Vogelstein next turned his attention colon cancer, the second biggest cancer killer in the U.S. after lung cancer.

He believed colon cancer might also arise from multiple "hits "on cancer suppressor genes, because it often seems to develop in stages.

It often is preceded by the development of polyps in the bowel, which in some cases become increasingly malignant in identifiable stages -- progressing from less severe to deadly -- as though a cascade of genetic damage might be occurring.

Dr. Vogelstein and a doctoral student, Eric Fearon, began months of tedious and often frustrating probing of the chromosomes searching for signs of genetic damage.

They began uncovering a confusing variety of genetic deletions, some existing only in benign polyps, others in malignant cells and many in both polyps and malignant cells.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: often
Target surface form: often
Context:
But in Baltimore, Dr. Vogelstein, a young molecular biologist at Johns Hopkins Medical School, believed Dr. Knudson was right, and set out to repeat the Cavenee experiment in cells from other cancers. His was one of two research teams in 1984 to report dual chromosome losses for a rare childhood cancer of the kidney called Wilm's tumor. Dr. Vogelstein next turned his attention colon cancer, the second biggest cancer killer in the U.S. after lung cancer. He believed colon cancer might also arise from multiple "hits "on cancer suppressor genes, because it often seems to develop in stages. It often is preceded by the development of polyps in the bowel, which in some cases become increasingly malignant in identifiable stages -- progressing from less severe to deadly -- as though a cascade of genetic damage might be occurring. Dr. Vogelstein and a doctoral student, Eric Fearon, began months of tedious and <t>often</t> frustrating probing of the chromosomes searching for signs of genetic damage. They began uncovering a confusing variety of genetic deletions, some existing only in benign polyps, others in malignant cells and many in both polyps and malignant cells.

Candidate senses:
1. sense_key=often%4:02:00:: | definition=many times at short intervals | synonyms=frequently, oftentimes, oft, ofttimes | examples=we often met over a cup of coffee
2. sense_key=often%4:02:01:: | definition=frequently or in great quantities | synonyms=much, a great deal | examples=I don't drink much; I don't travel much
3. sense_key=often%4:02:02:: | definition=in many cases or instances

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✓

WordNet 3.0 senses

  1. 1

    many times at short intervals

    often%4:02:00:: · frequently.r.01

    • we often met over a cup of coffee

    Synonyms: frequently, oftentimes, oft, ofttimes

    Gold Raganato Selected
  2. 2

    frequently or in great quantities

    often%4:02:01:: · much.r.05

    • I don't drink much
    • I don't travel much

    Synonyms: much, a great deal

  3. 3

    in many cases or instances

    often%4:02:02:: · often.r.03

    Gold lexEN Gold Maru 2022

yet Adv

Instance: senseval2.d001.s083.t003 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Evidence now is emerging that the p53 suppressor gene is involved in other cancers, too.

Researchers in Edinburgh, Scotland, have found that in 23 of 38 breast tumors, one copy of chromosome 17 was mutated at the spot where gene p53 lies.

The scientists say that since breast cancer often strikes multiple members of certain families, the gene, when inherited in a damaged form, may predispose women to the cancer.

The p53 gene has just been implicated in lung cancer.

In a report out last week, John Minna and colleagues at the National Cancer Institute say that about half the cells taken from lung cancer tissue they tested are missing this gene.

There also are reports from several labs, as yet unpublished, of missing p53 genes in tissue taken from kidney, brain and skin cancers.

At the same time, the Johns Hopkins team and others are rushing to pinpoint other tumor-suppressor genes.

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: yet
Target surface form: yet
Context:
Evidence now is emerging that the p53 suppressor gene is involved in other cancers, too. Researchers in Edinburgh, Scotland, have found that in 23 of 38 breast tumors, one copy of chromosome 17 was mutated at the spot where gene p53 lies. The scientists say that since breast cancer often strikes multiple members of certain families, the gene, when inherited in a damaged form, may predispose women to the cancer. The p53 gene has just been implicated in lung cancer. In a report out last week, John Minna and colleagues at the National Cancer Institute say that about half the cells taken from lung cancer tissue they tested are missing this gene. There also are reports from several labs, as <t>yet</t> unpublished, of missing p53 genes in tissue taken from kidney, brain and skin cancers. At the same time, the Johns Hopkins team and others are rushing to pinpoint other tumor-suppressor genes.

Candidate senses:
1. sense_key=yet%4:02:02:: | definition=up to the present time | examples=I have yet to see the results; details are yet to be worked out
2. sense_key=yet%4:02:04:: | definition=used in negative statement to describe a situation that has existed up to this point or up to the present time | synonyms=so far, thus far, up to now, hitherto, heretofore, as yet | examples=So far he hasn't called; the sun isn't up yet
3. sense_key=yet%4:02:03:: | definition=to a greater degree or extent; used with comparisons | synonyms=even, still | examples=looked sick and felt even worse; an even (or still) more interesting problem
4. sense_key=yet%4:02:00:: | definition=within an indefinite time or at an unspecified future time | synonyms=in time | examples=he longed for the flowers that were yet to show themselves; sooner or later you will have to face the facts
5. sense_key=yet%4:02:05:: | definition=used after a superlative | synonyms=so far | examples=this is the best so far; the largest drug bust yet
6. sense_key=yet%4:02:06:: | definition=despite anything to the contrary (usually following a concession) | synonyms=however, nevertheless, withal, still, all the same, even so | examples=although I'm a little afraid, however I'd like to try it; while we disliked each other, nevertheless we agreed

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    up to the present time

    yet%4:02:02:: · yet.r.01

    • I have yet to see the results
    • details are yet to be worked out
    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    used in negative statement to describe a situation that has existed up to this point or up to the present time

    yet%4:02:04:: · so_far.r.01

    • So far he hasn't called
    • the sun isn't up yet

    Synonyms: so far, thus far, up to now, hitherto, heretofore, as yet, til now, until now

    Selected
  3. 3

    to a greater degree or extent; used with comparisons

    yet%4:02:03:: · even.r.03

    • looked sick and felt even worse
    • an even (or still) more interesting problem

    Synonyms: even, still

  4. 4

    within an indefinite time or at an unspecified future time

    yet%4:02:00:: · yet.r.04

    • he longed for the flowers that were yet to show themselves
    • sooner or later you will have to face the facts

    Synonyms: in time

  5. 5

    used after a superlative

    yet%4:02:05:: · so_far.r.03

    • this is the best so far
    • the largest drug bust yet

    Synonyms: so far

  6. 6

    despite anything to the contrary (usually following a concession)

    yet%4:02:06:: · however.r.01

    • although I'm a little afraid, however I'd like to try it
    • while we disliked each other, nevertheless we agreed

    Synonyms: however, nevertheless, withal, still, all the same, even so, nonetheless, notwithstanding

here Adv

Instance: senseval2.d002.s005.t000 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Why ca not we teach our children to read, write and reckon?

It's not that we do not know how to, because we do.

It's that we do not want to.

And the reason we do not want to is that effective education would require us to relinquish some cherished metaphysical beliefs about human nature in general and the human nature of young people in particular, well as to violate some cherished vested interests.

These beliefs so dominate our educational establishment, our media, our politicians, and even our parents that it seems almost blasphemous to challenge them.

Here is an example.

If I were to ask a sample of American parents, "Do you wish the elementary schools to encourage creativity in your children? "the near-unanimous answer would be, "Yes, of course. "

Show raw prompt

system

You are an English linguist performing fine-grained Word Sense Disambiguation. WordNet glosses are schematic. Use the sentence context, the examples, and the synonyms to choose the best indexed sense. Do not hedge. Choose the single best indexed sense.

user

Target lemma: here
Target surface form: Here
Context:
Why ca not we teach our children to read, write and reckon? It's not that we do not know how to, because we do. It's that we do not want to. And the reason we do not want to is that effective education would require us to relinquish some cherished metaphysical beliefs about human nature in general and the human nature of young people in particular, well as to violate some cherished vested interests. These beliefs so dominate our educational establishment, our media, our politicians, and even our parents that it seems almost blasphemous to challenge them. <t>Here</t> is an example. If I were to ask a sample of American parents, "Do you wish the elementary schools to encourage creativity in your children? "the near-unanimous answer would be, "Yes, of course. "

Candidate senses:
1. sense_key=here%4:02:00:: | definition=in or at this place; where the speaker or writer is | examples=I work here; turn here
2. sense_key=here%4:02:02:: | definition=in this circumstance or respect or on this point or detail | examples=what do we have here?; here I must disagree
3. sense_key=here%4:02:01:: | definition=to this place (especially toward the speaker) | synonyms=hither | examples=come here, please
4. sense_key=here%4:02:04:: | definition=at this time; now | examples=we'll adjourn here for lunch and discuss the remaining issues this afternoon

Instructions:
1. Choose the single best indexed sense for the target word in context.
2. WordNet glosses are schematic; use the context, synonyms, and examples together.
3. Do not answer with a WordNet sense key.
4. Return only a JSON object exactly like {"sense_index": 3}.

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✓

WordNet 3.0 senses

  1. 1

    in or at this place; where the speaker or writer is

    here%4:02:00:: · here.r.01

    • I work here
    • turn here
    Gold Raganato Selected
  2. 2

    in this circumstance or respect or on this point or detail

    here%4:02:02:: · here.r.02

    • what do we have here?
    • here I must disagree
    Gold lexEN Gold Maru 2022
  3. 3

    to this place (especially toward the speaker)

    here%4:02:01:: · here.r.03

    • come here, please

    Synonyms: hither

  4. 4

    at this time; now

    here%4:02:04:: · here.r.04

    • we'll adjourn here for lunch and discuss the remaining issues this afternoon