Verified run

openrouter/deepseek/deepseek-v4-pro

deepseek-v4-pro-high-reasoning-p002-lexen-v1-20260614

Rank
47
Accuracy lexEN v1 · WordNet fine
91.50%
Correct
4448 / 4861
Actual run cost
$6.50
Cost / M items
$1,338

Accuracy across label schemes

The headline accuracy is lexEN v1 · WordNet fine-grained. Each cell re-scores this run's predictions against a different gold label set and sense granularity; coarse counts a prediction correct when its WordNet sense maps to the same Glite or CSI concept as a gold sense. About these label schemes → About coarsening →

Score against WordNet fine-grainedGlite coarse-grainedCSI coarse-grained (Lacerra 2020)
lexEN v1 91.50% 4448 / 4861 97.18% 4724 / 4861 94.45% 4591 / 4861
Maru 2022 (ALLamended) 89.10% 4331 / 4861 96.30% 4681 / 4861 92.86% 4514 / 4861
Raganato 2017 (original) 83.17% 4043 / 4861 93.75% 4557 / 4861 88.85% 4319 / 4861

Model

Display
openrouter/deepseek/deepseek-v4-pro
Requested
openrouter/deepseek/deepseek-v4-pro
Resolved
deepseek/deepseek-v4-pro-20260423
Vendor
DeepSeek
API provider
OpenRouter
Hosting
cloud_api
Source
open_source
License
n/a
Reasoning
high
Model URL
n/a
Endpoint
n/a
Resolved counts
deepseek/deepseek-v4-pro-20260423: 4904

Reproducibility

Created
2026-06-14T18:31:09.158241+00:00
Run schema
sensebench-run-v2
Commit
acc066861d4500c1144a3bb5cea415f8c47b8389
Dataset ID
lexen
Dataset version
lexen-v1
Dataset revision
n/a
Dataset items
4,861
Dataset hash
sha256:5fd4382b93f19087a1e31f6dd7d1db17c1eb17ff80fcbe1d3fdd55c0c3ecefe8
Prompt
p002
Prompt version
0.1.0
Repository files
Browse in GitHub

Runtime

Calls
4904
Total tokens
5,302,693
Tokens / item
1,090.9
Total cost
$6.50
Cost / M items
$1,338
Cost source
provider_reported
Elapsed
23.07m
Success items
4,860
Monosemous
0
No candidates
0
No valid vote
1
Invalid votes
1
Transport errors
0

Runner

GitHub
vassiliphilippov
Name
Vassili Philippov
Contact
n/a

Sampling

Temperature
n/a
Top P
n/a
Max tokens
8,192
Seed
n/a
Extra
n/a

Policy

Votes / item
1
Reasks / invalid vote
1
Tie break
earliest_vote
Monosemous
short_circuit

Cost And Token Breakdown

Actual recorded run cost: $6.50. Cost estimate source: provider_reported.

Token Type Tokens Cost Price / 1M tokens
Input uncached 1,122,748 $1.39 n/a
Input cached 128 n/a n/a
Output 2,067,138 $5.18 n/a
Reasoning output 2,112,679 included in output cost n/a

Reasoning output tokens are shown separately when the provider reports them; recorded cost is stored in the output cost bucket.

Download Raw Run Files

Slice Breakdown

Group Bucket Accuracy Correct Items
Candidate Count 11+ 89.29% 709 794
Candidate Count 2 94.85% 811 855
Candidate Count 3-5 91.47% 1779 1945
Candidate Count 6-10 90.69% 1149 1267
POS ADJ 91.81% 605 659
POS ADV 87.18% 170 195
POS NOUN 92.16% 2670 2897
POS VERB 90.36% 1003 1110
Source Dataset semeval2013 92.37% 1053 1140
Source Dataset semeval2015 92.29% 682 739
Source Dataset senseval2 91.28% 1487 1629
Source Dataset senseval3 90.61% 1226 1353

Worst-Bucket Examples

there Adv

Instance: semeval2015.d001.s032.t000 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

There you have a list of the declared variables.

Show raw prompt

user

Which numbered sense of "there" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: <t>There</t> you have a list of the declared variables.

Senses:
1. definition=in or at that place | examples=they have lived there for years
2. definition=in that matter | examples=I agree with you there
3. definition=to or toward that place; away from the speaker | examples=go there around noon!

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    in or at that place

    there%4:02:00:: · there.r.01

    • they have lived there for years
    • it's not there

    Synonyms: at that place, in that location

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    in that matter

    there%4:02:02:: · there.r.02

    • I agree with you there

    Synonyms: in that respect, on that point

    Selected
  3. 3

    to or toward that place; away from the speaker

    there%4:02:01:: · there.r.03

    • go there around noon!

    Synonyms: thither

only Adv

Instance: semeval2015.d003.s019.t003 Dataset: semeval2015 Bucket: POS / ADV

Incorrect

Also, the treatment of vomiting should only be together with other supportive measure or other veterinary therapy while addressing the underlying causes of the vomiting.

Show raw prompt

user

Which numbered sense of "only" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Also, the treatment of vomiting should <t>only</t> be together with other supportive measure or other veterinary therapy while addressing the underlying causes of the vomiting.

Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=without any others being included or involved | examples=was entirely to blame
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    only%4:02:02:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, just, but

  2. 2

    without any others being included or involved

    only%4:02:01:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, alone

    Gold lexEN Gold Maru 2022 Gold Raganato
  3. 3

    with nevertheless the final result

    only%4:02:00:: · only.r.03

    • He arrived only to find his wife dead
    • We won only to lose again in the next round
  4. 4

    in the final outcome

    only%4:02:04:: · only.r.04

    • These news will only make you more upset
  5. 5

    except that

    only%4:02:06:: · only.r.05

    • It was the same story; only this time she came out better
  6. 6

    never except when

    only%4:02:03:: · only.r.06

    • call me only if your cold gets worse

    Synonyms: only if, only when

    Selected
  7. 7

    as recently as

    only%4:02:05:: · only.r.07

    • I spoke to him only an hour ago

only Adv

Instance: senseval2.d000.s008.t001 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Now, only one local ringer remains: 64-year-old Derek Hammond.

Show raw prompt

user

Which numbered sense of "only" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Now, <t>only</t> one local ringer remains: 64-year-old Derek Hammond.

Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=without any others being included or involved | examples=was entirely to blame
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    only%4:02:02:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, just, but

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    without any others being included or involved

    only%4:02:01:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, alone

    Selected
  3. 3

    with nevertheless the final result

    only%4:02:00:: · only.r.03

    • He arrived only to find his wife dead
    • We won only to lose again in the next round
  4. 4

    in the final outcome

    only%4:02:04:: · only.r.04

    • These news will only make you more upset
  5. 5

    except that

    only%4:02:06:: · only.r.05

    • It was the same story; only this time she came out better
  6. 6

    never except when

    only%4:02:03:: · only.r.06

    • call me only if your cold gets worse

    Synonyms: only if, only when

  7. 7

    as recently as

    only%4:02:05:: · only.r.07

    • I spoke to him only an hour ago

just Adv

Instance: senseval2.d000.s012.t005 Dataset: senseval2 Bucket: POS / ADV

Incorrect

We've tried to train the youngsters, but they have their discos and their dances, and they just drift away. "

Show raw prompt

user

Which numbered sense of "just" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: We've tried to train the youngsters, but they have their discos and their dances, and they <t>just</t> drift away. "

Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=indicating exactness or preciseness | examples=he was doing precisely (or exactly) what she had told him to do
3. definition=only a moment ago | examples=he has just arrived
4. definition=absolutely | examples=I just can't take it anymore
5. definition=only a very short time before; ; ; ; ; - W.B.Yeats | examples=they could barely hear the speaker
6. definition=exactly at this moment or the moment described | examples=we've just finished painting the walls, so don't touch them

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    just%4:02:00:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, only, but

    Selected
  2. 2

    indicating exactness or preciseness

    just%4:02:05:: · precisely.r.01

    • he was doing precisely (or exactly) what she had told him to do
    • it was just as he said--the jewel was gone

    Synonyms: precisely, exactly

  3. 3

    only a moment ago

    just%4:02:01:: · just.r.03

    • he has just arrived
    • the sun just now came out

    Synonyms: just now

  4. 4

    absolutely

    just%4:02:03:: · just.r.04

    • I just can't take it anymore
    • he was just grand as Romeo

    Synonyms: simply

    Gold lexEN Gold Maru 2022 Gold Raganato
  5. 5

    only a very short time before; ; ; ; ; - W.B.Yeats

    just%4:02:04:: · barely.r.01

    • they could barely hear the speaker
    • we hardly knew them

    Synonyms: barely, hardly, scarcely, scarce

  6. 6

    exactly at this moment or the moment described

    just%4:02:06:: · just.r.06

    • we've just finished painting the walls, so don't touch them

away Adv

Instance: senseval2.d000.s012.t007 Dataset: senseval2 Bucket: POS / ADV

Incorrect

We've tried to train the youngsters, but they have their discos and their dances, and they just drift away. "

Show raw prompt

user

Which numbered sense of "away" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: We've tried to train the youngsters, but they have their discos and their dances, and they just drift <t>away</t>. "

Senses:
1. definition=from a particular thing or place or position (`forth' is obsolete) | examples=ran away from the lion
2. definition=from one's possession | examples=he gave out money to the poor
3. definition=out of the way (especially away from one's thoughts) | examples=brush the objections aside
4. definition=out of existence; ; - H.E.Scudder | examples=the music faded away
5. definition=at a distance in space or time | examples=the boat was 5 miles off (or away)
6. definition=indicating continuing action; continuously or steadily | examples=he worked away at the project for more than a year
7. definition=so as to be removed or gotten rid of | examples=cleared the mess away
8. definition=freely or at will | examples=fire away!
9. definition=in or into a proper place (especially for storage or safekeeping) | examples=put the toys away
10. definition=in a different direction | examples=turn aside
11. definition=in reserve; not for immediate use | examples=started setting aside money to buy a car

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    from a particular thing or place or position (`forth' is obsolete)

    away%4:02:00:: · away.r.01

    • ran away from the lion
    • wanted to get away from there

    Synonyms: off, forth

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    from one's possession

    away%4:02:01:: · away.r.02

    • he gave out money to the poor
    • gave away the tickets

    Synonyms: out

  3. 3

    out of the way (especially away from one's thoughts)

    away%4:02:03:: · aside.r.02

    • brush the objections aside
    • pushed all doubts away

    Synonyms: aside

  4. 4

    out of existence; ; - H.E.Scudder

    away%4:02:04:: · away.r.04

    • the music faded away
    • tried to explain away the affair of the letter
    Selected
  5. 5

    at a distance in space or time

    away%4:02:08:: · off.r.02

    • the boat was 5 miles off (or away)
    • the party is still 2 weeks off (or away)

    Synonyms: off

  6. 6

    indicating continuing action; continuously or steadily

    away%4:02:05:: · away.r.06

    • he worked away at the project for more than a year
    • the child kept hammering away as if his life depended on it
  7. 7

    so as to be removed or gotten rid of

    away%4:02:09:: · away.r.07

    • cleared the mess away
    • the rotted wood had to be cut away
  8. 8

    freely or at will

    away%4:02:10:: · away.r.08

    • fire away!
  9. 9

    in or into a proper place (especially for storage or safekeeping)

    away%4:02:07:: · away.r.09

    • put the toys away
    • her jewels are locked away in a safe
  10. 10

    in a different direction

    away%4:02:06:: · away.r.10

    • turn aside
    • turn away one's face

    Synonyms: aside

  11. 11

    in reserve; not for immediate use

    away%4:02:02:: · aside.r.06

    • started setting aside money to buy a car
    • put something by for her old age

    Synonyms: aside, by

only Adv

Instance: senseval2.d000.s017.t013 Dataset: senseval2 Bucket: POS / ADV

Incorrect

The less complicated version of playing tunes on bells, as do the carillons of continental Europe, is considered by the English to be childish, fit only for foreigners.

Show raw prompt

user

Which numbered sense of "only" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: The less complicated version of playing tunes on bells, as do the carillons of continental Europe, is considered by the English to be childish, fit <t>only</t> for foreigners.

Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=without any others being included or involved | examples=was entirely to blame
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    and nothing more

    only%4:02:02:: · merely.r.01

    • I was merely asking
    • it is simply a matter of time

    Synonyms: merely, simply, just, but

    Selected
  2. 2

    without any others being included or involved

    only%4:02:01:: · entirely.r.02

    • was entirely to blame
    • a school devoted entirely to the needs of problem children

    Synonyms: entirely, exclusively, solely, alone

    Gold lexEN Gold Maru 2022 Gold Raganato
  3. 3

    with nevertheless the final result

    only%4:02:00:: · only.r.03

    • He arrived only to find his wife dead
    • We won only to lose again in the next round
  4. 4

    in the final outcome

    only%4:02:04:: · only.r.04

    • These news will only make you more upset
  5. 5

    except that

    only%4:02:06:: · only.r.05

    • It was the same story; only this time she came out better
  6. 6

    never except when

    only%4:02:03:: · only.r.06

    • call me only if your cold gets worse

    Synonyms: only if, only when

  7. 7

    as recently as

    only%4:02:05:: · only.r.07

    • I spoke to him only an hour ago

straight Adv

Instance: senseval2.d000.s027.t004 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Totally absorbed, the ringers stare straight ahead, using peripheral vision (they call it "rope-sight ") to watch the other ropes and thus time their pulls.

Show raw prompt

user

Which numbered sense of "straight" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Totally absorbed, the ringers stare <t>straight</t> ahead, using peripheral vision (they call it "rope-sight ") to watch the other ropes and thus time their pulls.

Senses:
1. definition=without deviation | examples=the path leads directly to the lake
2. definition=in a forthright manner; candidly or frankly | examples=he didn't answer directly
3. definition=in a straight line; in a direct course | examples=the road runs straight

Prediction matches gold: lexEN ✗ Maru 2022 ✓ Raganato ✗

WordNet 3.0 senses

  1. 1

    without deviation

    straight%4:02:05:: · directly.r.01

    • the path leads directly to the lake
    • went direct to the office

    Synonyms: directly, direct

    Gold lexEN Gold Raganato
  2. 2

    in a forthright manner; candidly or frankly

    straight%4:02:04:: · directly.r.04

    • he didn't answer directly
    • told me straight out

    Synonyms: directly, flat

  3. 3

    in a straight line; in a direct course

    straight%4:02:00:: · straight.r.03

    • the road runs straight
    Gold Maru 2022 Selected

ahead Adv

Instance: senseval2.d000.s027.t005 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Totally absorbed, the ringers stare straight ahead, using peripheral vision (they call it "rope-sight ") to watch the other ropes and thus time their pulls.

Show raw prompt

user

Which numbered sense of "ahead" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Totally absorbed, the ringers stare straight <t>ahead</t>, using peripheral vision (they call it "rope-sight ") to watch the other ropes and thus time their pulls.

Senses:
1. definition=at or in the front | examples=I see the lights of a town ahead
2. definition=toward the future; forward in time | examples=I like to look ahead in imagination to what the future may bring
3. definition=in a forward direction | examples=go ahead
4. definition=ahead of time; in anticipation | examples=when you pay ahead (or in advance) you receive a discount
5. definition=to a more advanced or advantageous position | examples=a young man sure to get ahead
6. definition=to a different or a more advanced time (meaning advanced either toward the present or toward the future) | examples=moved the appointment ahead from Tuesday to Monday
7. definition=leading or ahead in a competition | examples=the horse was three lengths ahead going into the home stretch

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    at or in the front

    ahead%4:02:00:: · ahead.r.01

    • I see the lights of a town ahead
    • the road ahead is foggy

    Synonyms: in front, before

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    toward the future; forward in time

    ahead%4:02:06:: · ahead.r.02

    • I like to look ahead in imagination to what the future may bring
    • I look forward to seeing you

    Synonyms: forward

  3. 3

    in a forward direction

    ahead%4:02:02:: · ahead.r.03

    • go ahead
    • the train moved ahead slowly

    Synonyms: onward, onwards, forward, forwards, forrader

    Selected
  4. 4

    ahead of time; in anticipation

    ahead%4:02:01:: · ahead.r.04

    • when you pay ahead (or in advance) you receive a discount
    • We like to plan ahead

    Synonyms: in advance, beforehand

  5. 5

    to a more advanced or advantageous position

    ahead%4:02:05:: · ahead.r.05

    • a young man sure to get ahead
    • pushing talented students ahead
  6. 6

    to a different or a more advanced time (meaning advanced either toward the present or toward the future)

    ahead%4:02:04:: · ahead.r.06

    • moved the appointment ahead from Tuesday to Monday
    • pushed the deadline ahead from Tuesday to Wednesday
  7. 7

    leading or ahead in a competition

    ahead%4:02:03:: · ahead.r.07

    • the horse was three lengths ahead going into the home stretch
    • ahead by two pawns

    Synonyms: out front, in the lead

well Adv

Instance: senseval2.d000.s036.t000 Dataset: senseval2 Bucket: POS / ADV

Incorrect

This does not sit well with some clerics.

Show raw prompt

user

Which numbered sense of "well" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: This does not sit <t>well</t> with some clerics.

Senses:
1. definition=(often used as a combining form) in a good or proper or satisfactory manner or to a high standard (`good' is a nonstandard dialectal variant for `well') | examples=the children behaved well
2. definition=thoroughly or completely; fully; often used as a combining form; ; ; ; ; , | examples=The problem is well understood
3. definition=indicating high probability; in all likelihood | examples=I might well do it
4. definition=(used for emphasis or as an intensifier) entirely or fully | examples=a book well worth reading
5. definition=to a suitable or appropriate extent or degree | examples=the project was well underway
6. definition=favorably; with approval | examples=their neighbors spoke well of them
7. definition=to a great extent or degree | examples=I'm afraid the film was well over budget
8. definition=with great or especially intimate knowledge | examples=we knew them well
9. definition=with prudence or propriety | examples=You would do well to say nothing more
10. definition=with skill or in a pleasing manner | examples=she dances well
11. definition=in a manner affording benefit or advantage | examples=she married well
12. definition=in financial comfort | examples=They live well
13. definition=without unusual distress or resentment; with good humor | examples=took the joke well

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    (often used as a combining form) in a good or proper or satisfactory manner or to a high standard (`good' is a nonstandard dialectal variant for `well')

    well%4:02:00:: · well.r.01

    • the children behaved well
    • a task well done

    Synonyms: good

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    thoroughly or completely; fully; often used as a combining form; ; ; ; ; ,

    well%4:02:05:: · well.r.02

    • The problem is well understood
    • she was well informed
  3. 3

    indicating high probability; in all likelihood

    well%4:02:03:: · well.r.03

    • I might well do it
    • a mistake that could easily have ended in disaster

    Synonyms: easily

  4. 4

    (used for emphasis or as an intensifier) entirely or fully

    well%4:02:15:: · well.r.04

    • a book well worth reading
    • was well aware of the difficulties ahead
  5. 5

    to a suitable or appropriate extent or degree

    well%4:02:08:: · well.r.05

    • the project was well underway
    • the fetus has well developed organs
  6. 6

    favorably; with approval

    well%4:02:07:: · well.r.06

    • their neighbors spoke well of them
    • he thought well of the book
    Selected
  7. 7

    to a great extent or degree

    well%4:02:11:: · well.r.07

    • I'm afraid the film was well over budget
    • painting the room white made it seem considerably (or substantially) larger

    Synonyms: considerably, substantially

  8. 8

    with great or especially intimate knowledge

    well%4:02:14:: · well.r.08

    • we knew them well

    Synonyms: intimately

  9. 9

    with prudence or propriety

    well%4:02:13:: · well.r.09

    • You would do well to say nothing more
    • could not well refuse
  10. 10

    with skill or in a pleasing manner

    well%4:02:12:: · well.r.10

    • she dances well
    • he writes well
  11. 11

    in a manner affording benefit or advantage

    well%4:02:10:: · well.r.11

    • she married well
    • The children were settled advantageously in Seattle

    Synonyms: advantageously

  12. 12

    in financial comfort

    well%4:02:09:: · well.r.12

    • They live well
    • she has been able to live comfortably since her husband died

    Synonyms: comfortably

  13. 13

    without unusual distress or resentment; with good humor

    well%4:02:01:: · well.r.13

    • took the joke well
    • took the tragic news well

promptly Adv

Instance: senseval2.d000.s038.t012 Dataset: senseval2 Bucket: POS / ADV

Incorrect

Two years ago, the Rev. Jeremy Hummerstone, vicar of Great Torrington, Devon, got so fed up with ringers who didn't attend service he sacked the entire band; the ringers promptly set up a picket line in protest.

Show raw prompt

user

Which numbered sense of "promptly" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Two years ago, the Rev. Jeremy Hummerstone, vicar of Great Torrington, Devon, got so fed up with ringers who didn't attend service he sacked the entire band; the ringers <t>promptly</t> set up a picket line in protest.

Senses:
1. definition=with little or no delay | examples=the rescue squad arrived promptly
2. definition=in a punctual manner | examples=he did his homework promptly
3. definition=at once (usually modifies an undesirable occurrence) | examples=he promptly forgot the address

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    with little or no delay

    promptly%4:02:02:: · promptly.r.01

    • the rescue squad arrived promptly
    • come here, quick!

    Synonyms: quickly, quick

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    in a punctual manner

    promptly%4:02:00:: · promptly.r.02

    • he did his homework promptly

    Synonyms: readily, pronto

  3. 3

    at once (usually modifies an undesirable occurrence)

    promptly%4:02:01:: · promptly.r.03

    • he promptly forgot the address

    Synonyms: right away

    Selected

still Adv

Instance: senseval2.d000.s041.t000 Dataset: senseval2 Bucket: POS / ADV

Incorrect

But there still are not enough ringers to ring more than six of the eight bells.

Show raw prompt

user

Which numbered sense of "still" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: But there <t>still</t> are not enough ringers to ring more than six of the eight bells.

Senses:
1. definition=with reference to action or condition; without change, interruption, or cessation | examples=it's still warm outside
2. definition=despite anything to the contrary (usually following a concession) | examples=although I'm a little afraid, however I'd like to try it
3. definition=to a greater degree or extent; used with comparisons | examples=looked sick and felt even worse
4. definition=without moving or making a sound | examples=he sat still as a statue

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    with reference to action or condition; without change, interruption, or cessation

    still%4:02:01:: · still.r.01

    • it's still warm outside
    • will you still love me when we're old and grey?
    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    despite anything to the contrary (usually following a concession)

    still%4:02:04:: · however.r.01

    • although I'm a little afraid, however I'd like to try it
    • while we disliked each other, nevertheless we agreed

    Synonyms: however, nevertheless, withal, yet, all the same, even so, nonetheless, notwithstanding

    Selected
  3. 3

    to a greater degree or extent; used with comparisons

    still%4:02:00:: · even.r.03

    • looked sick and felt even worse
    • an even (or still) more interesting problem

    Synonyms: even, yet

  4. 4

    without moving or making a sound

    still%4:02:02:: · still.r.04

    • he sat still as a statue
    • time stood still

    Synonyms: stock-still

so Adv

Instance: senseval2.d000.s045.t000 Dataset: senseval2 Bucket: POS / ADV

Incorrect

"So crunch, crunch, crunch, bang, bang, bang -- here come the ringers from above, making a very obvious exit while the congregation is at prayer, "he says.

Show raw prompt

user

Which numbered sense of "so" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: "<t>So</t> crunch, crunch, crunch, bang, bang, bang -- here come the ringers from above, making a very obvious exit while the congregation is at prayer, "he says.

Senses:
1. definition=to a very great extent or degree | examples=the idea is so obvious
2. definition=in a manner that facilitates | examples=he observed the snakes so he could describe their behavior
3. definition=in such a condition or manner, especially as expressed or implied | examples=They're happy and I hope they will remain so
4. definition=to a certain unspecified extent or degree | examples=I can only go so far with this student
5. definition=in the same way; also | examples=I was offended and so was he
6. definition=in the way indicated; ; ; (`thusly' is a nonstandard variant) | examples=hold the brush so
7. definition=(usually followed by `that') to an extent or degree as expressed | examples=he was so tired he could hardly stand
8. definition=subsequently or soon afterward (often used as sentence connectors) | examples=then he left
9. definition=(used to introduce a logical conclusion) from that fact or reason or as a result | examples=therefore X must be true
10. definition=in truth (often tends to intensify) | examples=they said the car would break down and indeed it did

Prediction matches gold: lexEN ✗ Maru 2022 ✓ Raganato ✓

WordNet 3.0 senses

  1. 1

    to a very great extent or degree

    so%4:02:02:: · so.r.01

    • the idea is so obvious
    • never been so happy
  2. 2

    in a manner that facilitates

    so%4:02:06:: · so.r.02

    • he observed the snakes so he could describe their behavior
    • he stooped down so he could pick up his hat
  3. 3

    in such a condition or manner, especially as expressed or implied

    so%4:02:05:: · so.r.03

    • They're happy and I hope they will remain so
    • so live your life that old age will bring no regrets
  4. 4

    to a certain unspecified extent or degree

    so%4:02:07:: · so.r.04

    • I can only go so far with this student
    • can do only so much in a day
  5. 5

    in the same way; also

    so%4:02:08:: · so.r.05

    • I was offended and so was he
    • worked hard and so did she
  6. 6

    in the way indicated; ; ; (`thusly' is a nonstandard variant)

    so%4:02:04:: · thus.r.02

    • hold the brush so
    • set up the pieces thus

    Synonyms: thus, thusly

  7. 7

    (usually followed by `that') to an extent or degree as expressed

    so%4:02:03:: · so.r.07

    • he was so tired he could hardly stand
    • so dirty that it smells
  8. 8

    subsequently or soon afterward (often used as sentence connectors)

    so%4:02:09:: · then.r.01

    • then he left
    • go left first, then right

    Synonyms: then, and so, and then

    Gold Maru 2022 Gold Raganato Selected
  9. 9

    (used to introduce a logical conclusion) from that fact or reason or as a result

    so%4:02:10:: · therefore.r.01

    • therefore X must be true
    • the eggs were fresh and hence satisfactory

    Synonyms: therefore, hence, thence, thus

    Gold lexEN
  10. 10

    in truth (often tends to intensify)

    so%4:02:00:: · indeed.r.01

    • they said the car would break down and indeed it did
    • it is very cold indeed

    Synonyms: indeed