Verified run

claude-haiku-4-5

claude-haiku-4.5-low-reasoning-p002-lexen-v1-20260614

Rank
84
Accuracy lexEN v1 · WordNet fine
87.60%
Correct
4258 / 4861
Actual run cost
$9.71
Cost / M items
$1,997

Accuracy across label schemes

The headline accuracy is lexEN v1 · WordNet fine-grained. Each cell re-scores this run's predictions against a different gold label set and sense granularity; coarse counts a prediction correct when its WordNet sense maps to the same Glite or CSI concept as a gold sense. About these label schemes → About coarsening →

Score against WordNet fine-grainedGlite coarse-grainedCSI coarse-grained (Lacerra 2020)
lexEN v1 87.60% 4258 / 4861 95.19% 4627 / 4861 91.69% 4457 / 4861
Maru 2022 (ALLamended) 85.74% 4168 / 4861 94.63% 4600 / 4861 90.56% 4402 / 4861
Raganato 2017 (original) 80.13% 3895 / 4861 92.35% 4489 / 4861 86.73% 4216 / 4861

Model

Display
claude-haiku-4-5
Requested
claude-haiku-4-5
Resolved
claude-haiku-4-5-20251001
Vendor
Anthropic
API provider
Anthropic
Hosting
cloud_api
Source
proprietary
License
n/a
Reasoning
low
Model URL
n/a
Endpoint
n/a
Resolved counts
claude-haiku-4-5-20251001: 4992

Reproducibility

Created
2026-06-14T11:17:37.678627+00:00
Run schema
sensebench-run-v2
Commit
415124bb2edc42a803a928d665ca11733af26360
Dataset ID
lexen
Dataset version
lexen-v1
Dataset revision
n/a
Dataset items
4,861
Dataset hash
sha256:5fd4382b93f19087a1e31f6dd7d1db17c1eb17ff80fcbe1d3fdd55c0c3ecefe8
Prompt
p002
Prompt version
0.1.0
Repository files
Browse in GitHub

Runtime

Calls
4992
Total tokens
4,547,474
Tokens / item
935.5
Total cost
$9.71
Cost / M items
$1,997
Cost source
litellm_estimate
Elapsed
19.03m
Success items
4,861
Monosemous
0
No candidates
0
No valid vote
0
Invalid votes
0
Transport errors
0

Runner

GitHub
vassiliphilippov
Name
Vassili Philippov
Contact
n/a

Sampling

Temperature
n/a
Top P
n/a
Max tokens
8,192
Seed
n/a
Extra
n/a

Policy

Votes / item
1
Reasks / invalid vote
1
Tie break
earliest_vote
Monosemous
short_circuit

Cost And Token Breakdown

Actual recorded run cost: $9.71. Cost estimate source: litellm_estimate.

Token Type Tokens Cost Price / 1M tokens
Input uncached 1,378,426 $1.38 $1.00
Input cached 0 $0.00 $0.100
Output 1,665,326 $8.33 $5.00
Reasoning output 1,503,722 included in output cost n/a

Reasoning output tokens are shown separately when the provider reports them; recorded cost is stored in the output cost bucket.

Download Raw Run Files

Slice Breakdown

Group Bucket Accuracy Correct Items
Candidate Count 11+ 82.62% 656 794
Candidate Count 2 93.92% 803 855
Candidate Count 3-5 88.95% 1730 1945
Candidate Count 6-10 84.37% 1069 1267
POS ADJ 89.83% 592 659
POS ADV 83.08% 162 195
POS NOUN 89.47% 2592 2897
POS VERB 82.16% 912 1110
Source Dataset semeval2013 90.09% 1027 1140
Source Dataset semeval2015 87.28% 645 739
Source Dataset senseval2 86.86% 1415 1629
Source Dataset senseval3 86.55% 1171 1353

Worst-Bucket Examples

take_advantage Verb

Instance: semeval2013.d006.s012.t004 Dataset: semeval2013 Bucket: POS / VERB

Incorrect

Another of these advantages to which it is alluded to with the fare-play is to avoid that the team of the lawbreaker players takes advantage unjustly of the opposite team, which did not happen either.

Show raw prompt

user

Which numbered sense of "take_advantage" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Another of these advantages to which it is alluded to with the fare-play is to avoid that the team of the lawbreaker players <t>takes advantage</t> unjustly of the opposite team, which did not happen either.

Senses:
1. definition=draw advantages from | examples=he is capitalizing on her mistake
2. definition=make excessive use of | examples=You are taking advantage of my good will!

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    draw advantages from

    take_advantage%2:40:00:: · capitalize.v.01

    • he is capitalizing on her mistake
    • she took advantage of his absence to meet her lover

    Synonyms: capitalize, capitalise

    Gold lexEN Gold Maru 2022
  2. 2

    make excessive use of

    take_advantage%2:41:12:: · trespass.v.02

    • You are taking advantage of my good will!
    • She is trespassing upon my privacy

    Synonyms: trespass

    Selected

reach Verb

Instance: semeval2015.d000.s001.t009 Dataset: semeval2015 Bucket: POS / VERB

Incorrect

It explains how the Committee for Medicinal Products for Human Use (CHMP) assessed the studies performed, to reach their recommendations on how to use the medicine.

Show raw prompt

user

Which numbered sense of "reach" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: It explains how the Committee for Medicinal Products for Human Use (CHMP) assessed the studies performed, to <t>reach</t> their recommendations on how to use the medicine.

Senses:
1. definition=reach a destination, either real or abstract | examples=We hit Detroit by noon
2. definition=reach a point in time, or a certain state or level | examples=The thermometer hit 100 degrees
3. definition=move forward or upward in order to touch; also in a metaphorical sense | examples=Government reaches out to the people
4. definition=be in or establish communication with | examples=Our advertisements reach millions
5. definition=to gain with effort | examples=she achieved her goal despite setbacks
6. definition=to extend as far as | examples=The sunlight reached the wall
7. definition=reach a goal, e.g., | examples=make the first team
8. definition=place into the hands or custody of | examples=hand me the spoon, please
9. definition=to exert much effort or energy | examples=straining our ears to hear

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    reach a destination, either real or abstract

    reach%2:38:01:: · reach.v.01

    • We hit Detroit by noon
    • The water reached the doorstep

    Synonyms: make, attain, hit, arrive at, gain

    Selected
  2. 2

    reach a point in time, or a certain state or level

    reach%2:38:00:: · reach.v.02

    • The thermometer hit 100 degrees
    • This car can reach a speed of 140 miles per hour

    Synonyms: hit, attain

  3. 3

    move forward or upward in order to touch; also in a metaphorical sense

    reach%2:35:00:: · reach.v.03

    • Government reaches out to the people

    Synonyms: reach out

  4. 4

    be in or establish communication with

    reach%2:32:00:: · reach.v.04

    • Our advertisements reach millions
    • He never contacted his children after he emigrated to Australia

    Synonyms: get through, get hold of, contact

  5. 5

    to gain with effort

    reach%2:41:00:: · achieve.v.01

    • she achieved her goal despite setbacks

    Synonyms: achieve, accomplish, attain

    Gold lexEN Gold Maru 2022 Gold Raganato
  6. 6

    to extend as far as

    reach%2:42:00:: · reach.v.06

    • The sunlight reached the wall
    • Can he reach?

    Synonyms: extend to, touch

  7. 7

    reach a goal, e.g.,

    reach%2:38:06:: · reach.v.07

    • make the first team
    • We made it!

    Synonyms: make, get to, progress to

    Gold lexEN Gold Maru 2022 Gold Raganato
  8. 8

    place into the hands or custody of

    reach%2:40:00:: · pass.v.05

    • hand me the spoon, please
    • Turn the files over to me, please

    Synonyms: pass, hand, pass on, turn over, give

  9. 9

    to exert much effort or energy

    reach%2:33:00:: · strive.v.02

    • straining our ears to hear

    Synonyms: strive, strain

prevent Verb

Instance: semeval2015.d000.s017.t005 Dataset: semeval2015 Bucket: POS / VERB

Incorrect

When Alimta is given with cisplatin, an anti-emetic medicine (to prevent vomiting) and fluids (to prevent dehydration) should also be given before or after the cisplatin dose.

Show raw prompt

user

Which numbered sense of "prevent" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: When Alimta is given with cisplatin, an anti-emetic medicine (to prevent vomiting) and fluids (to <t>prevent</t> dehydration) should also be given before or after the cisplatin dose.

Senses:
1. definition=keep from happening or arising; make impossible | examples=My sense of tact forbids an honest answer
2. definition=stop (someone or something) from doing something or being in a certain state | examples=We must prevent the cancer from spreading

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✓

WordNet 3.0 senses

  1. 1

    keep from happening or arising; make impossible

    prevent%2:41:01:: · prevent.v.01

    • My sense of tact forbids an honest answer
    • Your role in the projects precludes your involvement in the competitive project

    Synonyms: forestall, foreclose, preclude, forbid

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    stop (someone or something) from doing something or being in a certain state

    prevent%2:41:00:: · prevent.v.02

    • We must prevent the cancer from spreading
    • His snoring kept me from falling asleep

    Synonyms: keep

    Gold Raganato Selected

delay Verb

Instance: semeval2015.d000.s018.t001 Dataset: semeval2015 Bucket: POS / VERB

Incorrect

Treatment should be delayed or discontinued, or the dose reduced, in patients whose blood counts are abnormal or who have certain other side effects.

Show raw prompt

user

Which numbered sense of "delay" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Treatment should be <t>delayed</t> or discontinued, or the dose reduced, in patients whose blood counts are abnormal or who have certain other side effects.

Senses:
1. definition=cause to be slowed down or delayed | examples=Traffic was delayed by the bad weather
2. definition=act later than planned, scheduled, or required | examples=Don't delay your application to graduate school or else it won't be considered
3. definition=stop or halt | examples=Please stay the bloodshed!
4. definition=slow the growth or development of | examples=The brain damage will retard the child's language development

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✓

WordNet 3.0 senses

  1. 1

    cause to be slowed down or delayed

    delay%2:30:00:: · delay.v.01

    • Traffic was delayed by the bad weather
    • she delayed the work that she didn't want to perform

    Synonyms: detain, hold up

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    act later than planned, scheduled, or required

    delay%2:42:00:: · delay.v.02

    • Don't delay your application to graduate school or else it won't be considered
    Gold Raganato Selected
  3. 3

    stop or halt

    delay%2:30:01:: · stay.v.06

    • Please stay the bloodshed!

    Synonyms: stay, detain

  4. 4

    slow the growth or development of

    delay%2:30:05:: · check.v.07

    • The brain damage will retard the child's language development

    Synonyms: check, retard

have Verb

Instance: semeval2015.d000.s018.t008 Dataset: semeval2015 Bucket: POS / VERB

Incorrect

Treatment should be delayed or discontinued, or the dose reduced, in patients whose blood counts are abnormal or who have certain other side effects.

Show raw prompt

user

Which numbered sense of "have" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Treatment should be delayed or discontinued, or the dose reduced, in patients whose blood counts are abnormal or who <t>have</t> certain other side effects.

Senses:
1. definition=have or possess, either in a concrete or an abstract sense | examples=She has $1,000 in the bank
2. definition=have as a feature | examples=This restaurant features the most famous chefs in France
3. definition=go through (mental or physical states or experiences) | examples=get an idea
4. definition=have ownership or possession of | examples=He owns three houses in Florida
5. definition=cause to move; cause to be in a certain position or condition | examples=He got his squad on the ball
6. definition=serve oneself to, or consume regularly | examples=Have another bowl of chicken soup!
7. definition=have a personal or business relationship with someone | examples=have a postdoc
8. definition=organize or be responsible for | examples=hold a reception
9. definition=have left | examples=I have two years left
10. definition=be confronted with | examples=What do we have here?
11. definition=undergo | examples=The stocks had a fast run-up
12. definition=suffer from; be ill with | examples=She has arthritis
13. definition=cause to do; cause to act in a specified manner | examples=The ads induced me to buy a VCR
14. definition=receive willingly something given or offered | examples=The only girl who would have him was the miller's daughter
15. definition=get something; come into possession of | examples=receive payment
16. definition=undergo (as of injuries and illnesses) | examples=She suffered a fracture in the accident
17. definition=achieve a point or goal | examples=Nicklaus had a 70
18. definition=cause to be born | examples=My wife had twins yesterday!
19. definition=have sex with; archaic use | examples=He had taken this woman when she was most vulnerable

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✓

WordNet 3.0 senses

  1. 1

    have or possess, either in a concrete or an abstract sense

    have%2:40:00:: · have.v.01

    • She has $1,000 in the bank
    • He has got two beautiful daughters

    Synonyms: have got, hold

  2. 2

    have as a feature

    have%2:42:00:: · have.v.02

    • This restaurant features the most famous chefs in France

    Synonyms: feature

  3. 3

    go through (mental or physical states or experiences)

    have%2:39:06:: · experience.v.03

    • get an idea
    • experience vertigo

    Synonyms: experience, receive, get

    Gold lexEN Gold Maru 2022
  4. 4

    have ownership or possession of

    have%2:40:04:: · own.v.01

    • He owns three houses in Florida
    • How many cars does she have?

    Synonyms: own, possess

  5. 5

    cause to move; cause to be in a certain position or condition

    have%2:30:00:: · get.v.03

    • He got his squad on the ball
    • This let me in for a big surprise

    Synonyms: get, let

  6. 6

    serve oneself to, or consume regularly

    have%2:34:00:: · consume.v.02

    • Have another bowl of chicken soup!
    • I don't take sugar in my coffee

    Synonyms: consume, ingest, take in, take

  7. 7

    have a personal or business relationship with someone

    have%2:41:00:: · have.v.07

    • have a postdoc
    • have an assistant
  8. 8

    organize or be responsible for

    have%2:36:00:: · hold.v.03

    • hold a reception
    • have, throw, or make a party

    Synonyms: hold, throw, make, give

  9. 9

    have left

    have%2:40:01:: · have.v.09

    • I have two years left
    • I don't have any money left
  10. 10

    be confronted with

    have%2:42:12:: · have.v.10

    • What do we have here?
    • Now we have a fine mess
  11. 11

    undergo

    have%2:30:01:: · have.v.11

    • The stocks had a fast run-up

    Synonyms: experience

    Gold lexEN Gold Maru 2022
  12. 12

    suffer from; be ill with

    have%2:29:02:: · have.v.12

    • She has arthritis
    Gold Raganato Selected
  13. 13

    cause to do; cause to act in a specified manner

    have%2:32:00:: · induce.v.02

    • The ads induced me to buy a VCR
    • My children finally got me to buy a computer

    Synonyms: induce, stimulate, cause, get, make

  14. 14

    receive willingly something given or offered

    have%2:40:05:: · accept.v.02

    • The only girl who would have him was the miller's daughter
    • I won't have this dog in my house!

    Synonyms: accept, take

  15. 15

    get something; come into possession of

    have%2:40:03:: · receive.v.01

    • receive payment
    • receive a gift

    Synonyms: receive

  16. 16

    undergo (as of injuries and illnesses)

    have%2:29:05:: · suffer.v.02

    • She suffered a fracture in the accident
    • He had an insulin shock after eating three candy bars

    Synonyms: suffer, sustain, get

    Gold Raganato
  17. 17

    achieve a point or goal

    have%2:40:02:: · have.v.17

    • Nicklaus had a 70
    • The Brazilian team got 4 goals

    Synonyms: get, make

  18. 18

    cause to be born

    have%2:29:00:: · give_birth.v.01

    • My wife had twins yesterday!

    Synonyms: give birth, deliver, bear, birth

  19. 19

    have sex with; archaic use

    have%2:35:00:: · take.v.35

    • He had taken this woman when she was most vulnerable

    Synonyms: take

see Verb

Instance: semeval2015.d000.s020.t002 Dataset: semeval2015 Bucket: POS / VERB

Incorrect

For more information, see the Summary of Product Characteristics (also part of the EPAR).

Show raw prompt

user

Which numbered sense of "see" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: For more information, <t>see</t> the Summary of Product Characteristics (also part of the EPAR).

Senses:
1. definition=perceive by sight or have the power to perceive by sight | examples=You have to be a good observer to see all the details
2. definition=perceive (an idea or situation) mentally | examples=Now I see!
3. definition=perceive or be contemporaneous with | examples=We found Republicans winning the offices
4. definition=imagine; conceive of; see in one's mind | examples=I can't see him on horseback!
5. definition=deem to be | examples=She views this quite differently from me
6. definition=get to know or become aware of, usually accidentally | examples=I learned that she has two grown-up children
7. definition=see or watch | examples=view a show on television
8. definition=come together | examples=I'll probably see you at the meeting
9. definition=find out, learn, or determine with certainty, usually by making an inquiry or other effort | examples=I want to see whether she speaks French
10. definition=be careful or certain to do something; make certain of something | examples=He verified that the valves were closed
11. definition=go to see for professional or business reasons | examples=You should see a lawyer
12. definition=go to see for a social visit | examples=I went to see my friend Mary the other day
13. definition=go to see a place, as for entertainment | examples=We went to see the Eiffel Tower in the morning
14. definition=take charge of or deal with | examples=Could you see about lunch?
15. definition=receive as a specified guest | examples=the doctor will see you now
16. definition=date regularly; have a steady relationship with | examples=Did you know that she is seeing an older man?
17. definition=see and understand, have a good eye | examples=The artist must first learn to see
18. definition=deliberate or decide | examples=See whether you can come tomorrow
19. definition=observe as if with an eye | examples=The camera saw the burglary and recorded it
20. definition=observe, check out, and look over carefully or inspect | examples=The customs agent examined the baggage
21. definition=go or live through | examples=We had many trials to go through
22. definition=accompany or escort | examples=I'll see you to the door
23. definition=match or meet | examples=I saw the bet of one of my fellow players
24. definition=make sense of; assign a meaning to | examples=What message do you see in this letter?

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    perceive by sight or have the power to perceive by sight

    see%2:39:00:: · see.v.01

    • You have to be a good observer to see all the details
    • Can you see the bird in that tree?
  2. 2

    perceive (an idea or situation) mentally

    see%2:31:01:: · understand.v.02

    • Now I see!
    • I just can't see your point

    Synonyms: understand, realize, realise

  3. 3

    perceive or be contemporaneous with

    see%2:39:02:: · witness.v.02

    • We found Republicans winning the offices
    • You'll see a lot of cheating in this school

    Synonyms: witness, find

  4. 4

    imagine; conceive of; see in one's mind

    see%2:36:00:: · visualize.v.01

    • I can't see him on horseback!
    • I can see what will happen

    Synonyms: visualize, visualise, envision, project, fancy, figure, picture, image

  5. 5

    deem to be

    see%2:31:00:: · see.v.05

    • She views this quite differently from me
    • I consider her to be shallow

    Synonyms: consider, reckon, view, regard

  6. 6

    get to know or become aware of, usually accidentally

    see%2:31:02:: · learn.v.02

    • I learned that she has two grown-up children
    • I see that you have been promoted

    Synonyms: learn, hear, get word, get wind, pick up, find out, get a line, discover

  7. 7

    see or watch

    see%2:39:03:: · watch.v.03

    • view a show on television
    • This program will be seen all over the world

    Synonyms: watch, view, catch, take in

  8. 8

    come together

    see%2:38:00:: · meet.v.01

    • I'll probably see you at the meeting
    • How nice to see you again!

    Synonyms: meet, run into, encounter, run across, come across

  9. 9

    find out, learn, or determine with certainty, usually by making an inquiry or other effort

    see%2:32:00:: · determine.v.08

    • I want to see whether she speaks French
    • See whether it works

    Synonyms: determine, check, find out, ascertain, watch, learn

    Selected
  10. 10

    be careful or certain to do something; make certain of something

    see%2:31:03:: · see.v.10

    • He verified that the valves were closed
    • See that the curtains are closed

    Synonyms: check, insure, see to it, ensure, control, ascertain, assure

  11. 11

    go to see for professional or business reasons

    see%2:41:11:: · see.v.11

    • You should see a lawyer
    • We had to see a psychiatrist
  12. 12

    go to see for a social visit

    see%2:41:12:: · see.v.12

    • I went to see my friend Mary the other day
  13. 13

    go to see a place, as for entertainment

    see%2:41:10:: · visit.v.01

    • We went to see the Eiffel Tower in the morning

    Synonyms: visit

  14. 14

    take charge of or deal with

    see%2:41:14:: · attend.v.02

    • Could you see about lunch?
    • I must attend to this matter

    Synonyms: attend, take care, look

  15. 15

    receive as a specified guest

    see%2:35:09:: · see.v.15

    • the doctor will see you now
    • The minister doesn't see anybody before noon
  16. 16

    date regularly; have a steady relationship with

    see%2:41:00:: · go_steady.v.01

    • Did you know that she is seeing an older man?
    • He is dating his former wife again!

    Synonyms: go steady, go out, date

  17. 17

    see and understand, have a good eye

    see%2:39:05:: · see.v.17

    • The artist must first learn to see
  18. 18

    deliberate or decide

    see%2:31:08:: · see.v.18

    • See whether you can come tomorrow
    • let's see--which movie should we see tonight?
  19. 19

    observe as if with an eye

    see%2:39:11:: · see.v.19

    • The camera saw the burglary and recorded it
  20. 20

    observe, check out, and look over carefully or inspect

    see%2:39:13:: · examine.v.02

    • The customs agent examined the baggage
    • I must see your passport before you can enter the country

    Synonyms: examine

    Gold lexEN Gold Maru 2022 Gold Raganato
  21. 21

    go or live through

    see%2:39:12:: · experience.v.01

    • We had many trials to go through
    • he saw action in Viet Nam

    Synonyms: experience, go through

  22. 22

    accompany or escort

    see%2:38:03:: · see.v.22

    • I'll see you to the door

    Synonyms: escort

  23. 23

    match or meet

    see%2:33:00:: · see.v.23

    • I saw the bet of one of my fellow players
  24. 24

    make sense of; assign a meaning to

    see%2:31:13:: · interpret.v.01

    • What message do you see in this letter?
    • How do you interpret his behavior?

    Synonyms: interpret, construe

divide Verb

Instance: semeval2015.d000.s022.t007 Dataset: semeval2015 Bucket: POS / VERB

Incorrect

The active substance in Alimta, pemetrexed, is a cytotoxic medicine (a medicine that kills cells that are dividing, such as cancer cells), which belongs to the group antimetabolites.

Show raw prompt

user

Which numbered sense of "divide" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: The active substance in Alimta, pemetrexed, is a cytotoxic medicine (a medicine that kills cells that are <t>dividing</t>, such as cancer cells), which belongs to the group antimetabolites.

Senses:
1. definition=separate into parts or portions | examples=divide the cake into three equal parts
2. definition=perform a division | examples=Can you divide 49 by seven?
3. definition=act as a barrier between; stand between | examples=The mountain range divides the two countries
4. definition=come apart | examples=The two pieces that we had glued separated
5. definition=make a division or separation
6. definition=force, take, or pull apart | examples=He separated the fighting children

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    separate into parts or portions

    divide%2:41:00:: · divide.v.01

    • divide the cake into three equal parts
    • The British carved up the Ottoman Empire after World War I

    Synonyms: split, split up, separate, dissever, carve up

    Selected
  2. 2

    perform a division

    divide%2:31:00:: · divide.v.02

    • Can you divide 49 by seven?

    Synonyms: fraction

  3. 3

    act as a barrier between; stand between

    divide%2:42:00:: · separate.v.01

    • The mountain range divides the two countries

    Synonyms: separate

  4. 4

    come apart

    divide%2:35:00:: · separate.v.12

    • The two pieces that we had glued separated

    Synonyms: separate, part

    Gold lexEN Gold Maru 2022 Gold Raganato
  5. 5

    make a division or separation

    divide%2:38:00:: · separate.v.07

    Synonyms: separate

    Gold Raganato
  6. 6

    force, take, or pull apart

    divide%2:35:01:: · separate.v.02

    • He separated the fighting children
    • Moses parted the Red Sea

    Synonyms: separate, disunite, part

involve Verb

Instance: semeval2015.d000.s027.t008 Dataset: semeval2015 Bucket: POS / VERB

Incorrect

For the treatment of non-small cell lung cancer, the effects of Alimta were compared with those of docetaxel (another anticancer medicine) in one study involving 571 patients with locally advanced or metastatic disease who had received chemotherapy in the past.

Show raw prompt

user

Which numbered sense of "involve" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: For the treatment of non-small cell lung cancer, the effects of Alimta were compared with those of docetaxel (another anticancer medicine) in one study <t>involving</t> 571 patients with locally advanced or metastatic disease who had received chemotherapy in the past.

Senses:
1. definition=connect closely and often incriminatingly | examples=This new ruling affects your business
2. definition=engage as a participant | examples=Don't involve me in your family affairs!
3. definition=have as a necessary feature | examples=This decision involves many changes
4. definition=require as useful, just, or proper | examples=It takes nerve to do what she did
5. definition=contain as a part | examples=Dinner at Joe's always involves at least six courses
6. definition=occupy or engage the interest of | examples=His story completely involved me during the entire afternoon
7. definition=make complex or intricate or complicated | examples=The situation was rather involved

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    connect closely and often incriminatingly

    involve%2:42:01:: · involve.v.01

    • This new ruling affects your business

    Synonyms: affect, regard

  2. 2

    engage as a participant

    involve%2:42:04:: · involve.v.02

    • Don't involve me in your family affairs!
    Gold lexEN Gold Maru 2022 Gold Raganato
  3. 3

    have as a necessary feature

    involve%2:42:00:: · imply.v.05

    • This decision involves many changes

    Synonyms: imply

  4. 4

    require as useful, just, or proper

    involve%2:42:07:: · necessitate.v.01

    • It takes nerve to do what she did
    • success usually requires hard work

    Synonyms: necessitate, ask, postulate, need, require, take, call for, demand

  5. 5

    contain as a part

    involve%2:42:02:: · involve.v.05

    • Dinner at Joe's always involves at least six courses
    Selected
  6. 6

    occupy or engage the interest of

    involve%2:31:00:: · involve.v.06

    • His story completely involved me during the entire afternoon
  7. 7

    make complex or intricate or complicated

    involve%2:30:00:: · involve.v.07

    • The situation was rather involved

involve Verb

Instance: semeval2015.d000.s028.t006 Dataset: semeval2015 Bucket: POS / VERB

Incorrect

Alimta was also compared with gemcitabine (another anticancer medicine), both in combination with cisplatin, in a study involving 1,725 patients who had not received chemotherapy for lung cancer in the past.

Show raw prompt

user

Which numbered sense of "involve" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Alimta was also compared with gemcitabine (another anticancer medicine), both in combination with cisplatin, in a study <t>involving</t> 1,725 patients who had not received chemotherapy for lung cancer in the past.

Senses:
1. definition=connect closely and often incriminatingly | examples=This new ruling affects your business
2. definition=engage as a participant | examples=Don't involve me in your family affairs!
3. definition=have as a necessary feature | examples=This decision involves many changes
4. definition=require as useful, just, or proper | examples=It takes nerve to do what she did
5. definition=contain as a part | examples=Dinner at Joe's always involves at least six courses
6. definition=occupy or engage the interest of | examples=His story completely involved me during the entire afternoon
7. definition=make complex or intricate or complicated | examples=The situation was rather involved

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    connect closely and often incriminatingly

    involve%2:42:01:: · involve.v.01

    • This new ruling affects your business

    Synonyms: affect, regard

  2. 2

    engage as a participant

    involve%2:42:04:: · involve.v.02

    • Don't involve me in your family affairs!
    Gold lexEN Gold Maru 2022 Gold Raganato
  3. 3

    have as a necessary feature

    involve%2:42:00:: · imply.v.05

    • This decision involves many changes

    Synonyms: imply

  4. 4

    require as useful, just, or proper

    involve%2:42:07:: · necessitate.v.01

    • It takes nerve to do what she did
    • success usually requires hard work

    Synonyms: necessitate, ask, postulate, need, require, take, call for, demand

  5. 5

    contain as a part

    involve%2:42:02:: · involve.v.05

    • Dinner at Joe's always involves at least six courses
    Selected
  6. 6

    occupy or engage the interest of

    involve%2:31:00:: · involve.v.06

    • His story completely involved me during the entire afternoon
  7. 7

    make complex or intricate or complicated

    involve%2:30:00:: · involve.v.07

    • The situation was rather involved

survive Verb

Instance: semeval2015.d000.s029.t006 Dataset: semeval2015 Bucket: POS / VERB

Incorrect

In all three studies, the main measure of effectiveness was how long the patients survived.

Show raw prompt

user

Which numbered sense of "survive" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: In all three studies, the main measure of effectiveness was how long the patients <t>survived</t>.

Senses:
1. definition=continue to live through hardship or adversity | examples=We went without water and food for 3 days
2. definition=continue in existence after (an adversity, etc.) | examples=He survived the cancer against all odds
3. definition=support oneself | examples=he could barely exist on such a low wage
4. definition=live longer than | examples=She outlived her husband by many years

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✓

WordNet 3.0 senses

  1. 1

    continue to live through hardship or adversity

    survive%2:42:01:: · survive.v.01

    • We went without water and food for 3 days
    • These superstitions survive in the backwaters of America

    Synonyms: last, live, live on, go, endure, hold up, hold out

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    continue in existence after (an adversity, etc.)

    survive%2:42:02:: · survive.v.02

    • He survived the cancer against all odds

    Synonyms: pull through, pull round, come through, make it

    Gold Raganato Selected
  3. 3

    support oneself

    survive%2:42:00:: · exist.v.02

    • he could barely exist on such a low wage
    • Can you live on $2000 a month in New York City?

    Synonyms: exist, live, subsist

  4. 4

    live longer than

    survive%2:42:03:: · outlive.v.01

    • She outlived her husband by many years

    Synonyms: outlive, outlast

increase Verb

Instance: semeval2015.d000.s031.t000 Dataset: semeval2015 Bucket: POS / VERB

Incorrect

Alimta increased the survival time of patients with malignant pleural mesothelioma.

Show raw prompt

user

Which numbered sense of "increase" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Alimta <t>increased</t> the survival time of patients with malignant pleural mesothelioma.

Senses:
1. definition=become bigger or greater in amount | examples=The amount of work increased
2. definition=make bigger or more | examples=The boss finally increased her salary

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✗

WordNet 3.0 senses

  1. 1

    become bigger or greater in amount

    increase%2:30:00:: · increase.v.01

    • The amount of work increased
    Selected
  2. 2

    make bigger or more

    increase%2:30:02:: · increase.v.02

    • The boss finally increased her salary
    • The university increased the number of students it admitted
    Gold lexEN Gold Maru 2022 Gold Raganato

survive Verb

Instance: semeval2015.d000.s032.t002 Dataset: semeval2015 Bucket: POS / VERB

Incorrect

Patients receiving Alimta and cisplatin survived for an average of 12.1 months, compared with 9.3 months in those receiving cisplatin alone.

Show raw prompt

user

Which numbered sense of "survive" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.

Sentence: Patients receiving Alimta and cisplatin <t>survived</t> for an average of 12.1 months, compared with 9.3 months in those receiving cisplatin alone.

Senses:
1. definition=continue to live through hardship or adversity | examples=We went without water and food for 3 days
2. definition=continue in existence after (an adversity, etc.) | examples=He survived the cancer against all odds
3. definition=support oneself | examples=he could barely exist on such a low wage
4. definition=live longer than | examples=She outlived her husband by many years

Prediction matches gold: lexEN ✗ Maru 2022 ✗ Raganato ✓

WordNet 3.0 senses

  1. 1

    continue to live through hardship or adversity

    survive%2:42:01:: · survive.v.01

    • We went without water and food for 3 days
    • These superstitions survive in the backwaters of America

    Synonyms: last, live, live on, go, endure, hold up, hold out

    Gold lexEN Gold Maru 2022 Gold Raganato
  2. 2

    continue in existence after (an adversity, etc.)

    survive%2:42:02:: · survive.v.02

    • He survived the cancer against all odds

    Synonyms: pull through, pull round, come through, make it

    Gold Raganato Selected
  3. 3

    support oneself

    survive%2:42:00:: · exist.v.02

    • he could barely exist on such a low wage
    • Can you live on $2000 a month in New York City?

    Synonyms: exist, live, subsist

  4. 4

    live longer than

    survive%2:42:03:: · outlive.v.01

    • She outlived her husband by many years

    Synonyms: outlive, outlast