Verified run
gpt-4.1
gpt-4.1-p002-lexen-v1-20260614
- Rank
- 83
- Accuracy lexEN v1 · WordNet fine
- 87.86%
- Correct
- 4271 / 4861
- Actual run cost
- $2.26
- Cost / M items
- $465
Accuracy across label schemes
The headline accuracy is lexEN v1 · WordNet fine-grained. Each cell re-scores this run's predictions against a different gold label set and sense granularity; coarse counts a prediction correct when its WordNet sense maps to the same Glite or CSI concept as a gold sense. About these label schemes → About coarsening →
Model
- Display
- gpt-4.1
- Requested
- gpt-4.1
- Resolved
- gpt-4.1-2025-04-14
- Vendor
- OpenAI
- API provider
- OpenAI
- Hosting
- cloud_api
- Source
- proprietary
- License
- n/a
- Reasoning
- n/a
- Model URL
- n/a
- Endpoint
- n/a
- Resolved counts
-
gpt-4.1-2025-04-14: 4861
Reproducibility
- Created
- 2026-06-14T10:31:43.235840+00:00
- Run schema
- sensebench-run-v2
- Commit
- 7239ecd78b43666ee56538642d09b75c992ee31f
- Dataset ID
- lexen
- Dataset version
- lexen-v1
- Dataset revision
- n/a
- Dataset items
- 4,861
- Dataset hash
- sha256:5fd4382b93f19087a1e31f6dd7d1db17c1eb17ff80fcbe1d3fdd55c0c3ecefe8
- Prompt
- p002
- Prompt version
- 0.1.0
- Repository files
- Browse in GitHub
Runtime
- Calls
- 4861
- Total tokens
- 1,127,609
- Tokens / item
- 232.0
- Total cost
- $2.26
- Cost / M items
- $465
- Cost source
- litellm_estimate
- Elapsed
- 33.88s
- Success items
- 4,861
- Monosemous
- 0
- No candidates
- 0
- No valid vote
- 0
- Invalid votes
- 0
- Transport errors
- 0
Runner
- GitHub
- vassiliphilippov
- Name
- Vassili Philippov
- Contact
- n/a
Sampling
- Temperature
- 0.0000
- Top P
- n/a
- Max tokens
- 256
- Seed
- n/a
- Extra
-
n/a
Policy
- Votes / item
- 1
- Reasks / invalid vote
- 1
- Tie break
- earliest_vote
- Monosemous
- short_circuit
Cost And Token Breakdown
Actual recorded run cost: $2.26. Cost estimate source: litellm_estimate.
Reasoning output tokens are shown separately when the provider reports them; recorded cost is stored in the output cost bucket.
Download Raw Run Files
Run Metadata
run.json
Submitted run.json with model, dataset, prompt, policy, totals, and costs.
2.3 KB
Predictions
predictions.jsonl
One JSON record per benchmark item with candidates, votes, and correctness.
5.9 MB
Raw Calls
calls.jsonl.gz
Gzipped JSONL call log with prompts, raw outputs, token usage, and costs.
1.6 MB
Worst-Bucket Examples
Alimta is used together with cisplatin (another anticancer medicine) when the cancer is unresectable (cannot be removed by surgery alone) and malignant (has spread, or is likely to spread easily, to other parts of the body), in patients who have not received chemotherapy (medicines for cancer) before advanced or metastatic non-small cell lung cancer that is not affecting the squamous cells.
Show raw prompt
user
Which numbered sense of "easily" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: Alimta is used together with cisplatin (another anticancer medicine) when the cancer is unresectable (cannot be removed by surgery alone) and malignant (has spread, or is likely to spread <t>easily</t>, to other parts of the body), in patients who have not received chemotherapy (medicines for cancer) before advanced or metastatic non-small cell lung cancer that is not affecting the squamous cells.
Senses:
1. definition=with ease (`easy' is sometimes used informally for `easily') | examples=she was easily excited
2. definition=without question | examples=easily the best book she's written
3. definition=indicating high probability; in all likelihood | examples=I might well do it
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
with ease (`easy' is sometimes used informally for `easily')
easily%4:02:01:: · easily.r.01
- she was easily excited
- was easily confused
Synonyms: easy
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
without question
easily%4:02:06:: · easily.r.02
- easily the best book she's written
-
3
indicating high probability; in all likelihood
easily%4:02:05:: · well.r.03
- I might well do it
- a mistake that could easily have ended in disaster
Synonyms: well
Selected
Alimta should only be given under the supervision of a doctor who is qualified in the use of anticancer chemotherapy.
Show raw prompt
user
Which numbered sense of "only" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: Alimta should <t>only</t> be given under the supervision of a doctor who is qualified in the use of anticancer chemotherapy.
Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=without any others being included or involved | examples=was entirely to blame
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
and nothing more
only%4:02:02:: · merely.r.01
- I was merely asking
- it is simply a matter of time
Synonyms: merely, simply, just, but
-
2
without any others being included or involved
only%4:02:01:: · entirely.r.02
- was entirely to blame
- a school devoted entirely to the needs of problem children
Synonyms: entirely, exclusively, solely, alone
Selected
-
3
with nevertheless the final result
only%4:02:00:: · only.r.03
- He arrived only to find his wife dead
- We won only to lose again in the next round
-
4
in the final outcome
only%4:02:04:: · only.r.04
- These news will only make you more upset
-
5
except that
only%4:02:06:: · only.r.05
- It was the same story; only this time she came out better
-
6
never except when
only%4:02:03:: · only.r.06
- call me only if your cold gets worse
Synonyms: only if, only when
Gold lexEN
Gold Maru 2022
Gold Raganato
-
7
as recently as
only%4:02:05:: · only.r.07
- I spoke to him only an hour ago
There were relatively few cases reported of attempts to involve users in service planning but their involvement in service provision was found to be more common.
Show raw prompt
user
Which numbered sense of "more" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: There were relatively few cases reported of attempts to involve users in service planning but their involvement in service provision was found to be <t>more</t> common.
Senses:
1. definition=used to form the comparative of some adjectives and adverbs | examples=more interesting
2. definition=comparative of much; to a greater degree or extent | examples=he works more now
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
used to form the comparative of some adjectives and adverbs
more%4:02:00:: · more.r.01
- more interesting
- more beautiful
Synonyms: to a greater extent
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
comparative of much; to a greater degree or extent
more%4:02:01:: · more.r.02
- he works more now
- they eat more than they should
Selected
Now, only one local ringer remains: 64-year-old Derek Hammond.
Show raw prompt
user
Which numbered sense of "now" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: <t>Now</t>, only one local ringer remains: 64-year-old Derek Hammond.
Senses:
1. definition=in the historical present; at this point in the narration of a series of past events | examples=President Kennedy now calls in the National Guard
2. definition=in these times; - Nancy Mitford | examples=it is solely by their language that the upper classes nowadays are distinguished
3. definition=used to preface a command or reproof or request | examples=now hear this!
4. definition=at the present moment | examples=goods now on sale
5. definition=without delay or hesitation; with no time intervening | examples=he answered immediately
6. definition=(prefatory or transitional) indicates a change of subject or activity | examples=Now the next problem is...
7. definition=in the immediate past | examples=told me just now
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
in the historical present; at this point in the narration of a series of past events
now%4:02:05:: · now.r.01
- President Kennedy now calls in the National Guard
- Washington now decides to cross the Delaware
-
2
in these times; - Nancy Mitford
now%4:02:00:: · nowadays.r.01
- it is solely by their language that the upper classes nowadays are distinguished
- we now rarely see horse-drawn vehicles on city streets
Synonyms: nowadays, today
Gold Raganato
-
3
used to preface a command or reproof or request
now%4:02:02:: · now.r.03
- now hear this!
- now pay attention
-
4
at the present moment
now%4:02:04:: · now.r.04
- goods now on sale
- the now-aging dictator
Synonyms: at present
Gold lexEN
Gold Maru 2022
-
5
without delay or hesitation; with no time intervening
now%4:02:01:: · immediately.r.01
- he answered immediately
- found an answer straightaway
Synonyms: immediately, instantly, straightaway, straight off, directly, right away, at once, forthwith, like a shot
-
6
(prefatory or transitional) indicates a change of subject or activity
now%4:02:07:: · now.r.06
- Now the next problem is...
Selected
-
7
in the immediate past
now%4:02:06:: · now.r.07
Now, only one local ringer remains: 64-year-old Derek Hammond.
Show raw prompt
user
Which numbered sense of "only" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: Now, <t>only</t> one local ringer remains: 64-year-old Derek Hammond.
Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=without any others being included or involved | examples=was entirely to blame
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
and nothing more
only%4:02:02:: · merely.r.01
- I was merely asking
- it is simply a matter of time
Synonyms: merely, simply, just, but
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
without any others being included or involved
only%4:02:01:: · entirely.r.02
- was entirely to blame
- a school devoted entirely to the needs of problem children
Synonyms: entirely, exclusively, solely, alone
Selected
-
3
with nevertheless the final result
only%4:02:00:: · only.r.03
- He arrived only to find his wife dead
- We won only to lose again in the next round
-
4
in the final outcome
only%4:02:04:: · only.r.04
- These news will only make you more upset
-
5
except that
only%4:02:06:: · only.r.05
- It was the same story; only this time she came out better
-
6
never except when
only%4:02:03:: · only.r.06
- call me only if your cold gets worse
Synonyms: only if, only when
-
7
as recently as
only%4:02:05:: · only.r.07
- I spoke to him only an hour ago
We've tried to train the youngsters, but they have their discos and their dances, and they just drift away. "
Show raw prompt
user
Which numbered sense of "just" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: We've tried to train the youngsters, but they have their discos and their dances, and they <t>just</t> drift away. "
Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=indicating exactness or preciseness | examples=he was doing precisely (or exactly) what she had told him to do
3. definition=only a moment ago | examples=he has just arrived
4. definition=absolutely | examples=I just can't take it anymore
5. definition=only a very short time before; ; ; ; ; - W.B.Yeats | examples=they could barely hear the speaker
6. definition=exactly at this moment or the moment described | examples=we've just finished painting the walls, so don't touch them
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
and nothing more
just%4:02:00:: · merely.r.01
- I was merely asking
- it is simply a matter of time
Synonyms: merely, simply, only, but
Selected
-
2
indicating exactness or preciseness
just%4:02:05:: · precisely.r.01
- he was doing precisely (or exactly) what she had told him to do
- it was just as he said--the jewel was gone
Synonyms: precisely, exactly
-
3
only a moment ago
just%4:02:01:: · just.r.03
- he has just arrived
- the sun just now came out
Synonyms: just now
-
4
absolutely
just%4:02:03:: · just.r.04
- I just can't take it anymore
- he was just grand as Romeo
Synonyms: simply
Gold lexEN
Gold Maru 2022
Gold Raganato
-
5
only a very short time before; ; ; ; ; - W.B.Yeats
just%4:02:04:: · barely.r.01
- they could barely hear the speaker
- we hardly knew them
Synonyms: barely, hardly, scarcely, scarce
-
6
exactly at this moment or the moment described
just%4:02:06:: · just.r.06
- we've just finished painting the walls, so don't touch them
Each variation, or change, can occur only once, the rules state.
Show raw prompt
user
Which numbered sense of "only" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: Each variation, or change, can occur <t>only</t> once, the rules state.
Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=without any others being included or involved | examples=was entirely to blame
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
and nothing more
only%4:02:02:: · merely.r.01
- I was merely asking
- it is simply a matter of time
Synonyms: merely, simply, just, but
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
without any others being included or involved
only%4:02:01:: · entirely.r.02
- was entirely to blame
- a school devoted entirely to the needs of problem children
Synonyms: entirely, exclusively, solely, alone
Selected
-
3
with nevertheless the final result
only%4:02:00:: · only.r.03
- He arrived only to find his wife dead
- We won only to lose again in the next round
-
4
in the final outcome
only%4:02:04:: · only.r.04
- These news will only make you more upset
-
5
except that
only%4:02:06:: · only.r.05
- It was the same story; only this time she came out better
-
6
never except when
only%4:02:03:: · only.r.06
- call me only if your cold gets worse
Synonyms: only if, only when
-
7
as recently as
only%4:02:05:: · only.r.07
- I spoke to him only an hour ago
Totally absorbed, the ringers stare straight ahead, using peripheral vision (they call it "rope-sight ") to watch the other ropes and thus time their pulls.
Show raw prompt
user
Which numbered sense of "straight" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: Totally absorbed, the ringers stare <t>straight</t> ahead, using peripheral vision (they call it "rope-sight ") to watch the other ropes and thus time their pulls.
Senses:
1. definition=without deviation | examples=the path leads directly to the lake
2. definition=in a forthright manner; candidly or frankly | examples=he didn't answer directly
3. definition=in a straight line; in a direct course | examples=the road runs straight
Prediction matches gold:
lexEN ✗
Maru 2022 ✓
Raganato ✗
WordNet 3.0 senses
-
1
without deviation
straight%4:02:05:: · directly.r.01
- the path leads directly to the lake
- went direct to the office
Synonyms: directly, direct
Gold lexEN
Gold Raganato
-
2
in a forthright manner; candidly or frankly
straight%4:02:04:: · directly.r.04
- he didn't answer directly
- told me straight out
Synonyms: directly, flat
-
3
in a straight line; in a direct course
straight%4:02:00:: · straight.r.03
Gold Maru 2022
Selected
Totally absorbed, the ringers stare straight ahead, using peripheral vision (they call it "rope-sight ") to watch the other ropes and thus time their pulls.
Show raw prompt
user
Which numbered sense of "thus" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: Totally absorbed, the ringers stare straight ahead, using peripheral vision (they call it "rope-sight ") to watch the other ropes and <t>thus</t> time their pulls.
Senses:
1. definition=(used to introduce a logical conclusion) from that fact or reason or as a result | examples=therefore X must be true
2. definition=in the way indicated; ; ; (`thusly' is a nonstandard variant) | examples=hold the brush so
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
(used to introduce a logical conclusion) from that fact or reason or as a result
thus%4:02:00:: · therefore.r.01
- therefore X must be true
- the eggs were fresh and hence satisfactory
Synonyms: therefore, hence, thence, so
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
in the way indicated; ; ; (`thusly' is a nonstandard variant)
thus%4:02:01:: · thus.r.02
- hold the brush so
- set up the pieces thus
Synonyms: thusly, so
Selected
Skilled ringers use their wrists to advance or retard the next swing, so that one bell can swap places with another in the following change.
Show raw prompt
user
Which numbered sense of "so" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: Skilled ringers use their wrists to advance or retard the next swing, <t>so</t> that one bell can swap places with another in the following change.
Senses:
1. definition=to a very great extent or degree | examples=the idea is so obvious
2. definition=in a manner that facilitates | examples=he observed the snakes so he could describe their behavior
3. definition=in such a condition or manner, especially as expressed or implied | examples=They're happy and I hope they will remain so
4. definition=to a certain unspecified extent or degree | examples=I can only go so far with this student
5. definition=in the same way; also | examples=I was offended and so was he
6. definition=in the way indicated; ; ; (`thusly' is a nonstandard variant) | examples=hold the brush so
7. definition=(usually followed by `that') to an extent or degree as expressed | examples=he was so tired he could hardly stand
8. definition=subsequently or soon afterward (often used as sentence connectors) | examples=then he left
9. definition=(used to introduce a logical conclusion) from that fact or reason or as a result | examples=therefore X must be true
10. definition=in truth (often tends to intensify) | examples=they said the car would break down and indeed it did
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
to a very great extent or degree
so%4:02:02:: · so.r.01
- the idea is so obvious
- never been so happy
-
2
in a manner that facilitates
so%4:02:06:: · so.r.02
- he observed the snakes so he could describe their behavior
- he stooped down so he could pick up his hat
Gold lexEN
Gold Maru 2022
Gold Raganato
-
3
in such a condition or manner, especially as expressed or implied
so%4:02:05:: · so.r.03
- They're happy and I hope they will remain so
- so live your life that old age will bring no regrets
-
4
to a certain unspecified extent or degree
so%4:02:07:: · so.r.04
- I can only go so far with this student
- can do only so much in a day
-
5
in the same way; also
so%4:02:08:: · so.r.05
- I was offended and so was he
- worked hard and so did she
-
6
in the way indicated; ; ; (`thusly' is a nonstandard variant)
so%4:02:04:: · thus.r.02
- hold the brush so
- set up the pieces thus
Synonyms: thus, thusly
-
7
(usually followed by `that') to an extent or degree as expressed
so%4:02:03:: · so.r.07
- he was so tired he could hardly stand
- so dirty that it smells
Selected
-
8
subsequently or soon afterward (often used as sentence connectors)
so%4:02:09:: · then.r.01
- then he left
- go left first, then right
Synonyms: then, and so, and then
-
9
(used to introduce a logical conclusion) from that fact or reason or as a result
so%4:02:10:: · therefore.r.01
- therefore X must be true
- the eggs were fresh and hence satisfactory
Synonyms: therefore, hence, thence, thus
-
10
in truth (often tends to intensify)
so%4:02:00:: · indeed.r.01
- they said the car would break down and indeed it did
- it is very cold indeed
Synonyms: indeed
It is a passion that usually stays in the tower, however.
Show raw prompt
user
Which numbered sense of "however" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: It is a passion that usually stays in the tower, <t>however</t>.
Senses:
1. definition=despite anything to the contrary (usually following a concession) | examples=although I'm a little afraid, however I'd like to try it
2. definition=by contrast; on the other hand | examples=the first part was easy; the second, however, took hours
3. definition=to whatever degree or extent | examples=The results, however general, are important
4. definition=in whatever way or manner | examples=Victory, however it was brought about, was sweet
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
despite anything to the contrary (usually following a concession)
however%4:02:00:: · however.r.01
- although I'm a little afraid, however I'd like to try it
- while we disliked each other, nevertheless we agreed
Synonyms: nevertheless, withal, still, yet, all the same, even so, nonetheless, notwithstanding
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
by contrast; on the other hand
however%4:02:04:: · however.r.02
- the first part was easy; the second, however, took hours
Selected
-
3
to whatever degree or extent
however%4:02:02:: · however.r.03
- The results, however general, are important
- they have begun, however reluctantly, to acknowledge the legitimacy of some of the opposition's concerns
-
4
in whatever way or manner
however%4:02:01:: · however.r.04
- Victory, however it was brought about, was sweet
- however he did it, it was very clever
More often than not, ringers think of the church as something stuck on the bottom of the belfry.
Show raw prompt
user
Which numbered sense of "more" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: <t>More</t> often than not, ringers think of the church as something stuck on the bottom of the belfry.
Senses:
1. definition=used to form the comparative of some adjectives and adverbs | examples=more interesting
2. definition=comparative of much; to a greater degree or extent | examples=he works more now
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
used to form the comparative of some adjectives and adverbs
more%4:02:00:: · more.r.01
- more interesting
- more beautiful
Synonyms: to a greater extent
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
comparative of much; to a greater degree or extent
more%4:02:01:: · more.r.02
- he works more now
- they eat more than they should
Selected