Verified run
gpt-4.1-mini
gpt-4.1-mini-p002-lexen-v1-20260614
- Rank
- 144
- Accuracy lexEN v1 · WordNet fine
- 80.54%
- Correct
- 3915 / 4861
- Actual run cost
- $0.457
- Cost / M items
- $93.99
Accuracy across label schemes
The headline accuracy is lexEN v1 · WordNet fine-grained. Each cell re-scores this run's predictions against a different gold label set and sense granularity; coarse counts a prediction correct when its WordNet sense maps to the same Glite or CSI concept as a gold sense. About these label schemes → About coarsening →
Model
- Display
- gpt-4.1-mini
- Requested
- gpt-4.1-mini
- Resolved
- gpt-4.1-mini-2025-04-14
- Vendor
- OpenAI
- API provider
- OpenAI
- Hosting
- cloud_api
- Source
- proprietary
- License
- n/a
- Reasoning
- n/a
- Model URL
- n/a
- Endpoint
- n/a
- Resolved counts
-
gpt-4.1-mini-2025-04-14: 4861
Reproducibility
- Created
- 2026-06-14T03:10:47.348916+00:00
- Run schema
- sensebench-run-v2
- Commit
- 78aebb2a6fd1412a1e8eab85f6209a64f8f441e8
- Dataset ID
- lexen
- Dataset version
- lexen-v1
- Dataset revision
- n/a
- Dataset items
- 4,861
- Dataset hash
- sha256:5fd4382b93f19087a1e31f6dd7d1db17c1eb17ff80fcbe1d3fdd55c0c3ecefe8
- Prompt
- p002
- Prompt version
- 0.1.0
- Repository files
- Browse in GitHub
Runtime
- Calls
- 4861
- Total tokens
- 1,127,609
- Tokens / item
- 232.0
- Total cost
- $0.457
- Cost / M items
- $93.99
- Cost source
- litellm_estimate
- Elapsed
- 11.16s
- Success items
- 4,861
- Monosemous
- 0
- No candidates
- 0
- No valid vote
- 0
- Invalid votes
- 0
- Transport errors
- 0
Runner
- GitHub
- vassiliphilippov
- Name
- Vassili Philippov
- Contact
- n/a
Sampling
- Temperature
- 0.0000
- Top P
- n/a
- Max tokens
- 256
- Seed
- n/a
- Extra
-
n/a
Policy
- Votes / item
- 1
- Reasks / invalid vote
- 1
- Tie break
- earliest_vote
- Monosemous
- short_circuit
Cost And Token Breakdown
Actual recorded run cost: $0.457. Cost estimate source: litellm_estimate.
Reasoning output tokens are shown separately when the provider reports them; recorded cost is stored in the output cost bucket.
Download Raw Run Files
Run Metadata
run.json
Submitted run.json with model, dataset, prompt, policy, totals, and costs.
2.3 KB
Predictions
predictions.jsonl
One JSON record per benchmark item with candidates, votes, and correctness.
5.9 MB
Raw Calls
calls.jsonl.gz
Gzipped JSONL call log with prompts, raw outputs, token usage, and costs.
1.6 MB
Worst-Bucket Examples
Alimta is used together with cisplatin (another anticancer medicine) when the cancer is unresectable (cannot be removed by surgery alone) and malignant (has spread, or is likely to spread easily, to other parts of the body), in patients who have not received chemotherapy (medicines for cancer) before advanced or metastatic non-small cell lung cancer that is not affecting the squamous cells.
Show raw prompt
user
Which numbered sense of "easily" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: Alimta is used together with cisplatin (another anticancer medicine) when the cancer is unresectable (cannot be removed by surgery alone) and malignant (has spread, or is likely to spread <t>easily</t>, to other parts of the body), in patients who have not received chemotherapy (medicines for cancer) before advanced or metastatic non-small cell lung cancer that is not affecting the squamous cells.
Senses:
1. definition=with ease (`easy' is sometimes used informally for `easily') | examples=she was easily excited
2. definition=without question | examples=easily the best book she's written
3. definition=indicating high probability; in all likelihood | examples=I might well do it
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
with ease (`easy' is sometimes used informally for `easily')
easily%4:02:01:: · easily.r.01
- she was easily excited
- was easily confused
Synonyms: easy
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
without question
easily%4:02:06:: · easily.r.02
- easily the best book she's written
-
3
indicating high probability; in all likelihood
easily%4:02:05:: · well.r.03
- I might well do it
- a mistake that could easily have ended in disaster
Synonyms: well
Selected
Alimta should only be given under the supervision of a doctor who is qualified in the use of anticancer chemotherapy.
Show raw prompt
user
Which numbered sense of "only" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: Alimta should <t>only</t> be given under the supervision of a doctor who is qualified in the use of anticancer chemotherapy.
Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=without any others being included or involved | examples=was entirely to blame
3. definition=with nevertheless the final result | examples=He arrived only to find his wife dead
4. definition=in the final outcome | examples=These news will only make you more upset
5. definition=except that | examples=It was the same story; only this time she came out better
6. definition=never except when | examples=call me only if your cold gets worse
7. definition=as recently as | examples=I spoke to him only an hour ago
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
and nothing more
only%4:02:02:: · merely.r.01
- I was merely asking
- it is simply a matter of time
Synonyms: merely, simply, just, but
-
2
without any others being included or involved
only%4:02:01:: · entirely.r.02
- was entirely to blame
- a school devoted entirely to the needs of problem children
Synonyms: entirely, exclusively, solely, alone
Selected
-
3
with nevertheless the final result
only%4:02:00:: · only.r.03
- He arrived only to find his wife dead
- We won only to lose again in the next round
-
4
in the final outcome
only%4:02:04:: · only.r.04
- These news will only make you more upset
-
5
except that
only%4:02:06:: · only.r.05
- It was the same story; only this time she came out better
-
6
never except when
only%4:02:03:: · only.r.06
- call me only if your cold gets worse
Synonyms: only if, only when
Gold lexEN
Gold Maru 2022
Gold Raganato
-
7
as recently as
only%4:02:05:: · only.r.07
- I spoke to him only an hour ago
It is actually not necessary to know MathML to use kalgebra.
Show raw prompt
user
Which numbered sense of "actually" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: It is <t>actually</t> not necessary to know MathML to use kalgebra.
Senses:
1. definition=in actual fact | examples=to be nominally but not actually independent
2. definition=used to imply that one would expect the fact to be the opposite of that stated; surprisingly | examples=you may actually be doing the right thing by walking out
3. definition=at the present moment | examples=the transmission screen shows the picture that is actually on the air
4. definition=as a sentence modifier to add slight emphasis | examples=actually, we all help clear up after a meal
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
in actual fact
actually%4:02:01:: · actually.r.01
- to be nominally but not actually independent
- no one actually saw the shark
Synonyms: really
Selected
-
2
used to imply that one would expect the fact to be the opposite of that stated; surprisingly
actually%4:02:00:: · actually.r.02
- you may actually be doing the right thing by walking out
- she actually spoke Latin
Synonyms: in reality
Gold lexEN
Gold Maru 2022
Gold Raganato
-
3
at the present moment
actually%4:02:03:: · actually.r.03
- the transmission screen shows the picture that is actually on the air
-
4
as a sentence modifier to add slight emphasis
actually%4:02:02:: · actually.r.04
- actually, we all help clear up after a meal
- actually, I haven't seen the film
Gold lexEN
Gold Maru 2022
Gold Raganato
Here's a screenshot of kalgebra main window
Show raw prompt
user
Which numbered sense of "here" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: <t>Here</t>'s a screenshot of kalgebra main window
Senses:
1. definition=in or at this place; where the speaker or writer is | examples=I work here
2. definition=in this circumstance or respect or on this point or detail | examples=what do we have here?
3. definition=to this place (especially toward the speaker) | examples=come here, please
4. definition=at this time; now | examples=we'll adjourn here for lunch and discuss the remaining issues this afternoon
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
in or at this place; where the speaker or writer is
here%4:02:00:: · here.r.01
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
in this circumstance or respect or on this point or detail
here%4:02:02:: · here.r.02
- what do we have here?
- here I must disagree
Selected
-
3
to this place (especially toward the speaker)
here%4:02:01:: · here.r.03
Synonyms: hither
-
4
at this time; now
here%4:02:04:: · here.r.04
- we'll adjourn here for lunch and discuss the remaining issues this afternoon
Below these tabs you will find an input field to type your functions or do your calculations.
Show raw prompt
user
Which numbered sense of "below" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: <t>Below</t> these tabs you will find an input field to type your functions or do your calculations.
Senses:
1. definition=in or to a place that is lower
2. definition=at a later place | examples=see below
3. definition=(in writing) see below | examples=vide infra
4. definition=on a floor below | examples=the tenants live downstairs
5. definition=further down | examples=see under for further discussion
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
in or to a place that is lower
below%4:02:01:: · below.r.01
Synonyms: at a lower place, to a lower place, beneath
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
at a later place
below%4:02:00:: · below.r.02
-
3
(in writing) see below
below%4:02:02:: · below.r.03
Synonyms: infra
-
4
on a floor below
below%4:02:06:: · downstairs.r.01
- the tenants live downstairs
Synonyms: downstairs, down the stairs, on a lower floor
-
5
further down
below%4:02:03:: · under.r.08
- see under for further discussion
Synonyms: under
Selected
I ve looked closely to maxima and maple s syntax so they look very similar.
Show raw prompt
user
Which numbered sense of "closely" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: I ve looked <t>closely</t> to maxima and maple s syntax so they look very similar.
Senses:
1. definition=in a close relation or position in time or space | examples=the onsets were closely timed
2. definition=in an attentive manner | examples=he remained close on his guard
3. definition=in a close manner | examples=the two phenomena are intimately connected
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
in a close relation or position in time or space
closely%4:02:00:: · closely.r.01
- the onsets were closely timed
- houses set closely together
-
2
in an attentive manner
closely%4:02:02:: · close.r.02
- he remained close on his guard
Synonyms: close, tight
Gold lexEN
Gold Maru 2022
Gold Raganato
-
3
in a close manner
closely%4:02:01:: · closely.r.03
- the two phenomena are intimately connected
- the person most nearly concerned
Synonyms: intimately, nearly
Selected
Just use the Add button when you are in List mode.
Show raw prompt
user
Which numbered sense of "just" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: <t>Just</t> use the Add button when you are in List mode.
Senses:
1. definition=and nothing more | examples=I was merely asking
2. definition=indicating exactness or preciseness | examples=he was doing precisely (or exactly) what she had told him to do
3. definition=only a moment ago | examples=he has just arrived
4. definition=absolutely | examples=I just can't take it anymore
5. definition=only a very short time before; ; ; ; ; - W.B.Yeats | examples=they could barely hear the speaker
6. definition=exactly at this moment or the moment described | examples=we've just finished painting the walls, so don't touch them
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
and nothing more
just%4:02:00:: · merely.r.01
- I was merely asking
- it is simply a matter of time
Synonyms: merely, simply, only, but
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
indicating exactness or preciseness
just%4:02:05:: · precisely.r.01
- he was doing precisely (or exactly) what she had told him to do
- it was just as he said--the jewel was gone
Synonyms: precisely, exactly
Selected
-
3
only a moment ago
just%4:02:01:: · just.r.03
- he has just arrived
- the sun just now came out
Synonyms: just now
-
4
absolutely
just%4:02:03:: · just.r.04
- I just can't take it anymore
- he was just grand as Romeo
Synonyms: simply
-
5
only a very short time before; ; ; ; ; - W.B.Yeats
just%4:02:04:: · barely.r.01
- they could barely hear the speaker
- we hardly knew them
Synonyms: barely, hardly, scarcely, scarce
-
6
exactly at this moment or the moment described
just%4:02:06:: · just.r.06
- we've just finished painting the walls, so don't touch them
There were relatively few cases reported of attempts to involve users in service planning but their involvement in service provision was found to be more common.
Show raw prompt
user
Which numbered sense of "more" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: There were relatively few cases reported of attempts to involve users in service planning but their involvement in service provision was found to be <t>more</t> common.
Senses:
1. definition=used to form the comparative of some adjectives and adverbs | examples=more interesting
2. definition=comparative of much; to a greater degree or extent | examples=he works more now
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
used to form the comparative of some adjectives and adverbs
more%4:02:00:: · more.r.01
- more interesting
- more beautiful
Synonyms: to a greater extent
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
comparative of much; to a greater degree or extent
more%4:02:01:: · more.r.02
- he works more now
- they eat more than they should
Selected
The tablets are pale-orange and have a score line on both sides so that they can be halved.
Show raw prompt
user
Which numbered sense of "so" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: The tablets are pale-orange and have a score line on both sides <t>so</t> that they can be halved.
Senses:
1. definition=to a very great extent or degree | examples=the idea is so obvious
2. definition=in a manner that facilitates | examples=he observed the snakes so he could describe their behavior
3. definition=in such a condition or manner, especially as expressed or implied | examples=They're happy and I hope they will remain so
4. definition=to a certain unspecified extent or degree | examples=I can only go so far with this student
5. definition=in the same way; also | examples=I was offended and so was he
6. definition=in the way indicated; ; ; (`thusly' is a nonstandard variant) | examples=hold the brush so
7. definition=(usually followed by `that') to an extent or degree as expressed | examples=he was so tired he could hardly stand
8. definition=subsequently or soon afterward (often used as sentence connectors) | examples=then he left
9. definition=(used to introduce a logical conclusion) from that fact or reason or as a result | examples=therefore X must be true
10. definition=in truth (often tends to intensify) | examples=they said the car would break down and indeed it did
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
to a very great extent or degree
so%4:02:02:: · so.r.01
- the idea is so obvious
- never been so happy
-
2
in a manner that facilitates
so%4:02:06:: · so.r.02
- he observed the snakes so he could describe their behavior
- he stooped down so he could pick up his hat
Gold lexEN
Gold Maru 2022
-
3
in such a condition or manner, especially as expressed or implied
so%4:02:05:: · so.r.03
- They're happy and I hope they will remain so
- so live your life that old age will bring no regrets
-
4
to a certain unspecified extent or degree
so%4:02:07:: · so.r.04
- I can only go so far with this student
- can do only so much in a day
-
5
in the same way; also
so%4:02:08:: · so.r.05
- I was offended and so was he
- worked hard and so did she
-
6
in the way indicated; ; ; (`thusly' is a nonstandard variant)
so%4:02:04:: · thus.r.02
- hold the brush so
- set up the pieces thus
Synonyms: thus, thusly
-
7
(usually followed by `that') to an extent or degree as expressed
so%4:02:03:: · so.r.07
- he was so tired he could hardly stand
- so dirty that it smells
Gold Raganato
-
8
subsequently or soon afterward (often used as sentence connectors)
so%4:02:09:: · then.r.01
- then he left
- go left first, then right
Synonyms: then, and so, and then
-
9
(used to introduce a logical conclusion) from that fact or reason or as a result
so%4:02:10:: · therefore.r.01
- therefore X must be true
- the eggs were fresh and hence satisfactory
Synonyms: therefore, hence, thence, thus
Selected
-
10
in truth (often tends to intensify)
so%4:02:00:: · indeed.r.01
- they said the car would break down and indeed it did
- it is very cold indeed
Synonyms: indeed
The results of the studies showed that Cerenia was more effective than the placebo: less vomiting was seen in dogs that received the medicine than in dogs that received a placebo, both in the treatment or in the prevention of vomiting.
Show raw prompt
user
Which numbered sense of "more" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: The results of the studies showed that Cerenia was <t>more</t> effective than the placebo: less vomiting was seen in dogs that received the medicine than in dogs that received a placebo, both in the treatment or in the prevention of vomiting.
Senses:
1. definition=used to form the comparative of some adjectives and adverbs | examples=more interesting
2. definition=comparative of much; to a greater degree or extent | examples=he works more now
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
used to form the comparative of some adjectives and adverbs
more%4:02:00:: · more.r.01
- more interesting
- more beautiful
Synonyms: to a greater extent
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
comparative of much; to a greater degree or extent
more%4:02:01:: · more.r.02
- he works more now
- they eat more than they should
Selected
Cerenia was generally well tolerated when administered daily at dosages up to 5 times the recommended doses for an extended period of time (up to 3 times the recommended maximum duration of treatment).
Show raw prompt
user
Which numbered sense of "well" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: Cerenia was generally <t>well</t> tolerated when administered daily at dosages up to 5 times the recommended doses for an extended period of time (up to 3 times the recommended maximum duration of treatment).
Senses:
1. definition=(often used as a combining form) in a good or proper or satisfactory manner or to a high standard (`good' is a nonstandard dialectal variant for `well') | examples=the children behaved well
2. definition=thoroughly or completely; fully; often used as a combining form; ; ; ; ; , | examples=The problem is well understood
3. definition=indicating high probability; in all likelihood | examples=I might well do it
4. definition=(used for emphasis or as an intensifier) entirely or fully | examples=a book well worth reading
5. definition=to a suitable or appropriate extent or degree | examples=the project was well underway
6. definition=favorably; with approval | examples=their neighbors spoke well of them
7. definition=to a great extent or degree | examples=I'm afraid the film was well over budget
8. definition=with great or especially intimate knowledge | examples=we knew them well
9. definition=with prudence or propriety | examples=You would do well to say nothing more
10. definition=with skill or in a pleasing manner | examples=she dances well
11. definition=in a manner affording benefit or advantage | examples=she married well
12. definition=in financial comfort | examples=They live well
13. definition=without unusual distress or resentment; with good humor | examples=took the joke well
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
(often used as a combining form) in a good or proper or satisfactory manner or to a high standard (`good' is a nonstandard dialectal variant for `well')
well%4:02:00:: · well.r.01
- the children behaved well
- a task well done
Synonyms: good
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
thoroughly or completely; fully; often used as a combining form; ; ; ; ; ,
well%4:02:05:: · well.r.02
- The problem is well understood
- she was well informed
-
3
indicating high probability; in all likelihood
well%4:02:03:: · well.r.03
- I might well do it
- a mistake that could easily have ended in disaster
Synonyms: easily
-
4
(used for emphasis or as an intensifier) entirely or fully
well%4:02:15:: · well.r.04
- a book well worth reading
- was well aware of the difficulties ahead
-
5
to a suitable or appropriate extent or degree
well%4:02:08:: · well.r.05
- the project was well underway
- the fetus has well developed organs
-
6
favorably; with approval
well%4:02:07:: · well.r.06
- their neighbors spoke well of them
- he thought well of the book
-
7
to a great extent or degree
well%4:02:11:: · well.r.07
- I'm afraid the film was well over budget
- painting the room white made it seem considerably (or substantially) larger
Synonyms: considerably, substantially
-
8
with great or especially intimate knowledge
well%4:02:14:: · well.r.08
Synonyms: intimately
-
9
with prudence or propriety
well%4:02:13:: · well.r.09
- You would do well to say nothing more
- could not well refuse
-
10
with skill or in a pleasing manner
well%4:02:12:: · well.r.10
- she dances well
- he writes well
-
11
in a manner affording benefit or advantage
well%4:02:10:: · well.r.11
- she married well
- The children were settled advantageously in Seattle
Synonyms: advantageously
-
12
in financial comfort
well%4:02:09:: · well.r.12
- They live well
- she has been able to live comfortably since her husband died
Synonyms: comfortably
-
13
without unusual distress or resentment; with good humor
well%4:02:01:: · well.r.13
- took the joke well
- took the tragic news well
Selected
The parishioners of St. Michael and All Angels stop to chat at the church door, as members here always have.
Show raw prompt
user
Which numbered sense of "always" (marked with <t></t>) is used in the sentence below? Return only the number of the correct sense.
Sentence: The parishioners of St. Michael and All Angels stop to chat at the church door, as members here <t>always</t> have.
Senses:
1. definition=at all times; all the time and on every occasion | examples=I will always be there to help you
2. definition=without variation or change, in every case | examples=constantly kind and gracious
3. definition=without interruption | examples=the world is constantly changing
4. definition=at any time or in any event | examples=you can always resign if you don't like it
5. definition=forever; throughout all time | examples=we will always be friends
Prediction matches gold:
lexEN ✗
Maru 2022 ✗
Raganato ✗
WordNet 3.0 senses
-
1
at all times; all the time and on every occasion
always%4:02:00:: · always.r.01
- I will always be there to help you
- always arrives on time
Synonyms: ever, e'er
Gold lexEN
Gold Maru 2022
Gold Raganato
-
2
without variation or change, in every case
always%4:02:04:: · constantly.r.01
- constantly kind and gracious
- he always arrives on time
Synonyms: constantly, invariably
Selected
-
3
without interruption
always%4:02:03:: · constantly.r.02
- the world is constantly changing
Synonyms: constantly, forever, perpetually, incessantly
-
4
at any time or in any event
always%4:02:02:: · always.r.04
- you can always resign if you don't like it
- you could always take a day off
-
5
forever; throughout all time
always%4:02:01:: · always.r.05
- we will always be friends
- I shall treasure it always