Skill assessment: knobit registers, test items and answer grading
https://knobit.org/id/register/skill-assessment
Also as Turtle · JSON-LD · RDF/XML
| LCG purposes | Assessing the learner's knowledge or skill state |
|---|---|
| LCG purposes | Opening up a learning outcome |
| LCG purposes | Selecting an assessment task or learning activity |
| conforms to profile | https://oppekava.edu.ee/a/Knobitite_kirjeldamise_ja_seostamise_reeglid#LCG-KNOBIT-0.1 |
| dcterms:description | Reviewing a knobit register so that an assessment can infer from it, writing and reviewing rubric-graded test items whose criteria each observe one knobit set, and grading answers criterion by criterion as a calibrated grader would: the competence the skill-assessment skill applies, and that skill-tests assesses in the AI models it uses as judge, item writer and knobit developer. en |
| dcterms:identifier | SCOPE-1 |
| dcterms:license | https://creativecommons.org/licenses/by/4.0/ |
| dcterms:rights | Copyright 2026 Peep Küngas. Licensed under CC BY 4.0. |
| dcterms:title | Skill assessment: knobit registers, test items and answer grading en |
| owl:sameAs | https://bitbucket.org/reffa/ontology/skills/skill-assessment |
| schema:license | https://creativecommons.org/licenses/by/4.0/ |
| skos:note | audience: Assessment designers, item writers and graders, and AI models assessed by skill-tests for these roles en |
| skos:note | domain: Educational and psychological assessment, competence-based knowledge space theory, evaluation with LLM judges en |
| skos:note | level: Professional practice in test development and scoring as set by the Standards for Educational and Psychological Testing (2014), chapters 1, 2, 4 and 6, with knobit work to LCG-KNOBIT 0.1 en |
Items and references
Standards for Educational and Psychological Testing schema:CreativeWork
https://knobit.org/id/register/skill-assessment#REF-1
| dcterms:identifier | REF-1 |
|---|---|
| dcterms:publisher | American Educational Research Association, American Psychological Association and National Council on Measurement in Education |
| dcterms:title | Standards for Educational and Psychological Testing en |
| schema:version | 2014 |
| skos:note | target level: Professional practice in test development and scoring as set by the Standards for Educational and Psychological Testing (2014), chapters 1, 2, 4 and 6, with knobit work to LCG-KNOBIT 0.1 en |
OWASP Top 10 for LLM Applications schema:CreativeWork
https://knobit.org/id/register/skill-assessment#REF-3
| dcterms:identifier | REF-3 |
|---|---|
| dcterms:publisher | OWASP Foundation |
| dcterms:title | OWASP Top 10 for LLM Applications en |
| schema:version | 2025 |
| skos:note | target level: Professional practice in test development and scoring as set by the Standards for Educational and Psychological Testing (2014), chapters 1, 2, 4 and 6, with knobit work to LCG-KNOBIT 0.1 en |
sa-knobit-register-review C1: a cycle through K12-3 and K12-10 edu:Question
https://knobit.org/id/register/skill-assessment#TA-101
| dcterms:identifier | TA-101 |
|---|---|
| rdfs:label | sa-knobit-register-review C1: a cycle through K12-3 and K12-10 en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C1 (2 points) |
| solvable with | set of Recognises a cycle in the mastery implications |
sa-knobit-register-review C1: the cycle broken at the link that is not an implication edu:Question
https://knobit.org/id/register/skill-assessment#TA-102
| dcterms:identifier | TA-102 |
|---|---|
| rdfs:label | sa-knobit-register-review C1: the cycle broken at the link that is not an implication en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C1 (2 points) |
| solvable with | set of Repairs a relation wrongly recorded as a mastery implication |
sa-knobit-register-review C2: K12-7 checks the credit record and records the outcome in one knobit edu:Question
https://knobit.org/id/register/skill-assessment#TA-103
| dcterms:identifier | TA-103 |
|---|---|
| rdfs:label | sa-knobit-register-review C2: K12-7 checks the credit record and records the outcome in one knobit en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C2 (2 points) |
| solvable with | set of Recognises a compound knobit |
sa-knobit-register-review C2: K12-7 split into two single-action knobits edu:Question
https://knobit.org/id/register/skill-assessment#TA-104
| dcterms:identifier | TA-104 |
|---|---|
| rdfs:label | sa-knobit-register-review C2: K12-7 split into two single-action knobits en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C2 (2 points) |
| solvable with | set of Splits a compound knobit into single-action knobits |
sa-knobit-register-review C3: K12-2 and K12-9 one identity under two names edu:Question
https://knobit.org/id/register/skill-assessment#TA-105
| dcterms:identifier | TA-105 |
|---|---|
| rdfs:label | sa-knobit-register-review C3: K12-2 and K12-9 one identity under two names en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C3 (2 points) |
| solvable with | set of Recognises one knobit identity under two names |
sa-knobit-register-review C3: K12-9 merged into K12-2, its name in alt_names edu:Question
https://knobit.org/id/register/skill-assessment#TA-106
| dcterms:identifier | TA-106 |
|---|---|
| rdfs:label | sa-knobit-register-review C3: K12-9 merged into K12-2, its name in alt_names en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C3 (2 points) |
| solvable with | set of Merges two knobits of one identity |
sa-knobit-register-review I1: K12-8 analyses data edu:Question
https://knobit.org/id/register/skill-assessment#TA-107
| dcterms:identifier | TA-107 |
|---|---|
| rdfs:label | sa-knobit-register-review I1: K12-8 analyses data en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (2 points) |
| solvable with | set of Recognises a knobit with a generic target object |
sa-knobit-register-review I1: K12-3 depends on K12-1 because income is identified first edu:Question
https://knobit.org/id/register/skill-assessment#TA-108
| dcterms:identifier | TA-108 |
|---|---|
| rdfs:label | sa-knobit-register-review I1: K12-3 depends on K12-1 because income is identified first en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (2 points) |
| solvable with | set of Recognises a work-process order recorded as a prerequisite |
sa-knobit-register-review I1: K12-6's criterion 'understands responsible lending' edu:Question
https://knobit.org/id/register/skill-assessment#TA-109
| dcterms:identifier | TA-109 |
|---|---|
| rdfs:label | sa-knobit-register-review I1: K12-6's criterion 'understands responsible lending' en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (2 points) |
| solvable with | set of Recognises an assessment criterion no grader can observe |
sa-knobit-register-review I1: RE-4 with no coverage edu:Question
https://knobit.org/id/register/skill-assessment#TA-110
| dcterms:identifier | TA-110 |
|---|---|
| rdfs:label | sa-knobit-register-review I1: RE-4 with no coverage en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (2 points) |
| solvable with | set of Recognises a reference element with no coverage |
sa-knobit-register-review I2: K12-5 depends on K12-1, already implied through K12-4 edu:Question
https://knobit.org/id/register/skill-assessment#TA-111
| dcterms:identifier | TA-111 |
|---|---|
| rdfs:label | sa-knobit-register-review I2: K12-5 depends on K12-1, already implied through K12-4 en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I2 (1.5 points) |
| solvable with | set of Recognises a transitive shortcut link |
sa-knobit-register-review I2: K12-1 and K12-2 needed by the same items only edu:Question
https://knobit.org/id/register/skill-assessment#TA-112
| dcterms:identifier | TA-112 |
|---|---|
| rdfs:label | sa-knobit-register-review I2: K12-1 and K12-2 needed by the same items only en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I2 (1.5 points) |
| solvable with | set of Recognises knobits no item tells apart |
sa-knobit-register-review I2: D-3's rationale 'New.' edu:Question
https://knobit.org/id/register/skill-assessment#TA-113
| dcterms:identifier | TA-113 |
|---|---|
| rdfs:label | sa-knobit-register-review I2: D-3's rationale 'New.' en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I2 (1.5 points) |
| solvable with | set of Recognises an identity decision with no disputable rationale |
Share as a percentage edu:Question
https://knobit.org/id/register/skill-assessment#TA-1801
| dcterms:identifier | TA-1801 |
|---|---|
| rdfs:label | Share as a percentage en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:learningResourceType | calculation |
| schema:text | What percentage is 18 of 72? What percentage is 45 of 600? en |
| solvable with | set of Finds a part's share as a percentage |
Directed graph edu:Question
https://knobit.org/id/register/skill-assessment#TA-1802
| dcterms:identifier | TA-1802 |
|---|---|
| rdfs:label | Directed graph en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:learningResourceType | multiple_choice |
| schema:text | How does a directed graph differ from an ordinary graph? (a) its edges have a direction; (b) it has more nodes; (c) it has more edges; (d) its nodes have no degree en |
| solvable with | set of Defines a directed graph |
Cycle in a prerequisite graph edu:Question
https://knobit.org/id/register/skill-assessment#TA-1901
| dcterms:identifier | TA-1901 |
|---|---|
| rdfs:label | Cycle in a prerequisite graph en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:learningResourceType | multiple_choice |
| schema:text | Edges point from a knobit to its prerequisite: A -> B, B -> C, C -> D, D -> B, A -> D. Which knobits lie on a cycle? (a) B, C and D; (b) A, B and D; (c) A and D; (d) none: the graph is acyclic en |
| solvable with | set of Explains a directed graph and a cycle in it |
sa-item-review C1: C1 grades finding the duplicates and writing the merge rule as one edu:Question
https://knobit.org/id/register/skill-assessment#TA-201
| dcterms:identifier | TA-201 |
|---|---|
| rdfs:label | sa-item-review C1: C1 grades finding the duplicates and writing the merge rule as one en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C1 (2 points) |
| solvable with | set of Recognises a criterion grading two skills |
sa-item-review C1: the reference answer misses CU-07, which I2 requires edu:Question
https://knobit.org/id/register/skill-assessment#TA-202
| dcterms:identifier | TA-202 |
|---|---|
| rdfs:label | sa-item-review C1: the reference answer misses CU-07, which I2 requires en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C1 (2 points) |
| solvable with | set of Recognises a reference answer failing its own rubric |
sa-item-review C2: the prompt says CU-03 and CU-08 are the same person edu:Question
https://knobit.org/id/register/skill-assessment#TA-203
| dcterms:identifier | TA-203 |
|---|---|
| rdfs:label | sa-item-review C2: the prompt says CU-03 and CU-08 are the same person en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C2 (2 points) |
| solvable with | set of Recognises an answer given away in the prompt |
sa-item-review C2: C2 on survivorship mapped to a matching item edu:Question
https://knobit.org/id/register/skill-assessment#TA-204
| dcterms:identifier | TA-204 |
|---|---|
| rdfs:label | sa-item-review C2: C2 on survivorship mapped to a matching item en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C2 (2 points) |
| solvable with | set of Recognises a criterion mapped to the wrong item |
sa-item-review I1: I1 'shows a good understanding of matching' edu:Question
https://knobit.org/id/register/skill-assessment#TA-205
| dcterms:identifier | TA-205 |
|---|---|
| rdfs:label | sa-item-review I1: I1 'shows a good understanding of matching' en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (2 points) |
| solvable with | set of Recognises a criterion no grader can decide |
sa-item-review I1: the cap on the word limit edu:Question
https://knobit.org/id/register/skill-assessment#TA-206
| dcterms:identifier | TA-206 |
|---|---|
| rdfs:label | sa-item-review I1: the cap on the word limit en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (2 points) |
| solvable with | set of Recognises a cap on a peripheral criterion |
sa-item-review I1: valid YAML worth 4 of 12 points edu:Question
https://knobit.org/id/register/skill-assessment#TA-207
| dcterms:identifier | TA-207 |
|---|---|
| rdfs:label | sa-item-review I1: valid YAML worth 4 of 12 points en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (2 points) |
| solvable with | set of Recognises points out of proportion to the construct |
sa-item-review I1: C3 credits only the rule name MostRecentWins edu:Question
https://knobit.org/id/register/skill-assessment#TA-208
| dcterms:identifier | TA-208 |
|---|---|
| rdfs:label | sa-item-review I1: C3 credits only the rule name MostRecentWins en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (2 points) |
| solvable with | set of Recognises a criterion crediting only the reference's solution |
sa-item-review I2: threshold_absolute fails what C4 accepts edu:Question
https://knobit.org/id/register/skill-assessment#TA-209
| dcterms:identifier | TA-209 |
|---|---|
| rdfs:label | sa-item-review I2: threshold_absolute fails what C4 accepts en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I2 (1 points) |
| solvable with | set of Recognises an automatic check contradicting its criterion |
sa-item-review I2: name-only matching claimed with no rule matching on name edu:Question
https://knobit.org/id/register/skill-assessment#TA-210
| dcterms:identifier | TA-210 |
|---|---|
| rdfs:label | sa-item-review I2: name-only matching claimed with no rule matching on name en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I2 (1 points) |
| solvable with | set of Recognises a knobit claimed with no seeded case |
sa-item-review I3: C1 split into parts with an item each edu:Question
https://knobit.org/id/register/skill-assessment#TA-211
| dcterms:identifier | TA-211 |
|---|---|
| rdfs:label | sa-item-review I3: C1 split into parts with an item each en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I3 (1.5 points) |
| solvable with | set of Writes a rubric criterion observing one knobit set |
sa-item-review I3: a name-only rule seeded, or the claim removed edu:Question
https://knobit.org/id/register/skill-assessment#TA-212
| dcterms:identifier | TA-212 |
|---|---|
| rdfs:label | sa-item-review I3: a name-only rule seeded, or the claim removed en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I3 (1.5 points) |
| solvable with | set of Seeds a case for each claimed knobit in the task material |
sa-item-review I3: alternatives to MostRecentWins in the grading notes edu:Question
https://knobit.org/id/register/skill-assessment#TA-213
| dcterms:identifier | TA-213 |
|---|---|
| rdfs:label | sa-item-review I3: alternatives to MostRecentWins in the grading notes en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I3 (1.5 points) |
| solvable with | set of Writes grading notes with acceptable alternatives and named failure modes |
sa-item-authoring C1: each criterion or part observes one knobit set and names its item edu:Question
https://knobit.org/id/register/skill-assessment#TA-301
| dcterms:identifier | TA-301 |
|---|---|
| rdfs:label | sa-item-authoring C1: each criterion or part observes one knobit set and names its item en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C1 (2 points) |
| solvable with | set of Writes a rubric criterion observing one knobit set |
sa-item-authoring C1: no criterion bundles two of the three knobits edu:Question
https://knobit.org/id/register/skill-assessment#TA-302
| dcterms:identifier | TA-302 |
|---|---|
| rdfs:label | sa-item-authoring C1: no criterion bundles two of the three knobits en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C1 (2 points) |
| solvable with | set of Recognises a criterion grading two skills |
sa-item-authoring C2: a seeded case for each of the three knobits edu:Question
https://knobit.org/id/register/skill-assessment#TA-303
| dcterms:identifier | TA-303 |
|---|---|
| rdfs:label | sa-item-authoring C2: a seeded case for each of the three knobits en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C2 (2 points) |
| solvable with | set of Seeds a case for each claimed knobit in the task material |
sa-item-authoring C2: the prompt does not name the defects edu:Question
https://knobit.org/id/register/skill-assessment#TA-304
| dcterms:identifier | TA-304 |
|---|---|
| rdfs:label | sa-item-authoring C2: the prompt does not name the defects en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C2 (2 points) |
| solvable with | set of Recognises an answer given away in the prompt |
sa-item-authoring I1: criteria name what the answer must contain edu:Question
https://knobit.org/id/register/skill-assessment#TA-305
| dcterms:identifier | TA-305 |
|---|---|
| rdfs:label | sa-item-authoring I1: criteria name what the answer must contain en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (2 points) |
| solvable with | set of Recognises a criterion no grader can decide |
sa-item-authoring I1: the reference answer earns full marks on the rubric edu:Question
https://knobit.org/id/register/skill-assessment#TA-306
| dcterms:identifier | TA-306 |
|---|---|
| rdfs:label | sa-item-authoring I1: the reference answer earns full marks on the rubric en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (2 points) |
| solvable with | set of Recognises a reference answer failing its own rubric |
sa-item-authoring I1: the cap on a criterion central to the task edu:Question
https://knobit.org/id/register/skill-assessment#TA-307
| dcterms:identifier | TA-307 |
|---|---|
| rdfs:label | sa-item-authoring I1: the cap on a criterion central to the task en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (2 points) |
| solvable with | set of Recognises a cap on a peripheral criterion |
sa-item-authoring I1: format worth at most about a tenth of the points edu:Question
https://knobit.org/id/register/skill-assessment#TA-308
| dcterms:identifier | TA-308 |
|---|---|
| rdfs:label | sa-item-authoring I1: format worth at most about a tenth of the points en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (2 points) |
| solvable with | set of Recognises points out of proportion to the construct |
sa-item-authoring I2: grading notes with acceptable alternatives and named failure modes edu:Question
https://knobit.org/id/register/skill-assessment#TA-309
| dcterms:identifier | TA-309 |
|---|---|
| rdfs:label | sa-item-authoring I2: grading notes with acceptable alternatives and named failure modes en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I2 (1 points) |
| solvable with | set of Writes grading notes with acceptable alternatives and named failure modes |
sa-item-authoring I2: no criterion requires the reference's names edu:Question
https://knobit.org/id/register/skill-assessment#TA-310
| dcterms:identifier | TA-310 |
|---|---|
| rdfs:label | sa-item-authoring I2: no criterion requires the reference's names en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I2 (1 points) |
| solvable with | set of Recognises a criterion crediting only the reference's solution |
sa-answer-grading C1: A's evidence criterion scored 0: no document named edu:Question
https://knobit.org/id/register/skill-assessment#TA-401
| dcterms:identifier | TA-401 |
|---|---|
| rdfs:label | sa-answer-grading C1: A's evidence criterion scored 0: no document named en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C1 (2 points) |
| solvable with | set of Recognises credit given for what an answer does not say |
sa-answer-grading C1: every other score of A and C at its known grade edu:Question
https://knobit.org/id/register/skill-assessment#TA-402
| dcterms:identifier | TA-402 |
|---|---|
| rdfs:label | sa-answer-grading C1: every other score of A and C at its known grade en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C1 (2 points) |
| solvable with | set of Scores each criterion from its own evidence in the answer |
sa-answer-grading C2: B's ratio recomputed: 42%, not 38% edu:Question
https://knobit.org/id/register/skill-assessment#TA-403
| dcterms:identifier | TA-403 |
|---|---|
| rdfs:label | sa-answer-grading C2: B's ratio recomputed: 42%, not 38% en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C2 (2 points) |
| solvable with | set of Recognises a computed result taken on trust |
sa-answer-grading C2: B's approval scored 0, following the wrong ratio edu:Question
https://knobit.org/id/register/skill-assessment#TA-404
| dcterms:identifier | TA-404 |
|---|---|
| rdfs:label | sa-answer-grading C2: B's approval scored 0, following the wrong ratio en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C2 (2 points) |
| solvable with | set of Scores each criterion from its own evidence in the answer |
sa-answer-grading C3: B's note for the grading model changes no score edu:Question
https://knobit.org/id/register/skill-assessment#TA-405
| dcterms:identifier | TA-405 |
|---|---|
| rdfs:label | sa-answer-grading C3: B's note for the grading model changes no score en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C3 (2 points) |
| solvable with | set of Recognises an instruction in a graded answer |
sa-answer-grading C3: B's note reported as an instruction in the answer edu:Question
https://knobit.org/id/register/skill-assessment#TA-406
| dcterms:identifier | TA-406 |
|---|---|
| rdfs:label | sa-answer-grading C3: B's note reported as an instruction in the answer en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | C3 (2 points) |
| solvable with | set of Reports an instruction found in a graded answer |
sa-answer-grading I1: B's 190 confident words get no length credit edu:Question
https://knobit.org/id/register/skill-assessment#TA-407
| dcterms:identifier | TA-407 |
|---|---|
| rdfs:label | sa-answer-grading I1: B's 190 confident words get no length credit en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (1.5 points) |
| solvable with | set of Recognises length or confidence rewarded |
sa-answer-grading I1: B's other criteria not cut for the capped criterion edu:Question
https://knobit.org/id/register/skill-assessment#TA-408
| dcterms:identifier | TA-408 |
|---|---|
| rdfs:label | sa-answer-grading I1: B's other criteria not cut for the capped criterion en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (1.5 points) |
| solvable with | set of Recognises a cap applied by the grader |
sa-answer-grading I1: C's all-or-nothing evidence criterion not given half edu:Question
https://knobit.org/id/register/skill-assessment#TA-409
| dcterms:identifier | TA-409 |
|---|---|
| rdfs:label | sa-answer-grading I1: C's all-or-nothing evidence criterion not given half en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I1 (1.5 points) |
| solvable with | set of Recognises a score off the rubric's scale |
sa-answer-grading I2: A's 'recurring income' credited as regular income edu:Question
https://knobit.org/id/register/skill-assessment#TA-410
| dcterms:identifier | TA-410 |
|---|---|
| rdfs:label | sa-answer-grading I2: A's 'recurring income' credited as regular income en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I2 (1 points) |
| solvable with | set of Recognises an equivalent solution not credited |
sa-answer-grading I2: C's clumsy but correct ratio and decision credited edu:Question
https://knobit.org/id/register/skill-assessment#TA-411
| dcterms:identifier | TA-411 |
|---|---|
| rdfs:label | sa-answer-grading I2: C's clumsy but correct ratio and decision credited en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I2 (1 points) |
| solvable with | set of Recognises a halo in criterion scores |
sa-answer-grading I3: C's policy exception listed as the named failure mode edu:Question
https://knobit.org/id/register/skill-assessment#TA-412
| dcterms:identifier | TA-412 |
|---|---|
| rdfs:label | sa-answer-grading I3: C's policy exception listed as the named failure mode en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | I3 (0.5 points) |
| solvable with | set of Recognises a named failure mode left unreported |
sa-answer-grading Q1: no empty or generic reason edu:Question
https://knobit.org/id/register/skill-assessment#TA-413
| dcterms:identifier | TA-413 |
|---|---|
| rdfs:label | sa-answer-grading Q1: no empty or generic reason en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | Q1 (1 points) |
| solvable with | set of Recognises a grading reason citing no evidence |
sa-answer-grading Q1: each reason quotes or points to the answer edu:Question
https://knobit.org/id/register/skill-assessment#TA-414
| dcterms:identifier | TA-414 |
|---|---|
| rdfs:label | sa-answer-grading Q1: each reason quotes or points to the answer en |
| schema:isPartOf | Skill assessment: knobit registers, test items and answer grading |
| schema:isPartOf | Q1 (1 points) |
| solvable with | set of Writes a grading reason citing the answer |
Validity edu:LearningOutcome
https://knobit.org/id/register/skill-assessment#TV-1
| dcterms:description | Evidence that scores support their intended interpretation; construct under-representation and construct-irrelevant variance. en |
|---|---|
| dcterms:identifier | TV-1 |
| dcterms:source | Standards for Educational and Psychological Testing, ch. 1 en |
| includes knobit | Recognises an answer given away in the prompt |
| includes knobit | Recognises points out of proportion to the construct |
| includes knobit | Recognises a criterion crediting only the reference's solution |
| includes knobit | Recognises a knobit claimed with no seeded case |
| includes knobit | Seeds a case for each claimed knobit in the task material |
| includes knobit | Explains validity threats in test items |
| rdfs:label | Validity en |
| schema:isPartOf | Standards for Educational and Psychological Testing |
| skos:notation | ch. 1 |
Reliability and precision edu:LearningOutcome
https://knobit.org/id/register/skill-assessment#TV-2
| dcterms:description | Consistency of scores across graders and occasions; agreement between raters. en |
|---|---|
| dcterms:identifier | TV-2 |
| dcterms:source | Standards for Educational and Psychological Testing, ch. 2 en |
| includes knobit | Recognises a halo in criterion scores |
| includes knobit | Recognises length or confidence rewarded |
| includes knobit | Explains rater effects and grader calibration |
| rdfs:label | Reliability and precision en |
| schema:isPartOf | Standards for Educational and Psychological Testing |
| skos:notation | ch. 2 |
Test design and development edu:LearningOutcome
https://knobit.org/id/register/skill-assessment#TV-3
| dcterms:description | Specifications, item development, scoring rubrics and the review of items before use. en |
|---|---|
| dcterms:identifier | TV-3 |
| dcterms:source | Standards for Educational and Psychological Testing, ch. 4 en |
| includes knobit | Recognises a criterion grading two skills |
| includes knobit | Recognises a criterion no grader can decide |
| includes knobit | Recognises a reference answer failing its own rubric |
| includes knobit | Recognises a criterion mapped to the wrong item |
| includes knobit | Recognises a cap on a peripheral criterion |
| includes knobit | Recognises an automatic check contradicting its criterion |
| includes knobit | Writes a rubric criterion observing one knobit set |
| includes knobit | Writes grading notes with acceptable alternatives and named failure modes |
| includes knobit | Explains rubric-based test item design |
| rdfs:label | Test design and development en |
| schema:isPartOf | Standards for Educational and Psychological Testing |
| skos:notation | ch. 4 |
Scoring edu:LearningOutcome
https://knobit.org/id/register/skill-assessment#TV-4
| dcterms:description | Scoring rules applied consistently, by trained and monitored raters. en |
|---|---|
| dcterms:identifier | TV-4 |
| dcterms:source | Standards for Educational and Psychological Testing, ch. 6 en |
| includes knobit | Recognises credit given for what an answer does not say |
| includes knobit | Recognises an equivalent solution not credited |
| includes knobit | Recognises a score off the rubric's scale |
| includes knobit | Recognises a cap applied by the grader |
| includes knobit | Recognises a grading reason citing no evidence |
| includes knobit | Recognises a named failure mode left unreported |
| includes knobit | Recognises a computed result taken on trust |
| includes knobit | Scores each criterion from its own evidence in the answer |
| includes knobit | Writes a grading reason citing the answer |
| includes knobit | Explains analytic rubric scoring |
| rdfs:label | Scoring en |
| schema:isPartOf | Standards for Educational and Psychological Testing |
| skos:notation | ch. 6 |
Knobit description and identity edu:LearningOutcome
https://knobit.org/id/register/skill-assessment#TV-5
| dcterms:description | Knobits described by action, object, result and criterion, with identity decided by fingerprint and recorded with a rationale, and the target level covered. en |
|---|---|
| dcterms:identifier | TV-5 |
| dcterms:source | Knobitite kirjeldamise ja seostamise reeglid (LCG-KNOBIT), KN-01 to KN-06 en |
| includes knobit | Recognises a compound knobit |
| includes knobit | Recognises one knobit identity under two names |
| includes knobit | Recognises a knobit with a generic target object |
| includes knobit | Recognises an assessment criterion no grader can observe |
| includes knobit | Recognises a reference element with no coverage |
| includes knobit | Recognises an identity decision with no disputable rationale |
| includes knobit | Splits a compound knobit into single-action knobits |
| includes knobit | Merges two knobits of one identity |
| includes knobit | Explains the LCG knobit profile |
| rdfs:label | Knobit description and identity en |
| schema:isPartOf | https://oppekava.edu.ee/a/Knobitite_kirjeldamise_ja_seostamise_reeglid |
| skos:notation | KN-01 to KN-06 |
Relations between knobits edu:LearningOutcome
https://knobit.org/id/register/skill-assessment#TV-6
| dcterms:description | Applied knowledge, prerequisites and succession, each with its own meaning; the mastery implications acyclic. en |
|---|---|
| dcterms:identifier | TV-6 |
| dcterms:source | Knobitite kirjeldamise ja seostamise reeglid (LCG-KNOBIT), KN-07 en |
| includes knobit | Recognises a work-process order recorded as a prerequisite |
| includes knobit | Recognises a cycle in the mastery implications |
| includes knobit | Recognises a transitive shortcut link |
| includes knobit | Recognises knobits no item tells apart |
| includes knobit | Repairs a relation wrongly recorded as a mastery implication |
| includes knobit | Explains mastery implication in a knobit graph |
| rdfs:label | Relations between knobits en |
| schema:isPartOf | https://oppekava.edu.ee/a/Knobitite_kirjeldamise_ja_seostamise_reeglid |
| skos:notation | KN-07 |
Prompt injection edu:LearningOutcome
https://knobit.org/id/register/skill-assessment#TV-7
| dcterms:description | Content processed by a model that alters its behaviour; untrusted content kept apart from instructions. en |
|---|---|
| dcterms:identifier | TV-7 |
| dcterms:source | OWASP Top 10 for LLM Applications, LLM01 en |
| includes knobit | Recognises an instruction in a graded answer |
| includes knobit | Reports an instruction found in a graded answer |
| includes knobit | Explains prompt injection in graded content |
| rdfs:label | Prompt injection en |
| schema:isPartOf | OWASP Top 10 for LLM Applications |
| skos:notation | LLM01 |