Presentation ASME Sept 03

Slide2: 

3500+ Questions for formative and summative assessment

Slide3: 

Systems Index Clinical Situations CVS GI Chest Pain Collapse GMC Learning Outcomes, Manchester, Sheffield, Newcastle Taxonomies are in your pack for reference

Slide5: 

Combining assessment taxonomies GMC learning outcomes Questions tagged reviewed quality assured Empty cells targeted

Slide6: 

Writing Questions Feedback Answer options Question Review

Slide7: 

Welcome Group EMQ writing Writing EMQ answer options Reviewing your answer options Writing EMQ stems Reviewing your EMQ stems Comments, evaluation and close

Slide8: 

Greater depth of knowledge than MCQs and T/F More reliable assessment of knowledge Structure Theme Options Up to 26, A-Z, all logical choices under the chosen theme Lead In E.g. For each of the following patients with X, select the most likely answer from the options above. Each option may be used once, more than once or not at all. Stems – usually 5 Contextualised cases, typically patient scenarios each giving roughly similar case details. For each stem, one answer is correct, but all are feasible choices.

Slide9: 

Stem A 54 year old man, who has been a heavy smoker since a teenager, presents with a seizure. On recovery, he says he has been well apart from a headache which is worse on awakening in the morning and is relieved by paracetamol. The headache has been gradually getting worse over the last 3 - 4 weeks. Theme: Diagnosis of Headache Lead in For each patient with Headache, select the most suitable Diagnosis. Each option may be used once, more than once or not at all. A Aseptic meningitis J. Primary Brain tumour B Bacterial meningitis K. Infective endocarditis C Brain abscess L. Subarachnoid haemorrhage D Brain metastases M. Temporal arteritis E Cerebrovascular occlusion N. Tension headache G Cluster headache O. Trigeminal neuralgia H Hypertensive crisis P. Post herpetic neuralgia I. Migraine Q. Post exertional headache

Slide10: 

Feedback The correct option is D, brain metastases. This is because the combination of a seizure and recent onset of headache suggests a structural lesion in the brain associated with raised intracranial pressure. The long smoking history makes a secondary tumour more likely than a primary tumour.

Slide14: 

Theme: Basic Anatomy of the Heart A Aorta L Left Ventricle B Aortic Semilunar Valve M Ligmentum Arteriosum C Brachiocephalic Artery N Papillary Muscles D Chordae Tendineae O Pulmonary Semilunar Valve E Interventricular Septum P Pulmonary Trunk F Left Atrioventricular Valve Q Right Atrioventricular Valve G Left atrium R Right Atrium H Left Common Carotid Artery S Right Coronary Artery I Left Pulmonary Artery T Right Ventricle J Left Pulmonary Veins U Superior Vena Cava K Left Subclavian Artery V Trabeculae Carneae All Logical Options Under a Given Theme

Slide15: 

Theme: Psychiatric Therapy A. Amitriptyline and chlordiazepoxide B. Amitriptyline or chlordiazepoxide C. Chlordiazepoxide and Family therapy D. Family therapy without chlodiazepoxide E. Chlorpromazine F. Chlorpromazine in combination with cognitive behavioural therapy G. Family therapy without cognitive behavioural therapy H. Either Family therapy or cognitive behavioural therapy For each of the following patients, select the most appropriate therapy from the list of options. Each option may be used once, more than once or not at all. OPTIONS NOT MUTUALLY EXCLUSIVE and, or, if - only complicate the task!

Slide16: 

Theme: Treatment of Back Pain A Flucloxacillin 250mg QDS B Flucloxacillin 500mg PO C Flucloxacillin 1mg D Flucloxacillin 1 capsule QDS PO E Flucloxacillin 500mg IV F None of the above VALUES NOT EXPRESSED IN RANGES Avoid specific or overlapping choices! ‘NONE OF THE ABOVE’ Not a logically equivalent option!

Slide17: 

A B C D E F G H I J K L M N O 5-option 81 * 4 0 14 * 1 15-option 2 59 2 13 1 2 1 0 1 7 2 2 8 0 0 DISTRIBUTION OF EMQ SCORES FOR 1 QUESTION Better discriminator of knowledgeable students Case SM, Swanson DB. Evaluating diagnostic pattern recognition: a psychometric comparison of items with 15, 5, and 2 options. Paper presentation, Annual Meeting of the AERA. San Francisco, CA: March 1989.

Slide18: 

F K B J H G A E D I Least Most Correct Correct Answer Answer Options Represent a Single Continuum of Choice From Susan M. Case & David B. Swanson: Constructing Written Test Questions, 2002 http://www.nbme.org/about/itemwriting.asp.

Slide20: 

Be creative with your knowledge! e.g. clinical themes but also basic sciences, epidemiology, public health, communication skills Be creative in your focus! e.g. treatment and diagnosis but also actions, diagnostic tests, clinical signs List all possible logical members of this list, up to 26

Slide25: 

Theme: Cardiovascular Anatomy A Through his aorta B Through his stomach C Through his chest D Via cardiac surgery E The same as the way to a woman’s Lead in For each of the following, please complete the sentence. Stem The way to a man’s heart is… SUBJECTIVE Depends on what examiner was thinking! From Susan M. Case & David B. Swanson: Constructing Written Test Questions, 2002 http://www.nbme.org/about/itemwriting.asp.

Slide26: 

Theme: Medical Advice A. 8 glasses of water per day B. Be given too much salty food C. Take folic acid in early stages of pregnancy D. Heart disease when they are over 40yrs old and should take aspirin daily For each of the following, select the most appropriate advisory option from the list above. Each option may be used once, more than once or not at all. Adults should drink Babies should not Adult females should 4 Men are at risk of ONLY ONE FEASIBLE OPTION PER STEM Easy to discriminate the answer!

Slide27: 

Theme: Breast disease A. Lactational abscess E. Breast cancer in pregnancy B. Post menopausal breast cancer F. Pregnancy-induced mastalgia C. Pre-pubertal breast bud hypertrophy G. Normal breast D. Gynaecomastia H. None of the above For each of the following patients with breast complaints, select the most likely diagnosis from the list above. Each option may be used once, more than once or not at all. A 73 year old woman presents with a fixed hard mass in the upper outer quadrant of the right breast. There is an associated lymphadenopathy. A mammogram reveals a spiculated density corresponding to the clinical abnormality A 25 year old woman presents with a breast lump 6 weeks following the birth of her first child. The mass is 3cm in diameter, red and tender to touch. She has a raised leukocyte count BASIC INFO GIVES THE ANSWER AWAY Answer by elimination - knowledge not required!

Slide28: 

Theme: Management of Orthopaedic Complaint A. Discharge to GP F Prescription for NSAID B. Keep in for observation G Refer for physiotherapy C. Admit to hospital Section under Mental Health Act Prepare for surgery For each of the following, select the most appropriate management option from the list above. Each option may be used once, more than once or not at all. Patient presents with a fractured rib and complains of frequent vomiting. A 16 year old female inpatient who has made significant progress following an operation to correct a double fractured femur, but requires high dosages of codeine to control her pain. On examination she is unsteady and on moving to a sitting position vomits copiously. Her WCC is 13,000. STEMS NOT HOMOGENOUS Stems should reinforce same core points!

Slide29: 

Stems should be long, options should be short Stems should stimulate synthesis of information Not too much or too subtle information that knowledgeable students will be disadvantaged

Slide30: 

What is the most likely renal abnormality in children with nephrotic syndrome and normal renal function? A B *C* D E Overall P-value 1 0 99 0 0 94 8 1 90 1 0 A 2-year-old boy has a 1-week history of oedema. Blood pressure is 100/60 mm Hg, and there is generalized oedema and ascites. Serum concentrations are: creatinine 0.4 mg/dL, albumin 1.4 g/dL, and cholesterol 569 mg/dL. Urinalysis shows 4+ protein and no blood. What is the most likely diagnosis? 0 0 98 2 0 88 5 2 82 8 1 A 2-year-old black child developed swelling of his eyes and ankles over the past week. Blood pressure is 100/60 mm Hg, pulse 110/min, and respirations 28/min. In addition to swelling of his eyes and 2+ pitting edema of his ankles, he has abdominal distension with a positive fluid wave. Serum concentrations are: creatinine 0.4 mg/dL, albumin 1.4 g/dL, and cholesterol 569 mg/dL. Urinalysis shows 4+ protein and no blood. 0 1 98 1 0 84 10 9 66 10 5 From Susan M. Case & David B. Swanson: Constructing Written Test Questions, 2002 http://www.nbme.org/about/itemwriting.asp.

Slide32: 

Pitch content at HO level knowledge Write up to 6 stems per set of options Feedback explaining why the answer is correct

Slide36: 

Relevant knowledge and applied knowledge Each option a feasible answer to each stem Correct answer? Similar information given in each of the stems Cover the options - can the knowledgeable student still arrive at the correct answer? Is the answer detectable by elimination?

Slide38: 

Question Review Questions checked Tagging formalised

Slide39: 

Any comments? Please use the form at the back of the pack Questions to be made available to all via LTSN. Please fill out the question writer information to keep you informed of access arrangements. We will collect completed packs from you Please place all documents in your packs