Assessing the quality of the A3 thinking tool for problem solving

Alison Amos, Kinley Taylor, Kendra Johnson, Elizabeth Comitz, Robert Adams, Gregg Tracton, Bhishamjit Chera, Lawrence Marks, Lukasz Mazur

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The objective of this pilot study was to assess the inter-rater reliability of a newly developed A3 Quality Assessment (QA) rubric to evaluate the quality of completed Plan-Do-Study-Act (PDSA) projects that used an A3 Thinking Tool (A3) for problem solving. One A3 was independently reviewed by 7 PDSA experts using 5 main levels and 22 sublevels. Evaluations were compared and coded for agreement and used for statistical analysis. Fleiss’ kappa statistics was performed to test for inter-rater reliability between experts across 5 main and 22 sublevels. Preliminary results suggest that the A3 QA rubric meets reliability criteria with a moderate level of agreement beyond chance alone (κ = 0.44) and it is applicable to measure progress on problem solving abilities spearheaded via PDSA cycles. Additional verification testing is needed across multiple A3 improvement projects completed in multiple A3 Thinking templates.

LanguageEnglish (US)
Title of host publicationAdvances in the Human Side of Service Engineering - Proceedings of the AHFE International Conference on the Human Side of Service Engineering, 2016
EditorsTareq Z. Ahram, Waldemar Karwowski
PublisherSpringer Verlag
Pages49-61
Number of pages13
ISBN (Print)9783319419466
DOIs
StatePublished - Jan 1 2017
EventInternational Conference on The Human Side of Service Engineering, 2016 - Walt Disney World, United States
Duration: Jul 27 2016Jul 31 2016

Publication series

NameAdvances in Intelligent Systems and Computing
Volume494
ISSN (Print)2194-5357

Other

OtherInternational Conference on The Human Side of Service Engineering, 2016
CountryUnited States
CityWalt Disney World
Period7/27/167/31/16

Fingerprint

Statistical methods
Statistics
Testing

Keywords

  • A3 quality
  • A3 thinking
  • Lean
  • PDSA
  • Problem solving
  • Radiation oncology

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science(all)

Cite this

Amos, A., Taylor, K., Johnson, K., Comitz, E., Adams, R., Tracton, G., ... Mazur, L. (2017). Assessing the quality of the A3 thinking tool for problem solving. In T. Z. Ahram, & W. Karwowski (Eds.), Advances in the Human Side of Service Engineering - Proceedings of the AHFE International Conference on the Human Side of Service Engineering, 2016 (pp. 49-61). (Advances in Intelligent Systems and Computing; Vol. 494). Springer Verlag. https://doi.org/10.1007/978-3-319-41947-3_6

Assessing the quality of the A3 thinking tool for problem solving. / Amos, Alison; Taylor, Kinley; Johnson, Kendra; Comitz, Elizabeth; Adams, Robert; Tracton, Gregg; Chera, Bhishamjit; Marks, Lawrence; Mazur, Lukasz.

Advances in the Human Side of Service Engineering - Proceedings of the AHFE International Conference on the Human Side of Service Engineering, 2016. ed. / Tareq Z. Ahram; Waldemar Karwowski. Springer Verlag, 2017. p. 49-61 (Advances in Intelligent Systems and Computing; Vol. 494).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Amos, A, Taylor, K, Johnson, K, Comitz, E, Adams, R, Tracton, G, Chera, B, Marks, L & Mazur, L 2017, Assessing the quality of the A3 thinking tool for problem solving. in TZ Ahram & W Karwowski (eds), Advances in the Human Side of Service Engineering - Proceedings of the AHFE International Conference on the Human Side of Service Engineering, 2016. Advances in Intelligent Systems and Computing, vol. 494, Springer Verlag, pp. 49-61, International Conference on The Human Side of Service Engineering, 2016, Walt Disney World, United States, 7/27/16. https://doi.org/10.1007/978-3-319-41947-3_6
Amos A, Taylor K, Johnson K, Comitz E, Adams R, Tracton G et al. Assessing the quality of the A3 thinking tool for problem solving. In Ahram TZ, Karwowski W, editors, Advances in the Human Side of Service Engineering - Proceedings of the AHFE International Conference on the Human Side of Service Engineering, 2016. Springer Verlag. 2017. p. 49-61. (Advances in Intelligent Systems and Computing). https://doi.org/10.1007/978-3-319-41947-3_6
Amos, Alison ; Taylor, Kinley ; Johnson, Kendra ; Comitz, Elizabeth ; Adams, Robert ; Tracton, Gregg ; Chera, Bhishamjit ; Marks, Lawrence ; Mazur, Lukasz. / Assessing the quality of the A3 thinking tool for problem solving. Advances in the Human Side of Service Engineering - Proceedings of the AHFE International Conference on the Human Side of Service Engineering, 2016. editor / Tareq Z. Ahram ; Waldemar Karwowski. Springer Verlag, 2017. pp. 49-61 (Advances in Intelligent Systems and Computing).
@inproceedings{03e18633f7304875838bb0acbe8d48eb,
title = "Assessing the quality of the A3 thinking tool for problem solving",
abstract = "The objective of this pilot study was to assess the inter-rater reliability of a newly developed A3 Quality Assessment (QA) rubric to evaluate the quality of completed Plan-Do-Study-Act (PDSA) projects that used an A3 Thinking Tool (A3) for problem solving. One A3 was independently reviewed by 7 PDSA experts using 5 main levels and 22 sublevels. Evaluations were compared and coded for agreement and used for statistical analysis. Fleiss’ kappa statistics was performed to test for inter-rater reliability between experts across 5 main and 22 sublevels. Preliminary results suggest that the A3 QA rubric meets reliability criteria with a moderate level of agreement beyond chance alone (κ = 0.44) and it is applicable to measure progress on problem solving abilities spearheaded via PDSA cycles. Additional verification testing is needed across multiple A3 improvement projects completed in multiple A3 Thinking templates.",
keywords = "A3 quality, A3 thinking, Lean, PDSA, Problem solving, Radiation oncology",
author = "Alison Amos and Kinley Taylor and Kendra Johnson and Elizabeth Comitz and Robert Adams and Gregg Tracton and Bhishamjit Chera and Lawrence Marks and Lukasz Mazur",
year = "2017",
month = "1",
day = "1",
doi = "10.1007/978-3-319-41947-3_6",
language = "English (US)",
isbn = "9783319419466",
series = "Advances in Intelligent Systems and Computing",
publisher = "Springer Verlag",
pages = "49--61",
editor = "Ahram, {Tareq Z.} and Waldemar Karwowski",
booktitle = "Advances in the Human Side of Service Engineering - Proceedings of the AHFE International Conference on the Human Side of Service Engineering, 2016",
address = "Germany",

}

TY - GEN

T1 - Assessing the quality of the A3 thinking tool for problem solving

AU - Amos, Alison

AU - Taylor, Kinley

AU - Johnson, Kendra

AU - Comitz, Elizabeth

AU - Adams, Robert

AU - Tracton, Gregg

AU - Chera, Bhishamjit

AU - Marks, Lawrence

AU - Mazur, Lukasz

PY - 2017/1/1

Y1 - 2017/1/1

N2 - The objective of this pilot study was to assess the inter-rater reliability of a newly developed A3 Quality Assessment (QA) rubric to evaluate the quality of completed Plan-Do-Study-Act (PDSA) projects that used an A3 Thinking Tool (A3) for problem solving. One A3 was independently reviewed by 7 PDSA experts using 5 main levels and 22 sublevels. Evaluations were compared and coded for agreement and used for statistical analysis. Fleiss’ kappa statistics was performed to test for inter-rater reliability between experts across 5 main and 22 sublevels. Preliminary results suggest that the A3 QA rubric meets reliability criteria with a moderate level of agreement beyond chance alone (κ = 0.44) and it is applicable to measure progress on problem solving abilities spearheaded via PDSA cycles. Additional verification testing is needed across multiple A3 improvement projects completed in multiple A3 Thinking templates.

AB - The objective of this pilot study was to assess the inter-rater reliability of a newly developed A3 Quality Assessment (QA) rubric to evaluate the quality of completed Plan-Do-Study-Act (PDSA) projects that used an A3 Thinking Tool (A3) for problem solving. One A3 was independently reviewed by 7 PDSA experts using 5 main levels and 22 sublevels. Evaluations were compared and coded for agreement and used for statistical analysis. Fleiss’ kappa statistics was performed to test for inter-rater reliability between experts across 5 main and 22 sublevels. Preliminary results suggest that the A3 QA rubric meets reliability criteria with a moderate level of agreement beyond chance alone (κ = 0.44) and it is applicable to measure progress on problem solving abilities spearheaded via PDSA cycles. Additional verification testing is needed across multiple A3 improvement projects completed in multiple A3 Thinking templates.

KW - A3 quality

KW - A3 thinking

KW - Lean

KW - PDSA

KW - Problem solving

KW - Radiation oncology

UR - http://www.scopus.com/inward/record.url?scp=84986255578&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84986255578&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-41947-3_6

DO - 10.1007/978-3-319-41947-3_6

M3 - Conference contribution

SN - 9783319419466

T3 - Advances in Intelligent Systems and Computing

SP - 49

EP - 61

BT - Advances in the Human Side of Service Engineering - Proceedings of the AHFE International Conference on the Human Side of Service Engineering, 2016

A2 - Ahram, Tareq Z.

A2 - Karwowski, Waldemar

PB - Springer Verlag

ER -