This project explores the use of conformal prediction to safely deploy AI tools in medical coding. The goal is to create a system that can be used to assist medical coders in their work, without replacing them. The system uses large language models to generate suggestions for codes, and conformal prediction to provide a measure of confidence in those suggestions. This allows the system to be used in a graded automation setting, where the coder can choose to accept or reject the suggestions based on their confidence level.

In contrast with existing paradigms which hand-design rules or protocols specifying domain of application of a model or assay, in this work the inputs to which an AI application can be safely applied are determined automatically, allowing the model to abstain on the most difficult examples until the designed performance specifications can be gauranteed. This allows for a more flexible and robust system, which can adapt automatically overtime to additional use-cases as the model improves (again, with certifiable analytic validity).