Inter-operator variability and source of errors in tumour response assessment for hepatocellular carcinoma treated with sorafenib

OBJECTIVES: To assess the inter-operator concordance and the potential sources of discordance in defining response to sorafenib in hepatocellular carcinoma (HCC). METHODS: All patients who received sorafenib between September 2008 and February 2015 were scrutinised for this retrospective study. Images were evaluated separately by three radiologists with different expertise in liver imaging (operator 1, >10 years; operator 2, 5 years; operator 3, no specific training in liver imaging), according to: response evaluation radiological criteria in solid tumours (RECIST) 1.1, modified RECIST (mRECIST) and response evaluation criteria in cancer of the liver (RECICL). RESULTS: The overall response concordance between the more expert operators was good, irrespective of the criteria (RECIST 1.1, ĸ = 0.840; mRECIST, ĸ = 0.871; RECICL, ĸ = 0.819). Concordance between the less expert operator and the other colleagues was lower. The most evident discordance was in target lesion response assessment, with expert operators disagreeing mostly on lesion selection and less expert operators on lesion measurement. As a clinical correlate, overall survival was more tightly related with "progressive disease" as assessed by the expert compared to the same assessment performed by operator 3. CONCLUSIONS: Decision on whether a patient is a responder or progressor under sorafenib may vary among different operators, especially in case of a non-specifically trained radiologist. Regardless of the adopted criteria, patients should be evaluated by experienced radiologists to minimise variability in this critical instance. KEY POINTS: • Inter-operator variability in the assessment of response to sorafenib is poorly known. • The concordance between operators with expertise in liver imaging was good. • Target lesions selection was the main source of discordance between expert operators. • Concordance with non-specifically trained operator was lower, independently from the response criteria. • The non-specifically trained operator was mainly discordant in measurements of target lesions.