 Research
 Open Access
 Published:
Predictive validity of a novel noninvasive estimation of effective shunt fraction in critically ill patients
Intensive Care Medicine Experimental volume 7, Article number: 49 (2019)
 The Correction to this article has been published in Intensive Care Medicine Experimental 2020 8:30
Abstract
Background
Accurate measurement of pulmonary oxygenation is important for classification of disease severity and quantification of outcomes in clinical studies. Currently, tensionbased methods such as P/F ratio are in widespread use, but are known to be less accurate than contentbased methods. However, contentbased methods require invasive measurements or sophisticated equipment that are rarely used in clinical practice. We devised two new methods to infer shunt fraction from a single arterial blood gas sample: (1) a noninvasive effective shunt (ES) fraction calculated using a rearrangement of the indirect Fick equation, standard constants, and a procedural inversion of the relationship between content and tension and (2) inferred values from a database of outputs from an integrated mathematical model of gas exchange (DB). We compared the predictive validity—the accuracy of predictions of P_{a}O_{2} following changes in F_{I}O_{2}—of each measure in a retrospective database of 78,159 arterial blood gas (ABG) results from critically ill patients.
Results
In a formal test set comprising 9,635 pairs of ABGs, the median absolute error (MAE) values for the four measures were as follows: alveolararterial difference, 7.30 kPa; P_{a}O_{2}/F_{I}O_{2} ratio, 2.41 kPa; DB, 2.13 kPa; and ES, 1.88 kPa. ES performed significantly better than other measures (p < 1010 in all comparisons). Further exploration of the DB method demonstrated that obtaining two blood gas measurements at different F_{I}O_{2} provides a more precise description of pulmonary oxygenation.
Conclusions
Effective shunt can be calculated using a computationally efficient procedure using routinely collected arterial blood gas data and has better predictive validity than other analytic methods. For practical assessment of oxygenation in clinical research, ES should be used in preference to other indices. ES can be calculated at http://baillielab.net/es.
Background
Hypoxia is the defining feature of respiratory failure. Accurate quantification of pulmonary oxygenation defect is essential to determine inclusion in clinical trials, to measure outcomes in research studies, and to observe changes in lung function in a clinical setting.
In severely hypoxic patients, direct measurement of intrapulmonary shunt provides the most accurate quantification of an oxygenation defect [1]. Tensionbased indices, including P_{a}O_{2}/F_{I}O_{2} (P/F) ratio and alveolararterial (Aa) difference have poor agreement with intrapulmonary shunt fraction [1–3]. The primary limitation in tensionbased indices is the marked and nonlinear change in P_{a}O_{2} when F_{I}O_{2} is changed [4]. Brochard and colleagues demonstrated that this can be predicted from a simple mathematical model [5].
The concept of predictive validity is a mathematical reality check for a clinical measure. For a given clinical measure, predictive validity quantifies the extent to which that measure predicts an unseen event. The intent is not to predict the future, but rather to provide a rigourous, unbiased test of how well a clinical measure is describing a real entity: the assumption is that whichever measure is closest to the truth should also provide the best prediction. This approach, using mortality as the predicted event, was used in the development of consensus definitions for both acute respiratory distress syndrome (ARDS) [6] and sepsis [7].
A measure that accurately reflects the true state of a patient’s lungs should not change markedly following a change in F_{I}O_{2}. Therefore, the prediction of a P_{a}O_{2} following a change in F_{I}O_{2}, assuming that the measure of the oxygenation defect remains unaltered, is a valid assessment for predictive validity.
We hypothesised that an easily understood, contentbased oxygenation index may be obtainable from routinelyacquired arterial blood gas (ABG) data, without any need for additional invasive measurements. In order to assess different approaches, we quantified the predictive validity of P/F, Aa, and two new methods of estimating shunt fraction (effective shunt fraction (ES) and a database method (DB)) in a simple test: prediction of P_{a}O_{2} following a change in F_{I}O_{2} in a large retrospective cohort.
Methods
Data source and filtering
We used a set of 78,159 arterial blood gas samples taken between 2011 and 2016 from 6511 patients on the general intensive care unit (ICU) at the Royal Infirmary of Edinburgh. The unit admits adult patients, with predominantly emergency medical, trauma, and general surgery conditions—not elective cardiac or thoracic surgery or neurosurgery. We did not study patients who had ECMO. The samples were routine analyses: the F_{I}O_{2} value was input by the clinician performing the analysis. The analysis machine was maintained by the clinical chemistry department and regularly calibrated against known standards.
To obtain sample sets in which underlying pulmonary pathology was unlikely to change substantially between samples, we limited the selection of samples to pairs of ABGs that met the following inclusion criteria: (1) taken within a 3h window, (2) taken from a mechanically ventilated patient, (3) where the F_{I}O_{2} was reduced between the first and the second sample, and (4) where alveolar ventilation was stable (change in P_{a}CO_{2} < 0.3 kPa).
Derivation of effective shunt fraction
ES expresses the shunt fraction that would be required to produce a given impairment in oxygenation, that is the proportion of cardiac output that would have to shunt in order to have this effect (i.e. to produce this degree of hypoxia). In clinical practice, it will almost never be the case that a given patient has pure shunt; ES is intended to provide an intuitive and consistent quantification of oxygenation impairment.
Full details of the methods used are given in the Additional file 1. Briefly, ES was first calculated from the blood gas results as follows.
The shunt equation is usually expressed in the following way [8]:
All of the necessary variables can be easily calculated from routine clinical measurements, with the exception of C_{v}O_{2}. We applied the Fick principle for oxygen uptake (see Additional file 2), in order to replace this term:
After estimating P_{A}O_{2} using the alveolar gas equation, arterial (C_{a}O_{2}) and endcapillary (C_{c’}O_{2}) oxygen contents were derived using model equations from Dash and Bassingthwaighte [9], using measured pH and P_{a}CO_{2} to estimate the PO_{2} at which Hb is 50% saturated (P_{50}). Values for oxygen consumption (VO_{2}) and cardiac output (Q) were set at single values in the physiological range (see Additional file 1).
Predictive validity
Each measure was quantified for the earlier ABG in each pair. For ES, P/F and Aa, the F_{I}O_{2} and P_{A}O_{2} for the second ABG was used in the rearranged equations (derived in Additional file 1) to estimate the new P_{a}O_{2}, using the same value for the oxygenation index under inspection.
For measurement of predictive validity in the ES method, a predicted C_{a}O_{2}, after change of F_{I}O_{2}, was calculated using F_{I}O_{2} and P_{A}O_{2} from the second ABG:
This value of C_{a}O_{2} was then converted to a predicted P_{a}O_{2} value using the method of Dash and Bassingthwaighte [9].
In order to minimise the noise generated (affecting all measures) by changes in ventilation or circulation, we focused our study on patients whose F_{I}O_{2} was being weaned downwards. The median absolute differences between predicted and observed P_{a}O_{2} across all ABG pairs were taken as the predictive validity for each measure. For the DB method, input settings for an integrated mathematical model of gas exchange were identified which matched to the first ABG. These were then extrapolated to the F_{I}O_{2} and P_{a}CO_{2} of the second ABG, and the mean P_{a}O_{2} of all matching model runs taken to be the prediction (Fig. 1b). Further details of the mathematical model used for the DB method can be found in supplementary content (see Additional file 2).
Research Ethics
Ethical approval was obtained from the Scotland A Research Ethics Committee [16/SS/0209].
Software and statistical analyses
All analysis was performed using Python 3.5.2 and scipy.stats version 0.18.1. A KruskalWallis H test was used to determine the difference in error rate between the different measures. MannWhitney U tests with Bonferroni correction were used as a stringent post test for pairwise comparisons.
Results
Comparison of oxygenation measures
From the total set of 78,159 ABGs from 6511 patients, an initial test set was selected at random, containing 54,115 ABGs from 4558 patients. From this random sample, we selected a formal test set comprising 9635 pairs of ABGs, which met the criteria listed above, pairs of ABGs taken from mechanically ventilated patients within a 3h window, where the F_{I}O_{2} was reduced between the first and the second sample, and where alveolar ventilation was stable. When we compared the predicted with the measured P_{a}O_{2} values in the second values of these pairs, the median absolute error (MAE) values for the four measures considered were as follows: Aa, 7.30 kPa; P/F, 2.41 kPa; DB, 2.13 kPa; and ES, 1.88 kPa. ES had significantly superior predictive validity than all other measures (Table 1).
Effective shunt values in this population ranged from 0 to 63% (mean 16.1%, SD 8.6%). P/F values (kPa) ranged from 6.65 to 84.7 (mean 32.2, SD 12.5). Aa values (kPa) ranged from 0 to 81.2 (mean 22.7, SD 13.6).
Validation of assumed values
Three key assumed values are required for the calculation of ES: respiratory exchange ratio (RER), cardiac output (Q), and metabolic oxygen consumption (VO_{2}). In order to prevent bias, we optimised estimation of these assumed parameters in a training set comprising 30% of the available ABGs (n=24,044), selected at random. Varying the three values over wide ranges (RER, 0.8 to 1.1; Q, 3 to 15 l.min^{−1}; VO_{2}, 0.15 to 1 l.min^{−1}) had minimal effect on the value of effective shunt.
Multiple ABGs under different conditions
Although the DB method does not perform as well as the simpler and less computationally demanding ES method, it provides an opportunity to test the effect of obtaining multiple ABGs at different F_{I}O_{2}. The model takes standard physiological inputs, including pure shunt fraction, V/Q heterogeneity index, cardiac output, and F_{I}O_{2} and returns blood gas results at steady state. The database of model results enables us to infer the possible physiological conditions that could give rise to a given ABG result. With one ABG, a wide range of possibilities remain (Fig. 1b); a second ABG at a different F_{I}O_{2} substantially constrains the range of possible physiological states that describe a given patient (Fig. 1c).
Discussion
This is the first study, to our knowledge, comparing the predictive validity of noninvasive effective shunt fraction with tensionbased measures. Our observations are consistent in magnitude and direction with previous work studying changes in measures of oxygenation in human participants [1–3].
The poor performance of Aa difference is consistent with the report by Cane and colleagues [1], who demonstrated that Aa was the least reliable measure compared with invasive measurements of Qs/Qt. The very low predictive validity of Aa in our study, together with previous work, leads us to conclude that this measure has no role in any context.
Our study has several limitations. The ABG data itself was obtained from electronic records whereby F_{I}O_{2} was entered by the treating clinician (nurse/doctor/nurse practitioner), which is a possible source of error. Importantly, this potential error applies equally to all measures of oxygenation.
To compare the integrity of each oxygenation measure alone, we assumed that baseline physiological function is not altered by the change of F_{I}O_{2}. However, increases in F_{I}O_{2} cause absorption atelectasis [10]. We have mitigated this by restricting our analysis to pairs of ABGs in which the F_{I}O_{2} was decreasing. Since the reversal of absorption atelectasis is slower than the onset [10], and there are fewer sudden changes in oxygenation in this group, we expect that restricting our analysis to weaning patients will mitigate this source of noise.
Marked changes in P_{a}O_{2} may occur within a 3h interval due to real changes in pulmonary function, for example due to recruitment, suction, diuresis, or change in posture. We therefore cannot draw any inference from the absolute value of the error in prediction, only a comparison between different methods. Noise caused by these and other factors is expected to limit the maximum possible accuracy of any prediction of P_{a}O_{2}. This minimum achievable error is reflected by the baseline values (Fig. 1a) showing the change in P_{a}O_{2} between pairs of ABGs meeting the other selection criteria, with no change in F_{I}O_{2}.
There is also a significant limitation in the concept of reducing the full complexity of pulmonary oxygenation to a single numerical value. All clinical measurements are subject to this limitation—they provide summary measurements that require informed interpretation. The lung is no different from any other system in this regard. Studies using the multiple gas elimination technique (MIGET) have confirmed that lung injury leads to substantial heterogeneity in the matching of ventilation to perfusion, which causes hypoxia without pure shunt [11, 12]. ES, by design, combines these mechanisms into a single value from a threecompartment model: the amount of pure shunt that would be needed to have a given effect on oxygenation.
Since V/Q heterogeneity and shunt are separate inputs into the physiological model used to generate the database (see Additional file 1), the database approach is expected to handle this distinction better than the other measures. However, as shown in Fig. 1b, there is insufficient information in a single ABG to distinguish between shunt and V/Q heterogeneity. In contrast, with two ABGs taken at different settings of F_{I}O_{2}, the patient’s oxygen responsiveness is quantified, greatly restricting the range of possible values for both shunt and V/Q heterogeneity (Fig. 1c). This double F_{I}O_{2} test may resolve the uncertainty in quantifying pulmonary shunt but is, at present, computationally demanding.
The striking superiority of ES in the context of critical illness may lead to an increase in clinical use. We support this, in part because the value itself is intuitive to critical care clinicians and is comparable across different health care systems and measurement units. Although it performs substantially better than other measures, it should be noted that ES is an imperfect measure and is not expected to be completely independent of extrapulmonary factors, including F_{I}O_{2}, alveolar ventilation, and intracardiac shunt.
Conclusions
The effective shunt fraction, a new, noninvasive method of estimating shunt, can be calculated on any ABG result, provided the F_{I}O_{2} is known. The computation is fast and simple. Hence, the method could be retrospectively applied to previous studies that hold ABG data in a machinereadable format. Whilst the simplicity of the P/F ratio will continue to make it a popular choice for clinical use, the superior predictive validity of ES makes it a better choice where accurate quantification of oxygenation defect is necessary.
Availability of data and materials
An online calculator to compute the effective shunt fraction is available at: http://baillielab.net/es Python code to calculate the effective shunt fraction is available from github: http://github.com/baillielab
Change history
10 July 2020
An amendment to this paper has been published and can be accessed via the original article.
Abbreviations
 Aa:

Alveolararterial difference
 ABG:

Arterial blood gas
 ARDS:

Acute respiratory distress syndrome
 C_{a}O_{2} :

Arterial blood oxygen content
 C_{c’}O_{2} :

Endcapillary blood oxygen content
 DB:

Database method
 ES:

Effective shunt
 F_{I}O_{2}; ICU:

Intensive care unit
 MWu:

MannWhitney U test
 P/F:

The ratio of P_{a}O_{2} in the arterial blood to the therapeutic MAE: Median absolute error
 Q:

Cardiac output
 Qs/Qt:

Shunt fraction
 RER:

Respiratory exchange ratio
 V/Q:

Ventilation/perfusion ratio
 VO_{2} :

Metabolic oxygen consumption
References
 1
Cane RD, Shapiro BA, Templin R, et al. (1988) Unreliability of oxygen tensionbased indices in reflecting intrapulmonary shunting in critically ill patients. Crit Care Med 16:1243–5.
 2
Drummond GB, Zhong NS (1983) Inspired oxygen and oxygen transfer during artificial ventilation for respiratory failure. Br J Anaesth 55:3–13.
 3
AllardetServent J, Forel JM, Roch A, et al. (2009) FIO2 and acute respiratory distress syndrome definition during lung protective ventilation. Crit Care Med 37:e4–6. https://doi.org/10.1097/CCM.0b013e31819261db.
 4
Gowda MS, Klocke RA (1997) Variability of indices of hypoxemia in adult respiratory distress syndrome. Crit Care Med 25:41–5.
 5
Aboab J, Louis B, Jonson B, et al. (2006) Relation between pao2/fio2 ratio and fio2: a mathematical description. Intensive Care Med 32:1494–7. https://doi.org/10.1007/s0013400603379.
 6
ARDS Definition Task Force, Ranieri VM, Rubenfeld GD, et al. (2012) Acute respiratory distress syndrome: the Berlin Definition. JAMA 307:2526–33. https://doi.org/10.1001/jama.2012.5669.
 7
ShankarHari M, Phillips GS, Levy ML, et al. (2016) Developing a new definition and assessing new clinical criteria for septic shock: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis3). JAMA 315:775–87. https://doi.org/10.1001/jama.2016.0289.
 8
Benatar SR, Hewlett AM, Nunn JF (1973) The use of isoshunt lines for control of oxygen therapy. Br J Anaesth 45:711–8.
 9
Dash RK, Bassingthwaighte JB (2010) Erratum to: Blood hbo2 and hbco2 dissociation curves at varied o2, co2, pH, 2,3dpg and temperature levels. Ann Biomed Eng 38:1683–701. https://doi.org/10.1007/s104390109948y.
 10
Santos C, Ferrer M, Roca J, et al. (2000) Pulmonary gas exchange response to oxygen breathing in acute lung injury. Am J Respir Crit Care Med 161:26–31. https://doi.org/10.1164/ajrccm.161.1.9902084.
 11
Duenges B, Vogt A, Bodenstein M, et al. (2009) A comparison of micropore membrane inlet mass spectrometryderived pulmonary shunt measurement with riley shunt in a porcine model. Anesth Analg 109:1831–5. https://doi.org/10.1213/ANE.0b013e3181bbc401.
 12
Wagner PD (2008) The multiple inert gas elimination technique (MIGET). Intensive Care Med 34:994–1001. https://doi.org/10.1007/s0013400811086.
Acknowledgments
This research was supported by The University of Edinburgh and NHS Lothian.
Funding
JKB is grateful to acknowledge funding support from a Wellcome Trust Intermediate Clinical Fellowship (103258/Z/13/Z) and a WellcomeBeit Prize (103258/Z/13/A), BBSRC Institute Strategic Programme Grant to the Roslin Institute, and the UK Intensive Care Society. A.B. is grateful to acknowledge funding from Edinburgh Clinical Academic Track and funding from the Wellcome Trust (204979/Z/16/Z).
Author information
Affiliations
Contributions
JKB and EMC designed the study and conducted the computational analysis. JKB and AB wrote the computational model of gas exchange. GBD contributed to the conception of hypothesis and interpretation of the results. JKB, EMC, and AB wrote the manuscript with assistance from GBD. All authors commented on or contributed to the final manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Ethical approval was obtained from the Scotland A Research Ethics Committee [16/SS/0209].
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional files
Additional file 1
Supplementary information. Includes the derivation of oxygenation measures and method of predicting P_{a}O_{2}, as well as optimisation of assumed variables using the test dataset. (PDF 227 kb)
Additional file 2
Integrated model of gas exchange. Structure of an integrated computational model of oxygen delivery which was used to generate a database of model results for inferring the possible physiological conditions that could give to rise to any given ABG result (“DB” method in the main manuscript). (PDF 80 kb)
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Chang, E.M., Bretherick, A., Drummond, G.B. et al. Predictive validity of a novel noninvasive estimation of effective shunt fraction in critically ill patients. ICMx 7, 49 (2019). https://doi.org/10.1186/s4063501902621
Received:
Accepted:
Published:
Keywords
 Oxygen
 Shunt fraction
 Arterial blood gas
 Respiratory failure