WPCVA ouIG:f=̅H.l+C048߸cľձ_NQ#vP4h $Mk"~kf8f3 HWvce2}YBHM7M NC즻cpw2*񰵵(]ɢőxd?]3&Oq0kL||^fD N|9"&= -hύܭ* 0Nr bz*諘Ի^$7V1LΔ2s[LǮL5ȯp;VV{4x?P1YI/hifYi8Ita,d(@ߵ7ׯ *G-0~-(,\V8B1!mPcIr1+ +-U(~+ 0v+ 0`X, 0`, 0`- 0`x- 0`- 088. 0p. 02B/ 0t/ 0:p0 0Y0 0_1 0 b1 0jl2 03 B*Z5 D/5 05 D/77 0f7 0648 0j8 0:r9 0^9 0< : 0NF:U6:U>:U*; A2; 0; 0g< 0= 0= 0> 0]? 1L@L@ DC@@U@A<6X9`(CourierX'  . 2  0 .   h``:Default ParaDefault Paragraph Font                                                                                                                                0 .FOOTER   !(#Q5\  `$Times NewRomanQ;1Q Y ai #;Q5\  `$Times NewRomanQ  5\  `$Times NewRoman0 .HEADER   !(#Q5\  `$Times NewRomanQ;1Q Y ai #;Q5\  `$Times NewRomanQ  < :FOOTNOTE TEX  ;14 <DL!T$;Q5\  `$Times NewRomanQ8.4 <DL!(#8Q5\  `$Times NewRomanQ  &uu$1  Q5\  `$Times NewRomanQQ5\  `$Times NewRomanQ  20Header = *=   !Q5\  `$Times NewRomanQ /  H/  >44 <DL!>XXXQ5\  `$Times NewRomanQ  d 'dxd'dxd4 y2Header2    9 "9    5+ 4 <DL!5  W'\  `*Times New RomanTTW  nW'\  `*Times New RomanTTW     /  H/5+ 4 <DL!5XXXQ5\  `$Times NewRomanQ   '\  `*Times New RomanTT4#2Header3  d     &&&Q5\  `$Times NewRomanQ  &&&Q5\  `$Times NewRomanQ   XXXQ5\  `$Times NewRomanQ   4&2Header4  d     &&&Q5\  `$Times NewRomanQ  &&&Q5\  `$Times NewRomanQ   XXXQ5\  `$Times NewRomanQ   4 2Quick 1..0..xhH  Z6Times New Roman Regular#$37;?CGKOS-*+xHP LaserJet 2100 Series PS0$35;AGMU]c2I.A.1.a.(1)(a)i)a)+,- &OLE 2.0 Box <=8C HKKKKW*3|O<6X9`(CourierXx6X@`7X@<6X9`(CourierNv6X@`7N@5\  `$Times NewRomanX\  P6G;P5\  `$Times NewRoman\  P6G;P5\  `$Times NewRomanXXj\  P6G;XP'\  `*Times New RomanTT[\  PEP5\  `$Times NewRomanTT&&a\  P6G;&P%\  `$Times NewRomangularXXj\  P6G;XP(J$N^XXXN^XXN^  Eko):i+00U Centre ANNs"backpropagation MSEBayes BaxtBuchmanchronicityGuerriereBuskard&compartmentalised MIRGNightengaleSolvenintensivistChalmers DECHDBasePOSTOPNONPOSTOPContinousMatlabfeedforwardHaykinWidrowoverfittingTrigg CCRMahalanobis MMDCmammograms MDCnonpostsubdatabaseCivetta NICUdiscriminant$AcknowledgementsNSERC PGS MRC CGAAMaryhelen ICUs GCSLippmannRPWGGRTGKubosKLAJSiegforthMJ connectionistsInformatics MRJDunfieldNickersonMedinfo HCEComputKnausDawantUckunMandersEJMisianoDRMeyerhoffCollisonShabotDemuthRumelhart LehrWeigendHubermanTouretzky edsCMBEC PEI icutelemedicine BAScUniversit)Universiteit(bioinstrumentation CCRsFrizeEnnettDrbehaviouri##37=CIQYag1.a.i.(1)(a)(i)1)a)c:\Corel\Office7\Template\Web\wblank.wpt(;3$2#  0  .3  0       сLevel 1Level 2Level 3Level 4Level 5(f3 $ !NROXXNROONR   ( $ Figure  1  +,- &OLE 2.0 Box <=8C HKKKK2J+M 0_level1  X 2( ` hp x 223  ..  5+ ` hp x 5  (O;$0  2#  a  .3  0` (#(#2;*OLE*WPC..!.> x 4   ''  ' ' --  $A=A=A---'-- AA$y"A "AAAAAAiAiNAN4A4A--'-- --'-- ==$y| =|==.=.j=j===W=W==--'-- $y-A ==AA--'-- -A =AArial JwSwgw -=>=A">"A>A>A>A>A>A>Ai>iAN>NA4>4A>A=A=@A=A@|=|@=@=@.=.@j=j@=@=@=@W=W@=@=--"System  -'--- ---'--- B?---'--- ??--M DDA6A6F:F:LFLFXXeecc_ _ ;u;u88)-=A=@- $?<S4T7@?5T2[2Z-2Y-0] $\/o'p*]2(p&u&t-&s-#z $y"z%-- $--  $  -- $-- $-- $-- $-- $-- $-- $  -- $---- $-- $"##"-!- $ )*!*)-(- $')*(*21-0-4 $455454-3- $267376-65- $489598-7- $7;;7;:-:9- $8<=9=<-;- $:>?;?>-=- $<BC=CB-A- $@FGAGF-FE- $DKLELK-J- $IQRJRQ-P- $OUVPVYX-W-`_-^- $]pq^qp-o- $oppop{ $z{-- $-- $-- $-- $-- $-- $-- $-- $-- ${~||-|- ${wz~xxx-x-uu-u-t $sruvss-ss- $rqturr-r- $q o rtp p -p p - $ omp rnn-n- $mknpll-l- $khknii-i-g"g!-g -e$ $#d/_0b$g`0`/-`.- $-_5\6_.b]6Y@Y?-Y>-Y? $>XOQPT?[RPRO-RN- $MQPPQSNTQQM\ $[L^K_N\OL_L^-L]- $\KkElH]NFlFk-Fi- $hEjDkGiHEkCrCp-Co-Bt $sAu@vCtDAvAu-AuAt- $s@;>tC<<-<- $~;8;>988-8-5 $436744-4- $314622-22- $1/2400-0- $/-02..-.- $-*-0+++-++- $*'*-((-(- $'&)*''-'- $&%()&&-&'- $'%%'%&-&&- $%$'(%%-%- $$#&'$$-$- $#"&  - - $"" -- $-M-- OK---'--- ??D FB---'--- ??A6 C8?4---'--- ??F: H<D8---'--- ??LF NHJD---'--- ??X ZV---'--- ??e gc---'--- ??c ea---'--- ??_  a] ---'--- ??;u =w9s---'--- ??8 :6---'--- ??) +'---'--- ??---'--- B?---'--- ---'--- ---'--- @A---'--- =A--QQ-  $PPRRQQ  $P O Q RP P $OOQQPP! $!O%O%Q!QP%O- $-N1N1P-PO1O9 $9N=N=P9PO=OE $ENINIPEPOINQ $QMUMUOQONUN] $]MaMaO]ONaNi $hMlLmNiOMmMu $uLyLyNuNMyM $LLNNML $KKMMLL $KKMMLL $KJLMKK $JJLLKK $JJLLKJ $IIKKJJ $IIKKJJ $IIKKJI $HHJJII $HHJJII $HG IJH H $GGIIHH $G!G!IIH!G) $)F-F-H)HG-G5 $5F9F9H5HG9GA $@FDEEGAHFEFM $MEQEQGMGFQFY $YE]E]GYGF]Ee $eDiDiFeFEiEq $qDuDuFqFEuE} $}DDF}FED $CCEEDD $CCEEDC $BBDDCC $BBDDCC $BBDDCB $AACCBBB---'-- @A--'-- --'-- --'-- --'--   2 6-45 2 -50 2 -55 2 -60 2 -65 2 -70 2 -75 2 |-80 2 b-85 2 G-90 2 --95 2 '100--'-- --'--   2 F;45 2 Fv50 2 F55 2 F60 2 F(65 2 Fd70 2 F75 2 F80 2 F85 2 FQ90 2 F95 2 F100--'-- Arial KJwSwgw K -----'-- mX+ 42 [-Samples in POSTOP Database [%]  ---'-- ----'-- %E Arial mJwSwgw m -2 %Correct Classification Rate (CCR) [%]----'-- ---  $y---'--- #z---'--- #z-  -  2 |ANNg---'---  #z---'---  #z- 10-  $0EE0EQ $QWWQWV-C"2 |YConstant Predictor--'-- #z--'-- #z--  $ $ $ $-2 | Linear (ANN)--'-- #z--'-- --'-- - -'  '  '(UhCEKQW]cioAutoList31.1.1.1.1.1.1.1.Level 1Level 2Level 3Level 4Level 5Level 1Level 2Level 3Level 4Level 5(ehCEKQW]cioAutoList41.1.1.1.1.1.1.1.Level 1Level 2Level 3Level 4Level 5(5hCEKQW]cioAutoList11.1.1.1.1.1.1.1.(EhCEKQW]cioAutoList21.1.1.1.1.1.1.1.WPWin 6.0/OLE 1.0 Prefix Information MarkerExcel.Chart.8ࡱ> >  !"#$%&'()*+,-./0123456789:;<=Root Entry!F̘@Ole CompObjbWorkbookz0 @QuickFonts !FMicrosoft Excel ChartBiff8Excel.Chart.89q Font ColorOh+'0HP\x n/aCarleton UniversityMicrosoft Excel@ے@Mޑ@̮ @\pCarleton University Ba= -=hKL,9X@"1Arial1Arial1Arial1Arial1Arial1Arial1Arial1Arial1Arial1Arial"$"#,##0_);\("$"#,##0\)!"$"#,##0_);[Red]\("$"#,##0\)""$"#,##0.00_);\("$"#,##0.00\)'""$"#,##0.00_);[Red]\("$"#,##0.00\)7*2_("$"* #,##0_);_("$"* \(#,##0\);_("$"* "-"_);_(@_).))_(* #,##0_);_(* \(#,##0\);_(* "-"_);_(@_)?,:_("$"* #,##0.00_);_("$"* \(#,##0.00\);_("$"* "-"??_);_(@_)6+1_(* #,##0.00_);_(* \(#,##0.00\);_(* "-"??_);_(@_)                + ) , *  `3Chart3OChart4"Sheet1.Sheet2/Sheet3`i"DeathICU0ICU1ICU14ICU4ICU5VENT0VENT12VENT24 VENT24-336VENT336VENT36VENT4VENT8Constant PredictorANN I  @MHP LaserJet 2100 Series PSG odLetterPRIV'''' )"d??3` 4#` 4#` 4# .?13d  3Q ;  ANNQ ;Q ;Q3_  NM  <4E4 3Q ; (Constant PredictorQ ;Q ;Q3_  NM ] <4E4 3QQQQ3_ O NM ] MM<4JK4D $% M 3O&Q4$% M 3O&Q4FAG ~ 3Oe 3*F@Y@@@F@#M! M43*F@Y@@@F@#M! M! M4% BX M3O& Q @Samples in POSTOP Database [%]'4% <rMZ3O& Q N%Correct Classification Rate (CCR) [%]'4523 O NM43d" 3O% M,3OQ443_ M NM ] MM<4444eJ4qN@F@}гYO@J4qN@٬\iP@}гYO@1w-!P@٬\iP@8gDioP@1w-!P@sFR@8gDioP@H.'T@sFR@ȘT@H.'T@vT@ȘT@ +!W@ vT@ W@  +!W@ _vO&X@ W@ V-gX@ _vO&X@ V-gX@ V-gX@Y@V-gX@Y@eq= ףV@F@ $(~V@J4qN@ +!W@}гYO@d]KV@٬\iP@zV@1w-!P@V@8gDioP@XmU@sFR@9#J{U@H.'T@&SU@ȘT@ 7dbW@ vT@ W@  +!W@ _L   @MHP LaserJet 2100 Series PSG odLetterPRIV'''' )"d??3` 4#` 4#` 4# .?13d  3Q ;  ANNQ ; Q ; Q3_  NM  d4E4 3Q ; (Constant PredictorQ ;Q ; Q3_  NM  d4E4 3QQQQ3_ O NM  MM<4JK4D $% M 3O&Q4$% M 3O&Q4FAG ~ 3Oe 3*F@Y@@@F@#M! M43*F@Y@@@F@#M! M4% X hM3O& Q F!Samples in NONPOSTOP Database [%]'4% <rMZ3O& Q N%Correct Classification Rate (CCR) [%]'4523  NM43d" 3O% M,3OQ44444e{P#H@F@QIM@{P#H@S㥛 N@QIM@@GO@S㥛 N@5^I "P@@GO@\(P@5^I "P@ F%u:R@\(P@lxz,R@ F%u:R@y):6S@lxz,R@ |a2U0NU@ y):6S@ b=U@ |a2U0NU@ U@ b=U@ d]KX@ U@ d]KX@ d]KX@Y@d]KX@Y@e/$T@F@,CT@{P#H@}bqT@QIM@-QT@S㥛 N@DUS@@GO@/$T@5^I "P@:pΔS@\(P@-QT@ F%u:R@/$T@lxz,R@ XmU@ y):6S@ S!uV@ |a2U0NU@ b=U@ U@ d]KX@d]KX@Y@e>  @  dMbP?_*+%"??U} }  } m  Test a priori                           F@F@F@F@ ~ @J4qN@J4qN@  /$T@{P#H@ {P#H@  $(~V@}гYO@}гYO@  ,CT@QIM@ QIM@   +!W@٬\iP@٬\iP@   }bqT@S㥛 N@ S㥛 N@  d]KV@1w-!P@1w-!P@   -QT@@GO@ @GO@ zV@8gDioP@8gDioP@  DUS@5^I "P@ 5^I "P@ V@sFR@sFR@  /$T@A@A@  XmU@ H.'T@ H.'T@   :pΔS@  F%u:R@  F%u:R@   9#J{U@ ȘT@ ȘT@    -QT@ lxz,R@ lxz,R@   &SU@ vT@ vT@    /$T@ y):6S@ y):6S@  7dbW@  +!W@  +!W@   XmU@ |a2U0NU@ |a2U0NU@  W@ W@ W@   S!uV@ b=U@ b=U@ _L@7 @  dMbP?_*+%"??U>@7 @  dMbP?_*+%"??U>@7 BOlePartOlePres000*?SummaryInformation(DocumentSummaryInformation88L*> x 4   ''  ' ' --  $A=A=A---'-- AA$y"A "AAAAAAiAiNAN4A4A--'-- --'-- ==$y| =|==.=.j=j===W=W==--'-- $y-A ==AA--'-- -A =AArial JwSwgw -=>=A">"A>A>A>A>A>A>Ai>iAN>NA4>4A>A=A=@A=A@|=|@=@=@.=.@j=j@=@=@=@W=W@=@=--"System  -'--- ---'--- B?---'--- ??--M DDA6A6F:F:LFLFXXeecc_ _ ;u;u88)-=A=@- $?<S4T7@?5T2[2Z-2Y-0] $\/o'p*]2(p&u&t-&s-#z $y"z%-- $--  $  -- $-- $-- $-- $-- $-- $-- $  -- $---- $-- $"##"-!- $ )*!*)-(- $')*(*21-0-4 $455454-3- $267376-65- $489598-7- $7;;7;:-:9- $8<=9=<-;- $:>?;?>-=- $<BC=CB-A- $@FGAGF-FE- $DKLELK-J- $IQRJRQ-P- $OUVPVYX-W-`_-^- $]pq^qp-o- $oppop{ $z{-- $-- $-- $-- $-- $-- $-- $-- $-- ${~||-|- ${wz~xxx-x-uu-u-t $sruvss-ss- $rqturr-r- $q o rtp p -p p - $ omp rnn-n- $mknpll-l- $khknii-i-g"g!-g -e$ $#d/_0b$g`0`/-`.- $-_5\6_.b]6Y@Y?-Y>-Y? $>XOQPT?[RPRO-RN- $MQPPQSNTQQM\ $[L^K_N\OL_L^-L]- $\KkElH]NFlFk-Fi- $hEjDkGiHEkCrCp-Co-Bt $sAu@vCtDAvAu-AuAt- $s@;>tC<<-<- $~;8;>988-8-5 $436744-4- $314622-22- $1/2400-0- $/-02..-.- $-*-0+++-++- $*'*-((-(- $'&)*''-'- $&%()&&-&'- $'%%'%&-&&- $%$'(%%-%- $$#&'$$-$- $#"&  - - $"" -- $-M-- OK---'--- ??D FB---'--- ??A6 C8?4---'--- ??F: H<D8---'--- ??LF NHJD---'--- ??X ZV---'--- ??e gc---'--- ??c ea---'--- ??_  a] ---'--- ??;u =w9s---'--- ??8 :6---'--- ??) +'---'--- ??---'--- B?---'--- ---'--- ---'--- @A---'--- =A--QQ-  $PPRRQQ  $P O Q RP P $OOQQPP! $!O%O%Q!QP%O- $-N1N1P-PO1O9 $9N=N=P9PO=OE $ENINIPEPOINQ $QMUMUOQONUN] $]MaMaO]ONaNi $hMlLmNiOMmMu $uLyLyNuNMyM $LLNNML $KKMMLL $KKMMLL $KJLMKK $JJLLKK $JJLLKJ $IIKKJJ $IIKKJJ $IIKKJI $HHJJII $HHJJII $HG IJH H $GGIIHH $G!G!IIH!G) $)F-F-H)HG-G5 $5F9F9H5HG9GA $@FDEEGAHFEFM $MEQEQGMGFQFY $YE]E]GYGF]Ee $eDiDiFeFEiEq $qDuDuFqFEuE} $}DDF}FED $CCEEDD $CCEEDC $BBDDCC $BBDDCC $BBDDCB $AACCBBB---'-- @A--'-- --'-- --'-- --'--   2 6-45 2 -50 2 -55 2 -60 2 -65 2 -70 2 -75 2 |-80 2 b-85 2 G-90 2 --95 2 '100--'-- --'--   2 F;45 2 Fv50 2 F55 2 F60 2 F(65 2 Fd70 2 F75 2 F80 2 F85 2 FQ90 2 F95 2 F100--'-- Arial KJwSwgw K -----'-- mX+ 42 [-Samples in POSTOP Database [%]  ---'-- ----'-- %E Arial mJwSwgw m -2 %Correct Classification Rate (CCR) [%]----'-- ---  $y---'--- #z---'--- #z-  -  2 |ANNg---'---  #z---'---  #z- 10-  $0EE0EQ $QWWQWV-C"2 |YConstant Predictor--'-- #z--'-- #z--  $ $ $ $-2 | Linear (ANN)--'-- #z--'-- --'-- - -'  '  '՜.+,0 PXd lt| 4 Sheet1Sheet2Sheet3Chart3Chart4  WorksheetsCharts METAFILEPICTL>L* x 4   ''  ' ' --  $A=A=A---'-- AA$y"A "AAAAAAiAiNAN4A4A--'-- --'-- ==$y| =|==.=.j=j===W=W==--'-- $y-A ==AA--'-- -A =AArial JwSwgw -=>=A">"A>A>A>A>A>A>Ai>iAN>NA4>4A>A=A=@A=A@|=|@=@=@.=.@j=j@=@=@=@W=W@=@=--"System  -'--- ---'--- B?---'--- ??--M DDA6A6F:F:LFLFXXeecc_ _ ;u;u88)-=A=@- $?<S4T7@?5T2[2Z-2Y-0] $\/o'p*]2(p&u&t-&s-#z $y"z%-- $--  $  -- $-- $-- $-- $-- $-- $-- $  -- $---- $-- $"##"-!- $ )*!*)-(- $')*(*21-0-4 $455454-3- $267376-65- $489598-7- $7;;7;:-:9- $8<=9=<-;- $:>?;?>-=- $<BC=CB-A- $@FGAGF-FE- $DKLELK-J- $IQRJRQ-P- $OUVPVYX-W-`_-^- $]pq^qp-o- $oppop{ $z{-- $-- $-- $-- $-- $-- $-- $-- $-- ${~||-|- ${wz~xxx-x-uu-u-t $sruvss-ss- $rqturr-r- $q o rtp p -p p - $ omp rnn-n- $mknpll-l- $khknii-i-g"g!-g -e$ $#d/_0b$g`0`/-`.- $-_5\6_.b]6Y@Y?-Y>-Y? $>XOQPT?[RPRO-RN- $MQPPQSNTQQM\ $[L^K_N\OL_L^-L]- $\KkElH]NFlFk-Fi- $hEjDkGiHEkCrCp-Co-Bt $sAu@vCtDAvAu-AuAt- $s@;>tC<<-<- $~;8;>988-8-5 $436744-4- $314622-22- $1/2400-0- $/-02..-.- $-*-0+++-++- $*'*-((-(- $'&)*''-'- $&%()&&-&'- $'%%'%&-&&- $%$'(%%-%- $$#&'$$-$- $#"&  - - $"" -- $-M-- OK---'--- ??D FB---'--- ??A6 C8?4---'--- ??F: H<D8---'--- ??LF NHJD---'--- ??X ZV---'--- ??e gc---'--- ??c ea---'--- ??_  a] ---'--- ??;u =w9s---'--- ??8 :6---'--- ??) +'---'--- ??---'--- B?---'--- ---'--- ---'--- @A---'--- =A--QQ-  $PPRRQQ  $P O Q RP P $OOQQPP! $!O%O%Q!QP%O- $-N1N1P-PO1O9 $9N=N=P9PO=OE $ENINIPEPOINQ $QMUMUOQONUN] $]MaMaO]ONaNi $hMlLmNiOMmMu $uLyLyNuNMyM $LLNNML $KKMMLL $KKMMLL $KJLMKK $JJLLKK $JJLLKJ $IIKKJJ $IIKKJJ $IIKKJI $HHJJII $HHJJII $HG IJH H $GGIIHH $G!G!IIH!G) $)F-F-H)HG-G5 $5F9F9H5HG9GA $@FDEEGAHFEFM $MEQEQGMGFQFY $YE]E]GYGF]Ee $eDiDiFeFEiEq $qDuDuFqFEuE} $}DDF}FED $CCEEDD $CCEEDC $BBDDCC $BBDDCC $BBDDCB $AACCBBB---'-- @A--'-- --'-- --'-- --'--   2 6-45 2 -50 2 -55 2 -60 2 -65 2 -70 2 -75 2 |-80 2 b-85 2 G-90 2 --95 2 '100--'-- --'--   2 F;45 2 Fv50 2 F55 2 F60 2 F(65 2 Fd70 2 F75 2 F80 2 F85 2 FQ90 2 F95 2 F100--'-- Arial KJwSwgw K -----'-- mX+ 42 [-Samples in POSTOP Database [%]  ---'-- ----'-- %E Arial mJwSwgw m -2 %Correct Classification Rate (CCR) [%]----'-- ---  $y---'--- #z---'--- #z-  -  2 |ANNg---'---  #z---'---  #z- 10-  $0EE0EQ $QWWQWV-C"2 |YConstant Predictor--'-- #z--'-- #z--  $ $ $ $-2 | Linear (ANN)--'-- #z--'-- --'-- - -'  '  'GH*OLE*Level 1Level 2Level 3Level 4Level 5TABLE B)Hairline dTABLE ATABLE B $i)2G+M 0_level2   /%` ` hp x /23  ..  5+ ` hp x 5  WPC..!.02  9   ''  ' ' --  $A=A=A---'-- AA)y"A "AAAAAAiAiNAN4A4A--'-- --'-- ==)y| =|==.=.j=j===W=W==--'-- )y-A ==AA--'-- -A =AArial JwSwgw -=>=A">"A>A>A>A>A>A>Ai>iAN>NA4>4A>A=A=@A=A@|=|@=@=@.=.@j=j@=@=@=@W=W@=@=--"System  -'--- ---'--- C>---'--- @>--wh zz||))w@w@||wweeM*-=A=@- $?<S4T7@?5T0_ $^/h+i._2,i,h-,h,g- $f+y"z%g.#z $! $   $ $-- $-- $ $-- $#$$*)-)(- $';<(<A@-@?- $>QR?R] $\pq]q| ${|-- $ $-- $-- $ $}~y $x p s{q l $kf ing g-gg- $f*a+dib+b*-b*b)- $(a<Y=\)dZ=Z<-Z<Z;- $:YMQNT;\RNMY $XLlCmFYODm@x $w?69xB72 $1-04..-./- $/--/-.-..- $-%(0&! $ #-wh-- zkte---'--- @> ---'--- @>z }w---'--- @>| y---'--- @>) ,&---'--- @>w@ zCt=---'--- @> ---'--- @>| y---'--- @>w zt---'--- @>e h"b---'--- @>M* P-J'---'--- @>---'--- C>---'--- ---'--- ---'--- @A---'--- =A--hg-  $gkkgks $swwsw $ $ $ $ $ $ $ $ $ $ $~~ $~}~~ $}|~}} $||~~}|' $'{+{+}'}|+{3 $3z7z7|3|{7z? $?yCyC{?{zCyK $KxOxOzKzyOxW $Ww[w[yWyx[xc $bwfvgxcywgwo $nvruswoxvsv{ ${uuw{wvu $ttvvut $ssuuts $rrttsr $qqssrq $pprrqq $poqrpp $onpqoo $nnppon $mmoonm $llnnml $kkmmlk  $ jjl lkj $iikkjj" $!i%h&j"ki&i. $-h1g2i.jh2h: $:g>g>i:ih>gF $FfJfJhFhgJfR $ReVeVgRgfVe^ $^dbdbf^febdj $jcncnejedncv $vbzbzdvdczc $bacdbb $a`bcaa $``bba` $__aa`_ $^^``_^ $]]__^] $\\^^]]]---'-- @A--'-- --'-- --'-- --'--   2 6-45 2 -50 2 -55 2 -60 2 -65 2 -70 2 -75 2 |-80 2 b-85 2 G-90 2 --95 2 '100--'-- --'--   2 F;45 2 Fv50 2 F55 2 F60 2 F(65 2 Fd70 2 F75 2 F80 2 F85 2 FQ90 2 F95 2 F100--'-- Arial JwSwgw -----'-- mX  92 ["!Samples in NONPOSTOP Database [%]  ---'-- ----'-- %E Arial 5JwSwgw 5 -2 %Correct Classification Rate (CCR) [%]----'-- ---  )y---'--- (z---'--- (z-  -  2 |ANNP---'---  (z---'---  (z- 0/-  $/DD/DP $PYYPYX-D"2 |[Constant Predictor--'-- (z--'-- (z--  $ $ $ $-2 | Linear (ANN)--'-- (z--'-- --'-- - -'  '  'WPWin 6.0/OLE 1.0 Prefix Information MarkerExcel.Chart.8tࡱ> 8  !"#$%&'()*+,-./01234567Root Entry!FZǒ@Ole CompObjbWorkbookz0 @QuickFonts !FMicrosoft Excel ChartBiff8Excel.Chart.89q Font ColorOh+'0HP\x n/aCarleton UniversityMicrosoft Excel@ے@Mޑ@!<ǒ @\pCarleton University Ba= .=hKL,9X@"1Arial1Arial1Arial1Arial1Arial1Arial1Arial1Arial1Arial1Arial"$"#,##0_);\("$"#,##0\)!"$"#,##0_);[Red]\("$"#,##0\)""$"#,##0.00_);\("$"#,##0.00\)'""$"#,##0.00_);[Red]\("$"#,##0.00\)7*2_("$"* #,##0_);_("$"* \(#,##0\);_("$"* "-"_);_(@_).))_(* #,##0_);_(* \(#,##0\);_(* "-"_);_(@_)?,:_("$"* #,##0.00_);_("$"* \(#,##0.00\);_("$"* "-"??_);_(@_)6+1_(* #,##0.00_);_(* \(#,##0.00\);_(* "-"??_);_(@_)                + ) , *  `3Chart3OChart4"Sheet1.Sheet2/Sheet3`i"DeathICU0ICU1ICU14ICU4ICU5VENT0VENT12VENT24 VENT24-336VENT336VENT36VENT4VENT8Constant PredictorANN I  @MHP LaserJet 2100 Series PSG odLetterPRIV'''' )"d??3` 4#` 4#` 4# .?13d  3Q ;  ANNQ ;Q ;Q3_  NM  <4E4 3Q ; (Constant PredictorQ ;Q ;Q3_  NM ] <4E4 3QQQQ3_ O NM ] MM<4JK4D $% M 3O&Q4$% M 3O&Q4FAG ~ 3Oe 3*F@Y@@@F@#M! M43*F@Y@@@F@#M! M! M4% BX M3O& Q @Samples in POSTOP Database [%]'4% <rMZ3O& Q N%Correct Classification Rate (CCR) [%]'4523 O NM43d" 3O% M,3OQ443_ M NM ] MM<4444eJ4qN@F@}гYO@J4qN@٬\iP@}гYO@1w-!P@٬\iP@8gDioP@1w-!P@sFR@8gDioP@H.'T@sFR@ȘT@H.'T@vT@ȘT@ +!W@ vT@ W@  +!W@ _vO&X@ W@ V-gX@ _vO&X@ V-gX@ V-gX@Y@V-gX@Y@eq= ףV@F@ $(~V@J4qN@ +!W@}гYO@d]KV@٬\iP@zV@1w-!P@V@8gDioP@XmU@sFR@9#J{U@H.'T@&SU@ȘT@ 7dbW@ vT@ W@  +!W@ _L   @MHP LaserJet 2100 Series PSG odLetterPRIV'''' )"d??3` 4#` 4#` 4# .?13d  3Q ;  ANNQ ; Q ; Q3_  NM  d4E4 3Q ; (Constant PredictorQ ;Q ; Q3_  NM  d4E4 3QQQQ3_ O NM  MM<4JK4D $% M 3O&Q4$% M 3O&Q4FAG ~ 3Oe 3*F@Y@@@F@#M! M43*F@Y@@@F@#M! M4% X hM3O& Q F!Samples in NONPOSTOP Database [%]'4% <rMZ3O& Q N%Correct Classification Rate (CCR) [%]'4523  NM43d" 3O% M,3OQ44444e{P#H@F@QIM@{P#H@S㥛 N@QIM@@GO@S㥛 N@5^I "P@@GO@\(P@5^I "P@ F%u:R@\(P@lxz,R@ F%u:R@y):6S@lxz,R@ |a2U0NU@ y):6S@ b=U@ |a2U0NU@ U@ b=U@ d]KX@ U@ d]KX@ d]KX@Y@d]KX@Y@e/$T@F@,CT@{P#H@}bqT@QIM@-QT@S㥛 N@DUS@@GO@/$T@5^I "P@:pΔS@\(P@-QT@ F%u:R@/$T@lxz,R@ XmU@ y):6S@ S!uV@ |a2U0NU@ b=U@ U@ d]KX@d]KX@Y@e>  @  dMbP?_*+%"??U} }  } m  Test a priori                           F@F@F@F@ ~ @J4qN@J4qN@  /$T@{P#H@ {P#H@  $(~V@}гYO@}гYO@  ,CT@QIM@ QIM@   +!W@٬\iP@٬\iP@   }bqT@S㥛 N@ S㥛 N@  d]KV@1w-!P@1w-!P@   -QT@@GO@ @GO@ zV@8gDioP@8gDioP@  DUS@5^I "P@ 5^I "P@ V@sFR@sFR@  /$T@A@A@  XmU@ H.'T@ H.'T@   :pΔS@  F%u:R@  F%u:R@   9#J{U@ ȘT@ ȘT@    -QT@ lxz,R@ lxz,R@   &SU@ vT@ vT@    /$T@ y):6S@ y):6S@  7dbW@  +!W@  +!W@   XmU@ |a2U0NU@ |a2U0NU@  W@ W@ W@   S!uV@ b=U@ b=U@ _L@7 @  dMbP?_*+%"??U>@7 @  dMbP?_*+%"??U>@7 BOlePartOlePres000h2SummaryInformation(DocumentSummaryInformation88L*.2  9   ''  ' ' --  $A=A=A---'-- AA)y"A "AAAAAAiAiNAN4A4A--'-- --'-- ==)y| =|==.=.j=j===W=W==--'-- )y-A ==AA--'-- -A =AArial JwSwgw -=>=A">"A>A>A>A>A>A>Ai>iAN>NA4>4A>A=A=@A=A@|=|@=@=@.=.@j=j@=@=@=@W=W@=@=--"System  -'--- ---'--- C>---'--- @>--wh zz||))w@w@||wweeM*-=A=@- $?<S4T7@?5T0_ $^/h+i._2,i,h-,h,g- $f+y"z%g.#z $! $   $ $-- $-- $ $-- $#$$*)-)(- $';<(<A@-@?- $>QR?R] $\pq]q| ${|-- $ $-- $-- $ $}~y $x p s{q l $kf ing g-gg- $f*a+dib+b*-b*b)- $(a<Y=\)dZ=Z<-Z<Z;- $:YMQNT;\RNMY $XLlCmFYODm@x $w?69xB72 $1-04..-./- $/--/-.-..- $-%(0&! $ #-wh-- zkte---'--- @> ---'--- @>z }w---'--- @>| y---'--- @>) ,&---'--- @>w@ zCt=---'--- @> ---'--- @>| y---'--- @>w zt---'--- @>e h"b---'--- @>M* P-J'---'--- @>---'--- C>---'--- ---'--- ---'--- @A---'--- =A--hg-  $gkkgks $swwsw $ $ $ $ $ $ $ $ $ $ $~~ $~}~~ $}|~}} $||~~}|' $'{+{+}'}|+{3 $3z7z7|3|{7z? $?yCyC{?{zCyK $KxOxOzKzyOxW $Ww[w[yWyx[xc $bwfvgxcywgwo $nvruswoxvsv{ ${uuw{wvu $ttvvut $ssuuts $rrttsr $qqssrq $pprrqq $poqrpp $onpqoo $nnppon $mmoonm $llnnml $kkmmlk  $ jjl lkj $iikkjj" $!i%h&j"ki&i. $-h1g2i.jh2h: $:g>g>i:ih>gF $FfJfJhFhgJfR $ReVeVgRgfVe^ $^dbdbf^febdj $jcncnejedncv $vbzbzdvdczc $bacdbb $a`bcaa $``bba` $__aa`_ $^^``_^ $]]__^] $\\^^]]]---'-- @A--'-- --'-- --'-- --'--   2 6-45 2 -50 2 -55 2 -60 2 -65 2 -70 2 -75 2 |-80 2 b-85 2 G-90 2 --95 2 '100--'-- --'--   2 F;45 2 Fv50 2 F55 2 F60 2 F(65 2 Fd70 2 F75 2 F80 2 F85 2 FQ90 2 F95 2 F100--'-- Arial JwSwgw -----'-- mX  92 ["!Samples in NONPOSTOP Database [%]  ---'-- ----'-- %E Arial 5JwSwgw 5 -2 %Correct Classification Rate (CCR) [%]----'-- ---  )y---'--- (z---'--- (z-  -  2 |ANNP---'---  (z---'---  (z- 0/-  $/DD/DP $PYYPYX-D"2 |[Constant Predictor--'-- (z--'-- (z--  $ $ $ $-2 | Linear (ANN)--'-- (z--'-- --'-- - -'  '  '՜.+,0 PXd lt| 4 Sheet1Sheet2Sheet3Chart3Chart4  WorksheetsCharts METAFILEPICTL62L*  9   ''  ' ' --  $A=A=A---'-- AA)y"A "AAAAAAiAiNAN4A4A--'-- --'-- ==)y| =|==.=.j=j===W=W==--'-- )y-A ==AA--'-- -A =AArial JwSwgw -=>=A">"A>A>A>A>A>A>Ai>iAN>NA4>4A>A=A=@A=A@|=|@=@=@.=.@j=j@=@=@=@W=W@=@=--"System  -'--- ---'--- C>---'--- @>--wh zz||))w@w@||wweeM*-=A=@- $?<S4T7@?5T0_ $^/h+i._2,i,h-,h,g- $f+y"z%g.#z $! $   $ $-- $-- $ $-- $#$$*)-)(- $';<(<A@-@?- $>QR?R] $\pq]q| ${|-- $ $-- $-- $ $}~y $x p s{q l $kf ing g-gg- $f*a+dib+b*-b*b)- $(a<Y=\)dZ=Z<-Z<Z;- $:YMQNT;\RNMY $XLlCmFYODm@x $w?69xB72 $1-04..-./- $/--/-.-..- $-%(0&! $ #-wh-- zkte---'--- @> ---'--- @>z }w---'--- @>| y---'--- @>) ,&---'--- @>w@ zCt=---'--- @> ---'--- @>| y---'--- @>w zt---'--- @>e h"b---'--- @>M* P-J'---'--- @>---'--- C>---'--- ---'--- ---'--- @A---'--- =A--hg-  $gkkgks $swwsw $ $ $ $ $ $ $ $ $ $ $~~ $~}~~ $}|~}} $||~~}|' $'{+{+}'}|+{3 $3z7z7|3|{7z? $?yCyC{?{zCyK $KxOxOzKzyOxW $Ww[w[yWyx[xc $bwfvgxcywgwo $nvruswoxvsv{ ${uuw{wvu $ttvvut $ssuuts $rrttsr $qqssrq $pprrqq $poqrpp $onpqoo $nnppon $mmoonm $llnnml $kkmmlk  $ jjl lkj $iikkjj" $!i%h&j"ki&i. $-h1g2i.jh2h: $:g>g>i:ih>gF $FfJfJhFhgJfR $ReVeVgRgfVe^ $^dbdbf^febdj $jcncnejedncv $vbzbzdvdczc $bacdbb $a`bcaa $``bba` $__aa`_ $^^``_^ $]]__^] $\\^^]]]---'-- @A--'-- --'-- --'-- --'--   2 6-45 2 -50 2 -55 2 -60 2 -65 2 -70 2 -75 2 |-80 2 b-85 2 G-90 2 --95 2 '100--'-- --'--   2 F;45 2 Fv50 2 F55 2 F60 2 F(65 2 Fd70 2 F75 2 F80 2 F85 2 FQ90 2 F95 2 F100--'-- Arial JwSwgw -----'-- mX  92 ["!Samples in NONPOSTOP Database [%]  ---'-- ----'-- %E Arial 5JwSwgw 5 -2 %Correct Classification Rate (CCR) [%]----'-- ---  )y---'--- (z---'--- (z-  -  2 |ANNP---'---  (z---'---  (z- 0/-  $/DD/DP $PYYPYX-D"2 |[Constant Predictor--'-- (z--'-- (z--  $ $ $ $-2 | Linear (ANN)--'-- (z--'-- --'-- - -'  '  '2D+M 0_level3   ," hp x ,23  ..  5+ ` hp x 5  2A+M 0_level4  ` ) hp x )23  ..  5+ ` hp x 5  2>+M 0_level5   &hhp x &23  ..  5+ ` hp x 5  2;+M 0_level6   #p x #23  ..  5+ ` hp x 5  28+M 0_level7  h  p x 23  ..  5+ ` hp x 5  25+M 0_level8   pp x 23  ..  5+ ` hp x 5  22+M 0_level9    x 23  ..  5+ ` hp x 5  2JM 0_levsl1  X 2( ` hp x 223  Ԁ  5+ ` hp x 5  2GM 0_levsl2   /%` ` hp x /23  Ԁ  5+ ` hp x 5    d2DM 0_levsl3   ," hp x ,23  Ԁ  5+ ` hp x 5  2AM 0_levsl4  ` ) hp x )23  Ԁ  5+ ` hp x 5  2>M 0_levsl5   &hhp x &23  Ԁ  5+ ` hp x 5  2;M 0_levsl6   #p x #23  Ԁ  5+ ` hp x 5  28M 0_levsl7  h  p x 23  Ԁ  5+ ` hp x 5  25M 0_levsl8   pp x 23  Ԁ  5+ ` hp x 5  22M 0_levsl9    x 23  Ԁ  5+ ` hp x 5  2JM 0_levnl1  X 2( ` hp x 223   5+ ` hp x 5  2GM 0_levnl2   /%` ` hp x /23   5+ ` hp x 5  2DM 0_levnl3   ," hp x ,23   5+ ` hp x 5  2AM 0_levnl4  ` ) hp x )23   5+ ` hp x 5  2>M 0_levnl5   &hhp x &23   5+ ` hp x 5  2;M 0_levnl6   #p x #23   5+ ` hp x 5  28M 0_levnl7  h  p x 23   5+ ` hp x 5  25M 0_levnl8   pp x 23   5+ ` hp x 5  22M 0_levnl9    x 23   5+ ` hp x 5  0.Normal<:Definition T<AA:Definition L , 5+ ` hp x 5  5+ ` hp x 586Definition(''&H1      XXX(&H2  XXX(&H3  XXX(&H4 XXX XXX(&H5  XXX(&H6  XXX20Address8MM6Blockquote , , 5+ ` hp x 5   5+ ` hp x 5,*CITE,dl*CODEK<6X9`(Courier NewKXXXS\  `&Times New RomanS42Emphasis64Hyperlink    <:FollowedHype    4go2Keyboard K<6X9`(Courier NewK XXXS\  `&Times New RomanS<:Preformatted/%  ,Kk %#/K<6X9`(Courier NewKXXXS\  `&Times New RomanS/%  ,Kk %#/<:zBottom of 7x(X7  ?%2A`Arial?7 (X7  XXXS\  `&Times New RomanS&0 d dw)1dxd<:zTop of For7z(X7  ?%2A`Arial?7 (X7  XXXS\  `&Times New RomanSw)2dxd0KS.SampleK<6X9`(Courier NewKS\  `&Times New RomanS0.Strong 8dl6TypewriterK<6X9`(Courier NewKXXXS\  `&Times New RomanS42Variable: 8HTML MarkupB      2 0CommentB  (O$<6X9`(Courier New\  `&Times New Roman%2A`Arial+,- &OLE 2.0 Box <=8C HKKKK(b$0  0` (#(#2#   .3  0 ` (#` (#(xir$0  0` (#(#0 ` (#` (#2#(  0  )3  0 (# (#($0  0` (#(#0 ` (#` (#0 (# (#2#(  a  )3  0h(#(#(F$0  0` (#(#0 ` (#` (#0 (# (#0h(#(#2#(   )3  0h(#h(#($0  0` (#(#0 ` (#` (#0 (# (#0h(#(#0h(#h(#2#  0  )3  0(#(#({$0  0` (#(#0 ` (#` (#0 (# (#0h(#(#0h(#h(#0(#(#2#  a  )3  0p(#(#)0hBAG_cgkosw{Bullet ListBullet List@@@@7Hairlinedxd(@ Z(Times New Roman !NROXXNROONR   _9   <(d  l t$|,!X<X|XONRXZXXX|  XXZ8ZXXdd8uClinicalDecisionSupportSystemsforIntensiveCareUnits:  UsingArtificialNeuralNetworks#X Xu#   MoniqueFrize,PhD,PEng1,2,ColleenM.Ennett,MASc1,MaryhelenStevenson,PhD3,   HeatherC.E.Trigg3  1DepartmentofSystemsandComputerEngineering,CarletonUniversity,Ottawa,ON,Canada   2SchoolofInformationTechnologyandEngineering,UniversityofOttawa,Ottawa,ON,Canada m  3UniversityofNewBrunswick,Fredericton,NB,Canada M    Mailingaddress:Dr.MoniqueFrizeSystemsandComputerEngineering,4456MacKenzieBuilding,CarletonUniversity1125ColonelByDrive,Ottawa,ON,K1S5B6Monique_Frize@carleton.ca ABSTRACT   $XdX$  Thepaperprovidesanoverviewofapplicationsofartificialneuralnetworks(ANNs)tovariousmedicalproblems,withaparticularfocusontheintensivecareunitenvironment(ICU).SeveraltechnicalapproachesweretestedtoseewhethertheyimprovetheANNperformanceinestimatingmedicaloutcomesandresourceutilizationinadultICUs.Theseexperimentsinclude:(1)Theuseoftheweighteliminationcostfunction;(2)theuseof highand lownodesfortheinputvariables;(3)verifyingtheeffectofthetotalnumberofinputvariablesontheresults;(4)andtestingtheimpactofthevalueoftheconstantpredictorontheperformanceoftheANNs.Thedevelopmentspresentedintendtohelpmedicalandnursingpersonneltoassesspatientstatus,assistinmakingadiagnosis,andfacilitatetheselectionofacourseoftherapy. Keywords Decisionsupport;Artificialneuralnetworks;Outcomesestimation;Intensivecare   medicine;Systemperformance. **1.INTRODUCTION  w#"   Becauseoftheirnonlinearmodelingcapabilities,artificialneuralnetworks(ANNs)havebeenwidelyappliedtononlinearstatisticalmodelingproblemsandareanaturalchoiceformodelinglargeandcomplexdatabasesofmedicalinformation.ThegoaloftraininganANNistoadjusttheweightsofthenetworksoastooptimizetheperformanceofthenetworkinestimatingoutcomesforaparticularinputspace.Forexample,theinputspacecanbeasetofmedicalparameterscollectedatthetimeofpatientadmissiontoasurgicalormedicalintensivecareunit(ICU),orthedatacanbecollectedatdifferentpointsintime.Thebackpropagationtrainingalgorithm,apopularapproachusedwithmedicaldatabases,adjuststheweightsofanANNtominimizeacostfunction.Acommonlychosencostfunctionistheaveragesumofsquarederrorsbetweenthedesiredoutputsandactualoutputs. {-(- ** *  *    ItiswellknownthatanetworkwhichhasbeentrainedasaclassifierwillcloselyapproximateaBayesclassifierwhenthenetworkarchitectureissufficientlycomplex,thetrainingsetissufficientlyrich,andthetrainingalgorithmsucceedsinminimizingthemeansquarederror[1,2].BecauseaBayesclassifierisoptimalinthesensethatitminimizestheprobabilityofclassificationerror,successfullytraininganetworkusingthebackpropagationalgorithmcanresultinapowerfultool[3].Inpractice,however,theANNerrorrateofabackpropagationtrainednetworkishigherthantheBayeserrorrate.Thereasonsforthisincludethefactthatthereareoftenlimitationsplacedonthetrainingsetsizeand/ornetworksize,andthefactthatthenetworktrainingalgorithmmaysettleintoalocalratherthanaglobalminimum[13].  TheliteraturereportsseveralapplicationsofANNstotherecognitionofaparticularpathology.Forexample,BaxtusedANNsasanaidtodiagnoseacutecoronaryocclusion[4]andlaterformyocardialinfarction[5];Kuntz[6]designedacascadecorrelationANNtoestimatemortalityandlengthofstayforpatientswithclosedheadinjuries;Buchmanetal.[7]estimatedchronicityinasurgicalintensivecareunit;Lau[8]discussedtheprinciplesbehindthedesignandvalidationofadecisionsupportsystemforcardiovascularICUs;TuandGuerriere[9]reportedestimationsoflengthofstayandmortalityinICUs;andBuskardetal.andFrizeetal.[1013]addedstudiesofestimateddurationofartificialventilationtotheestimationofmortalityandlengthofstayagaininadultICUs. 2.BACKGROUNDANDMEDICALCONTEXT  4   Fornearlytwodecades,scoringsystemshavebeenusedtopredictmedicaloutcomes[14,15].However,thesehavebeenmoreusefulinestimatingoutcomesforagroupofpatientsratherthanforasinglepatient.Newapproachesshouldattempttomakeestimatesonapatientbypatientbasis,andthishasbeenamainfocusoftheworkreportedhere.Inordertoremainascloseaspossibletothemannerinwhichthemedicalmodelworks,anANNmodelwasselectedwhich,whenproperlytrained,providesanestimateofselectedclinicaloutcomes,simulatingacliniciansconsiderationofpotentialpatientoutcomes.Forexample,thephysicianmaythink: Andforthisparticularpatient,thisiswhatIthinkwillhappen.  Anotherconsiderationistheparticularlyfastpaceofillnessincriticallyillpatients.Thisrealityhasspawnedthedevelopmentofmanytypesoftestingandmonitoringtechnologies,rapidlyevolvingintocomplexsystems[16,17].Whilemedicaldevicesusedincriticalcareunitstypicallygeneratehugevolumesofinformationinashortamountoftime,muchofitcanbelostbecausephysiciansandnursesdonothavetimetoreadthroughvoluminousamountsofoutputinacriticalcaresetting.Inaddition,fewdevicesarelinkedtohospitalinformationsystemsandeachgeneratesitsownseparateoutput.Thispointstotheneedforinvestigatinganintegrated,ratherthancompartmentalised,approachtocriticalcare(andothermedicalenvironments).This,andtheneedtogeneralizeandtestthetoolsforeffectivenessandrelevanceinavarietyofmedicalcontexts,havebeenthebasisonwhichtheworkreportedherewasengaged.Themovetousingtemporal(timevarying)datashouldeventuallyresultina dynamicsystemthatestimatespatientstatusinrealtime.  *;&*  3.METHODOLOGY     Theimportanceofacquiringagoodqualitydatabase,errorfreeandwithastandardizedapproachtodatacollectioniswellrecognized[18].Anotherimportantstepbeforeanalyzingthedataistoensurethatoutliersandobviouserrorsinthedataareremovedbeforeproceedingtotheanalysis.Yale[19]statesthat80percentofthetimespenttogetanANNsystemupandrunningistypicallyusedfor massagingthesetoftrainingdata.TheadultICUdatabaseusedinthestudiesreportedherewasdevelopedandassessedwiththeintentiontoremainascloseaspossibletothesestandards. TheAdultICUDatabase       Wehadaccesstoamedicaldatabaseofover3000adultICUpatients,containing98fieldsofclinicalandadministrativeinformationonpatientsadmittedtotheICUattheDoctorEverettChalmersHospital(DECH)inFredericton,NB,Canada.Datacollectionwasprimarilyprospective,withsomeretrospectivechartreview.Uptosevenmedicaldiagnosesandmultipleproceduralinformationcouldbeentered,withauxiliaryspaceforfreeformcomments.Significanteventsandcomplicationswerealsonotedforeachpatient.AsubsetofthisdatabasewiththerawAPACHEIIvariablesextractedintoanewdatabasewasusedforallexperimentsreportedherewith;thesizeofthisdatabasebeinglimitedbythecostofthemedicalassistanttocompilethedata.Thenewdatabasecontained51inputvariableswiththemostcompleteprofilesandexcludedpatientsundertheageof12,whichresultedinadatabasewith1491cases[20].ThevariablelistofthisdatabaseisshowninTable1.Thedatabasewasalsoseparatedintopostoperative(surgical,883cases)andnonpostoperative(medical,608cases)patientsforexperimentalpurposesbecausethesetwopatientsetscanhavedrasticallydifferentcharacteristics. DataPreProcessing  \   Thenonbinaryvaluedinputsinthetwodatasetsdescribedabovewerestandardized.Thesevariableswerescaledsothatzeroinputvaluesrepresented normalvaluesoftheinputvariables,negativeinputsrepresented lowerthannormalvaluesandpositiveinputsrepresented higherthannormalvalues.The normalvalueswereselectedinconsultationwiththephysician(Dr.F.G.Solven,intensivistatDECH).Toobtaininputdataofnearlyuniformmagnitude,the normalvalueofeachnonbinaryvaluedvariablewassubtractedfromeachinputvalueandtheresultingdifferencesweredividedbythreestandarddeviationspertainingtoeachinputvariablevalueovertheentiredataset.Ԁ!&Azerowasassignedasthe normalvalueforbinaryvariables[20].  Oneremainingandmostdifficultproblem,inourexperience,isthequestionofhowtotreatvariableswithmissinginformation.ANNsdonotfunctionwellwithvariablesthatcontainalotofmissinginformation.Acommonapproachistoeliminatethecaseswithmissingdataandusetheremainingdatasettotrainandtotestthenetwork.Onecanalsoreplacethemissingvaluesbythe meanor medianvalueforthevariable.Intheworkreportedhere,wedecidedtoreplacethemissingvaluesbythe normalvalueforthevariable,whichisathird,wellacceptedapproach.Thethinkingbehindthisdecisionisthat,inasetofICUpatients,themeanmaybebiasedtowardssomeparticularpathologyoroutcome,whereas normalvaluesareexpectedtohavetheleastimpactontheoutputs.Anexceptiontothistechniquewasmadewhenalmostallcriticalinformationwas -(- missing.Infact,therewereonlytwelvecasesmissingasignificantamountofdata,sotheserecordswereeliminatedbeforeproceedingtotheexperimentalstage.Anotherreasontoselectthisapproachwasbasedonthephysiciansknowledgethat,frequently,missingdataoccursinmedicaldatabaseswhenatestisnotdoneforthatpatientortheinformationwasdeemedtohavelittleimportancewithrespecttotheoutcomesofinterestforthatparticularpatient.Moreover,itwasfeltthataddinga normalvaluewouldnotdisrupttheresultsfor abnormalvaluessincethelatterareexpectedtohavethelargestimpactonpoormedicaloutcomes.ThisprocesssuccessfullyallowedtheANNtouseinformationcontainedinincompleterecords,thusprovidingalargernumberofrecordstotrainandtestthenetworks[20]. ArtificialNeuralNetwork(ANN)DesignstoEstimateOutcomes        Architecture First,thedatabasewasdividedintoitstwomainpartssincethepatienttypesin   eachwerequitedistinct:postoperativepatientsadmittedtotheICU(calledPOSTOP,883patients)andthosewhowerenotadmittedafterasurgery(calledNONPOSTOP,608patients).Foreachgroup(POSTOPandNONPOSTOP),thedatawasdividedintoatrainingsetandatestsetusingtwothirdsofthedatasettotraintheANNandtheremainingonethirdtotestitsperformance.UsingMatlabsNeuralNetworkToolbox[21],afeedforwardANNwastrainedusingthebackpropagationalgorithmtoestimatethefollowingmedicaloutcomes:mortality,durationofartificialventilation,andlengthofstayintheadultICU[10].  Severalarchitecturesweredesigned,butthebestresultswereobtainedwithasimplenetworkwithonehiddenlayer(i.e.,athreelayernetwork:inputlayer,hiddenlayer,outputlayer)[20].Thistypeofnetworkwaschosenbecauseofitsrelativeeaseofimplementationandsuccessincompletingvariousclassificationtasks,asdemonstratedbyHaykin[22]andWidrowetal.[23].Althoughpreliminaryresultswerepromising,theANNsexhibitedthebehaviourcharacteristicofnetworkmemorization(or overfitting)[10].Ratherthanjustreducingtheinputnetworksize(andthereforenetworkcomplexity)byeliminatingwhatmightpotentiallybeusefulinputinformation,wechosetouseatechniquecalled weighteliminationtoovercometheoverfittingproblem[24].  ThefollowingANNnetworkparameterswereadjustedtooptimizethenetworksperformance:learningrateanditsadaptiveparameters,momentum,weighteliminationscalefactoranditsadaptiveparameters,weightdecayconstant,anderrorratio.ItwasalsopossibletoadjustthenumberofhiddenlayersandhiddennodesforeachANNexperiment. FourTechniquestoImproveANNPerformance $# (1)ImpactoftheWeightEliminationCostFunction  c&!%   Theweighteliminationcostfunctionincludesapenaltyterm(inadditiontotheaveragesquarederror)thatservestoreducetheweightsoftheleastimportantvariablestozeroornearzero,therebyremovingtheirinfluencefromthenetwork.AseriesofexperimentsforthePOSTOPdatabasewererunusingANNswithandwithouttheweighteliminationcostfunction,andresultswerecomparedfortheparticularoutcome: durationofartificialventilationlessthanorequalto8hoursor morethan8hours.Inthiswork,Trigg[20]verifiedtheaccuracyofthecodebytestingitspredictivepowersonsunspotdatacollectedbyTongin1983[25].TheANNexperimentswithoutweight -(- eliminationsimplyusedthesumofsquarederrorscostfunction. (2)Impactof High/LowNodeApproachfortheInputVariables     Theweighteliminationapproachwasfurthercombinedwithanoveltechniquewheredatawerepresentedtoapairof highand lownodes,dependingonthevalueoftheparameter,again,usingthePOSTOPdatabase.Thismeansthateachofthe14nonbinaryvaluedvariableswaspresentedtotwonodes(ahighnodeandalownode)ratherthantoonenode(i.e.,intheseexperiments,therewere65inputnodes,asopposedto51asinthefirstsetofexperiments).Toimplementthehigh/lownodetechnique,thestandardizedvaluesofnonbinaryvaluedvariableswereassignedasfollows:(1)ifthevalueofthevariablewaszeroorgreater,itwaspresentedtothehighinputnodeforthatvariable,andthecorrespondinglowinputwasassignedavalueofzero;(2)ifthevalueofthevariablewasnegative,itsabsolutevaluewasassignedtothelowinputnodeandthecorrespondinghighinputnodewassettozero.Itwashopedthatthistechniquewouldfacilitatetheindependentinterpretationofhigherorlowerthannormalvaluesofinputparametersinpredictingmedicaloutcomesratherthansimply abnormalvalues.Forexample,afever(i.e.,ahigherthannormalvalue)clinicallypresentsdifferentchallengesthananabnormallylowbodytemperature[20].  b (43$b (3)ReducedNetworkComplexity(NumberofInputVariables)  c   Here,theresearchquestionwas:Howwillreducingthenumberofinputparametersaffecttheanalysistime(thenumberofepochsneededtoreachthehighestcorrectclassificationrate)andtheclassificationrateitself,whencomparedtousingthefullnumberofvariables(51)intheoriginaldatabase?Thisquestionwastestedusingthesameoutcome durationofartificialventilation:lessthanorequalto8hours;ormorethan8hoursforPOSTOPpatients.Twoinputdatasetswereconstructedwithadifferentnumberofinputvariables.Thefirstdatasetcontainedtheoriginal51inputvariablesaslistedinTable1.Theseconddatasetwasconstructedfromthesixvariablesthatattainedthelargestweightsaftertheapplicationoftheweighteliminationcostfunction.Thesixparametersthatremainedafterweighteliminationwere:heartrate,respiratoryrate,fractionofinspiredoxygen,partialpressureofoxygeninblood,arterialpH,andGlasgowComaScore[2628]. Note: SinceTriggsresults[20]withthisparticulardatabaseonlyshowedamarginalbenefitof  9 usingthreelayernetworks(a1%improvementoftheclassificationrate),twolayernetworkswereconstructedtodothiscomparativeanalysis. (4)ImpactoftheConstantPredictorValue $#   Ourresearchgroupfurtherexpandedthisresearchintoananalysisofhowtheconstantpredictor d&!% affectstheperformanceofourANNs[2628].Aconstantpredictorisastatisticalbenchmarkwhereallcasesareclassifiedasbelongingtotheclasswiththehighestaprioriprobability.Inthisseries 6(#' ofexperiments,weinvestigatedhowwelltheANNsclassifiedcasesintotwooutputclassesastherepresentationofthedominantclassapproached100%.Sixdifferentdichotomoussituationsinvolvingthenumberofhoursofmechanicalventilationwereinvestigated:lessthanorequalto4hours,12,24,36and336hours,andbetween24and336hours;andestimatesofthelengthofICUstay:0days,lessthanorequalto1,4,5,and14days).Acommonlyestimatedmedicaloutcomeismortality(or survivalrate),therefore,thisoutputvariablewasalsoinvestigated.Eachoftheabove -(- outcomesunderinvestigationhaddifferentoutcomedistributionswiththedominantclassrangingfrom50.8to98.1%. 4.RESULTSANDDISCUSSION k   MeasuresofPerformance 󀀀Thenetworkperformancewasevaluatedbasedonthecorrect ?  classificationrateofthetestset(i.e.,thenumberofcorrectlyclassifiedcasesdividedbythetotalnumberofcases)andtheareaunderthereceiveroperatingcharacteristic(ROC)curve.Thenumberofepochsrequiredtoreachthebesttestsetclassificationratewasnotedtoprovideameasureoftheconvergencespeedofthetrainingalgorithm.Theresultsarecomparedtotheconstantpredictorandtheminimumdistanceclassifier.Theseclassifiersgaugethedifficultyoftheclassificationproblem,andprovidealowerboundforthenetworksachievableperformance.ThestandarderrorforthereportedclassificationratesandareasundertheROCcurveswasapproximatedbymeasuringthemaximumvariationintheresultsobservedwheneachnetworkwastrainedfromasetoffivedifferentinitialweightconditionsandestimatingtheappropriatevalueashalfthemaximumvariationobserved.  \    Thecostofmisclassificationisanimportantpointtoconsider.Forexample,predictingthatapatientwillnotsurvivesurgery(whenthepatientactuallylives)hasadifferentassociatedcostthanforetellingsurvival,wheninactualfactthepatientwilldie.Inourcase,wearepredictingthedurationofartificialventilationforpatientsintheICU.Misclassificationmayupsetthemanagementofequipmentusageintheunit,however,itwouldnothaveasignificantimpact(negativeorpositive)onthepatient.Thisdecisiontoolisdesignedtoaidtheclinicianinestimatingthedurationofventilationthatthepatientrequires,whichisusefulforconsultationswiththepatientandhis/herfamily,aswellasforresourcemanagement. (1)ImpactoftheWeightEliminationCostFunction _    Theweighteliminationcostfunctionwastestedwiththeoutcome durationofartificial 3 ventilationlessthanorequalto8hoursor morethan8hoursforthePOSTOPpatients.Table2showsthattheweighteliminationANNsachievedacorrectclassificationratethatwasapproximately1.7%betterthanthatofthenoweighteliminationnetworksforthetwolayerANNs,andapproximately1.3%betterforthethreelayernetworks.Weighteliminationalsoeradicatedtheproblemofoverfittingpreviouslymentioned.TheROCresultsshowthatthenetworksdiscriminatedwellbetweenthetwopatientsets,however,theboundsoftheirstandarderrorsoverlapslightly(0.91820.0213and0.93010.0195forthetwoandthreelayernetworks,respectively).  Forthesakeofcompleteness,Table2alsoreportsthattheweighteliminationnetworksexceededtheperformanceoftheconstantpredictorandminimumdistanceclassifier(improvementsof19.4%and4.4%forthetwolayernetworks,and20.7%and5.7%forthethreelayerANNs,respectively).TheseresultsshowthatusingtheweighteliminationcostfunctioncanimprovetheclassificationperformanceoftheseANNs.[20]. (2)Impactof High/ LowNodeApproachfortheInputVariables  +*'+   ,(,   Fourteencontinuousvaluedphysiologicalvariableswereseparatedintohighandlownodesas describedinthemethodologysection,accordingtotheirvaluesrelativetothephysiologicalnormalvalues.Table3comparesresultsobtainedwithusinghigh/lownodesfortheANNswiththeweighteliminationcostfunction,andwithANNsagainusingweighteliminationbutwiththeregulardatarepresentationtechnique(i.e.,allvaluesofthevariablearepresentedtothesamenode,whethertheyarehighorlow).Table3showsaslightdeclineintheclassificationrateforANNsusingthehigh/lownodeformatcomparedtotheregulardatapresentationtechnique(approximately1%and0.3%fortwoandthreelayernetworks,respectively).However,thesenetworksstillattainedhigherclassificationratesthaneithertheconstantpredictorortheminimumdistanceclassifier(18.4%and2.4%forthetwolayerANNS,and20.4%and4.4%forthethreelayernetworks,respectively).ThepoorerperformancecomparedtotheANNswiththeregulardatapresentationapproachcouldbeduetotheincreasednumberofinputvariableswiththehigh/lownetworks.Moreover,theANNsusinghigh/lownodesweremorecomplexgiventhatthethreelayerANNrequired8hiddennodescomparedtotheregularweighteliminationnetworkwhichonlyused2nodesinthehiddenlayer[20]. (3)ReducedNetworkComplexity(NumberofInputVariables)  @   Whenthenumberofinputvariableswasreducedfrom51to6,theANNsclassificationperformanceimproved.Table4showstheresultsfromthetwosimulations.Theconstantpredictoroftheoriginaldatabasewas71.1%.Thehighestcorrectclassificationrateforthistestsetwas88.8%,animprovementof17.7%overtheconstantpredictor.Theresultsstabilizedafterapproximately394epochs.Forthedatasetwithonly6inputvariables,theoriginalassumptionwasthattheresultsmaynotbeasgoodasfortheothercase.Wehypothesizedthattheremightbeunknowninteractionsbetweenvariablesthatwouldnotbepresentwhenusingonlyapartiallist.However,theconstantpredictorforthisdatasetwas72.4%(thedifferenceisduetocasesamplingwhendividingthetrainingandtestsets)andafteronly130epochs,aclassificationrateof90.5%wasobtained[26].  Comparingtheresultsoftheseexperiments,thesimplestdatabasewithonlysixinputvariableswhichusedonlyinputswhoseweightsdidnotgotozeroinTriggsweighteliminationexperiments,producedthehighestcorrectclassificationrate(90.5%)afterjust130epochs[26].Theresultsoftheseexperimentsindicatethatreducingthecomplexityofthissystemincreaseditsgeneralizationabilitytoallowforabettercorrectclassificationofthetestpatternsbasedontheinformationprovidedbythetrainingsetinthefewestepochs. (4)ImpactoftheConstantPredictorValue  u% $   Asthedistributionoftheoutcomevariablesincreasesfrom50.8to98.1%,becomingskewedtowardsonepossibleoutcome,theANNislessabletosignificantlyexceedtheclassificationperformanceoftheconstantpredictor.Figure1illustratestherelationshipbetweenthecorrectclassificationrateoftheANNandthatoftheconstantpredictorforthePOSTOPdatabase;Figure2showsthesamevariablesfortheNONPOSTOPdatabase.ThesefiguresillustratehowtheclassificationrateoftheANNandoftheconstantpredictorconvergetoatheoreticallimitforthesuperiorperformanceoftheANN.Thisoccursasthedivisionbetweenthetwodesiredoutputclassesbecomeshighlyskewedtowards100%.Incaseswherethenumberofsamplepatternsfor -(- aparticularcasearequitesmall,afterthefirstfewpassesthroughtheANN,everythingbecomesclassifiedasbelongingtothelargestclass"inessence,theANNclassifieslikeaconstantpredictor.ThispointisthelowerlimitforacceptableANNperformance.Fromthisinformation,onecandeducetheminimumnumberoftrainingpatternsrequiredfortheANNtoidentifyrareoutcomes.CEUS.,  UsinglinearregressionoftheclassificationratefortheANN,weidentifiedthislimit[2628].ThepointatwhichthelinearregressionlinecrossestheconstantpredictorprojectionsisthetheoreticallimitfortheANNsperformanceabilities.Afterthispoint,asthedivisionbetweentheoutputclassesbecomesmoreskewed,theANNstartsclassifyinglikeaconstantpredictororitsclassificationperformanceisworsethanaconstantpredictorduetomisclassificationofpatientcases.FromFigures1and2,thedominantoutputclassmayrepresentatmost92.0%ofthePOSTOPdatabase,and84.5%fortheNONPOSTOPdatabaseunderconsideration.Theselimitationscannotnecessarilybedirectlyappliedtootherdatabases(medicalorotherwise)becausetheANNreliesheavilyontherelationshipsbetweentheinputparameters.However,thisinformationcouldbeusedasaguidelineforverifyingtheusefulnessofANNsasapredictorwithavarietyofdatabasesandstatisticaldistributions.US.,CE.,0f  TheresultsofthesesimulationsimplythattheANNhadmoredifficultyclassifyingtheNONPOSTOPpatientsthanthePOSTOPpatientcases.ApossibleexplanationisthattheextremediversityofthecircumstancessurroundingthepatientsintheNONPOSTOPsubdatabase,makesthemmoredifficulttoclassifyorthatmoresuchcasesareneededtoimprovetheperformance. 5.CONCLUSION ANDFUTUREWORK      ThenewANNexperimentsyieldedinterestingresults,allowingalargereductioninthe u complexityofthesystemwhilemaintaininghighcorrectclassificationrates.TheresultsarevalidfortheadultICUdatabasesusedintheseexperiments.Otherdatabasesarecurrentlybeingtestedtoseewhetherthisapproachisvalidinavarietyofcontexts.Ongoingworkisapplyingthesametechniquestoneonatalintensivecarepatients(NICU)andtocardiacsurgerypatients[2931].  i   TheworkreportedhereattemptedtoidentifyhowtheperformanceofANNscouldbeimproved.WeconcludethattheweighteliminationcostfunctionnotonlyimprovedthecorrectclassificationratefortheadultICUdatabase,itovercamethenetworkmemorizationproblem.Ontheotherhand,thehigh/lownodeapproach,inthiscase,didnotimprovetheperformanceoftheANNs.Thistechniqueshould,however,beappliedtootherdatabasesbeforeconsideringitineffective.Thethirdapproachhasagreatpotential.Reducingthecomplexityofthedatabasebyeliminatingvariablesthathavelittleimpactontheoutcomewillmakethesystemeasiertoimplementinaclinicalenvironment.Forexample,enteringsixvariablesintoadatabasetoassistwithclinicaldecisionmakingislesstimeconsumingcomparedtosystemsrequiringthefull51variablesusedinthefirstexperiments.Thesmallerdatabasesizeandreducedcomplexitywillfacilitatespeedyoutcomeestimations(duetoalowercomputationaldemandonthesystem)forphysiciansandnurseswhentheprototypeisusedinaclinicalsettingforthisparticulardataset.Finally,theimpactoftheconstantpredictorvalueonANNperformancemaybeusedasaguidelinewhenlookingintoappropriateapproachesforoutcomeestimation. -(-   Infuturework,themissingdataproblemwillbereaddressed.Intheworkdescribedhere,themissingdatawasreplacedwith normalvaluesandtheresultswerequitegood.Themissingvaluequestionmustbefurthertestedwithavarietyofdatabasesandmedicalenvironments. Acknowledgements: ThisworkwascompletedwiththeassistanceofMRCGrantCGAA-45088 T andNSERCGrant20297297.TheresearchgroupisalsogratefultoDr.F.G.SolvenforprovidingtheintensivecarepatientdatabasefromtheDr.E.ChalmersHospitalinFredericton,NB,Canada.ThanksarealsoduetoHelenaHowhoperformedsomeexperimentsdescribedinthispaper. REFERENCES 'rXX'  4  b 9(5h84b )3   )3v25h  1  .3  0 r   HaykinS.Neuralnetworks:acomprehensivefoundation.NewYork:MacMillan1994.)3vv݌ r(#r(# Ќ   )3   )3w25h  2  .3  0 r   HintonGE.Howneuralnetworkslearnfromexperience.SciAm1992;Sep267(9):144151.)3wx݌ r(#r(# Ќ   )3   )3x25h  3  .3  0 r   RichardMD,LippmannRP.NeuralnetworkclassifiersestimateBayesianaposteriori   probabilities.NeuralComp1991;3(4):461483.)3x*y݌ r(#r(# Ќ   )3   )3Rz25h  4  .3  0 r   BaxtWG.Useofanartificialneuralnetworkfordataanalysisinclinicaldecisionmaking:the t  diagnosisofacutecoronaryocclusion.NeuralComp1990;2(4):480489.)3Rz}z݌ r(#r(# Ќ   )3   )3{25h  5  .3  0 r   BaxtWG.Useofanartificialneuralnetworkforthediagnosisofmyocardialinfarction.Ann F InternMed1991;115:843848.)3{{݌ r(#r(# Ќ   )3   )3}25h  6  .3  0 r   KuntzGR,HammondJ.Predictingoutcomesafterclosedheadinjury(CHI)usingacascade h correlationartificialneuralnetwork.CritCareMed1994;22(1):A25.)3}A}݌ r(#r(# Ќ   )3   )3~25h  7  .3  0 r   BuchmanTG,KubosKL,SeidlerAJ,SiegforthMJ.Acomparisonofstatisticaland : connectionistsmodelsforthepredictionofchronicityinasurgicalintensivecareunit.CritCareMed1994;22(5):750762.)3~~݌ r(#r(# Ќ   )3   )3$25h  8  .3  0 r   LauF.Developmentandvalidationofadecisionsupportsystemforcardiovascularintensive  care.CanMedlInfMay/June1994:2829.)3$O݌ r(#r(# Ќ   )3   )3{25h  9  .3  0 r   TuJV,GuerriereMRJ.Useofaneuralnetworkasapredictiveinstrumentforlengthofstay w intheintensivecareunitfollowingcardiacsurgery.ComputBiomedRes1993;26:220229.)3{݌ r(#r(# Ќ   )3   )325h  10  .3  0 r   BuskardT,StevensonM,FrizeM,SolvenFG.Estimationofventilation,lengthofstay,and I mortalityusingartificialneuralnetworks.ProcCanadianConfElec&ComputEng1994:726729.)3.݌ r(#r(# Ќ   )3   )325h  11  .3  0 r   FrizeM,BuskardT,StevensonM,SolvenFG,DunfieldV.Preliminaryresultsofthe  T developmentofanartificialneuralnetworkfortheintensivecare.ProcIEEEEMBS1993:296.)3݌ r(#r(# Ќ   )3   )325h  12  .3  0 r   FrizeM,SolvenFG,StevensonM,NickersonB,BuskardT,TaylorK.Computer-assisted !&  decisionsupportsystemsforpatientmanagementinanintensivecareunit.Medinfo951995:1009-1012.)3@݌ r(#r(# Ќ   )3   )325h  13  .3  0 r   FrizeM,SolvenF,StevensonM,NickersonBG,McGowanHCE.Informationtechnologies $# approachanddevelopmentforvariousmedicalapplications.ProcCanadianConfElec&ComputEng1996:365368.)3͇݌ r(#r(# Ќ   )3   )3725h  14  .3  0 r   CEUS.,KnausWA,ZimmermanJE,WagnerDP,DraperEA,LawrenceDE.APACHEAcute L'"& physiologyandchronichealthevaluation:aphysiologicallybasedclassificationsystem.CritCareMed1981;9:591597.)37b݌ r(#r(# Ќ   )3   )3ۊ25h  15  .3  0 r   KnausWA,DraperEA,WagnerDP,ZimmermanJE.Anevaluationofoutcomefromintensive *W%) careinmajormedicalcentres.AnnInternMed1986;104:410418.US.,CE.,)3ۊ݌ r(#r(# Ќ   )3   )3^25h  16  .3  0 r   DawantBM,UckunS,MandersEJ,LindstromDP.TheSIMONProject:modelbasedsignal +)'+ acquisition,analysisandinterpretationinintelligentpatientmonitoring.IEEEEngMedBiolMag1993;12(4):8291.)3^݌-(-r(#r(# Ќ   )3   )325h  17  .3  0 r   MisianoDR,MeyerhoffME,CollisonME.Currentandfuturedirectionsinthetechnology  relatingtobedsidetestingofcriticallyillpatients.Chest1990:204S214S.)3-݌ r(#r(# Ќ   )3   )3z25h  18  .3  0 r   GardnerRM,ShabotMM.ComputerizedICUdatamanagement:pitfallsandpromises.IntJ  ClinMonitComput1990;7:99105.)3z݌ r(#r(# Ќ   )3   )3Ɛ25h  19  .3  0 r   YaleK.Preparingtherightdatadietfortrainingneuralnetworks.IEEESpectrum,March T 1997:6466.)3Ɛ݌ r(#r(# Ќ   )3   )325h  20  .3  0 r   TriggHCE.Aninvestigationofmethodstoenhancetheperformanceofartificialneural & v networksusedtoestimatemedicaloutcomes.Mastersthesis,ElectricalEngineeringDept,UniversityofNewBrunswick1997.)3)݌ r(#r(# Ќ   )3   )325h  21  .3  0 r   DemuthH,BealeM.Matlab:NeuralNetworkToolboxUsersGuide.Natick,MA:TheMath  1  WorksInc.1997.)3ғ݌ r(#r(# Ќ   )3   )325h  22  .3  0 r   HaykinS.NeuralnetworksexpandSPshorizons.IEEESignalProcessingMag1996;13(2).)3݌ r(#r(# Ќ   )3   )325h  23  .3  0 r   WidrowB,RumelhartDE,LehrMA.Neuralnetworks:applicationsinindustry,business,and   science.CommoftheACM1994;37(3):93105.)3/݌ r(#r(# Ќ   )3   )3^25h  24  .3  0 r   WeigendAS,RumelhartDE,HubermanBA.Generalizationbyweighteliminationwith n  applicationtoforecasting.InRPLippmann,JMoody,DSTouretzky,eds.AdvancesinNeuralInformationProcessingSystems(NIPS90)SanMateo:MorganKaufmann1991;3:875882.)3^݌ r(#r(# Ќ   )3   )3125h  25  .3  0 r   TongH.ThresholdModelsinNonLinearTimeSeriesAnalysis.Springer,1983.)31\݌)yr(#r(# Ќ   )3   )3E25h  26  .3  0 r   EnnettCM,FrizeM.Investigationintothestrengthsandlimitationsofartificialneural b networks:anapplicationtoanadultICUpatientdatabase.ProcAMIASymp1998:998.)3Ep݌ r(#r(# Ќ   )3   )3ƛ25h  27  .3  0 r   FrizeM,WangL,EnnettCM,NickersonB,SolvenFG,StevensonM.Newadvancesand 4 validationofknowledgemanagementtoolsforcriticalcareusingclassifiertechniques.ProcAMIASymp1998:553558.)3ƛ݌ r(#r(# Ќ   )3   )3a25h  28  .3  0 r   HoH.Astudyoftheperformanceofartificialneuralnetworkstoestimateoutcomesinthe  intensivecareunit.Undergraduateseniorthesis,UniversityofOttawa,Ottawa,ON,1998.)3a݌ r(#r(# Ќ   )3   )325h  29  .3  0 r   TongY,FrizeM,WalkerR.Estimatingventilationusingartificialneuralnetworksinintensive q careunits.ProcBMES/EMBS,1999.)3݌ r(#r(# Ќ   )3   )3?25h  30  .3  0 r   WalkerR,FrizeM,TongY.Dataanalysisusingartificialneuralnetworksandcomputeraided C decisionmakingintheneonatalintensivecareunit.PaediatricAcademySociety1999AnnMeeting,PaediatricRes1999;45:231A.)3?j݌ r(#r(# Ќ   )3   )325h  31  .3  0 r   EnnettCM,FrizeM,ShawRE.Methodologiesforpredictingcoronarysurgeryoutcomes.Proc N BMES/EMBS1999. )3݌ 7r(#r(# Ќ  MoniqueFrize (P.Eng.,O.C.)completedherBAScinElectricalEngineeringattheUniversityof " ! Ottawain1966,anM.Phil.atImperialCollegeofScienceandTechnologyinLondon(UK)in1970,anMBAatUniversitdeMonctonin1986,andadoctorateatErasmusUniversiteitinRotterdam(TheNetherlands)in1989.Dr.Frizehaspublishedover80papersinrefereedjournalsandconferenceproceedingsandreceivedfourhonourarydegreessince1992andwasinductedasanOfficeroftheOrderofCanadain1993. ColleenM.Ennett completedherBSc(Eng)inBiologicalEngineeringattheUniversityofGuelph )l$( in1997,andherMAScinElectricalEngineeringattheUniversityofOttawain1999.ColleeniscurrentlyworkingonherdoctorateinSystemsandComputerEngineeringunderthesupervisionofDr.FrizeatCarletonUniversityinOttawa,ON.Herresearchinterestsincludedevelopingmedicaldecisionaidsystems,bioinstrumentationformeasuringphysiologicalparameters,andenhancedassistivetechnologiesforrehabilitationengineering. -(- Ї MaryhelenStevenson receivedtheB.E.E.degreefromtheGeorgiaInstituteofTechnologyin1984.  ShereceivedM.S.andPh.D.degreesinElectricalEngineeringfromStanfordUniversityin1984and1991respectively.ShewasamemberoftheTechnicalStaffatHughesAircraftcompanybetween1983and1989,andhasbeenwiththeDepartmentofElectricalandComputerEngineeringattheUniversityofNewBrunswicksince1990.Herresearchinterestsincludepatternrecognition,adaptivesignalprocessingandneuralnetworks.Dr.StevensonisamemberofIEEE,INNS,PiKappaPhi,EtaKappaNu,andTauBetaPi;sheisaprofessionalengineerintheprovinceofNewBrunswick,Canada. HeatherC.E.Trigg completedherB.Sc.degreeinPhysicsatMcMasterUniversityin1994.She  4  thenwentontodograduatestudiesinthefieldofElectricalEngineeringattheUniversityofNewBrunswick,graduatingin1997withaM.Sc.Eng.degree.Herthesis,entitled"AnInvestigationofMethodstoEnhancethePerformanceofArtificialNeuralNetworksusedtoEstimateICUOutcomes"lookedatwaystoimproveANNoutcomeestimationforICUdataandwasthebasisforthecontinuedworkdescribedinthispaper.SheisnowworkingasanATMTestArchitectatNortelNetworksCorporationinOttawa,ON,Canada. RUNNINGHEADLINE: Frizeetal.,CDSSsforICU:UsingNeuralNetworks  / Table1:ListofvariablesintheadultICUdatabase  .UUhUUhUU(#(#.7hUUhUUhUU  7DemographicsandAdministrative  Information  l ub )0(BAx89b )3   0FrXrX0)3Ѯ2BA3  0 F   AssignedchronichealthpointsinAPACHE V IIscoring)3Ѯ,݌ F8F8 Ќ   )3   )32BA3  0 F   EmergencysurgerypriortoICUadmission)3݌( xF8F8 Ќ   )3   )3а2BA3  0 F   Surgerypriortoadmission)3а݌ aF8F8 Ќ   )3   )32BA3  0 F   Patientgender)3ɱ݌ JF8F8 Ќ   )3   )3`2BA3  0 F   Positionindatasequence*)3`݌ 3 F8F8 Ќ   )3   )3.2BA3  0 F   Patientage(years)*)3.Y݌  F8F8 Ќ    APACHEII(AdmissionInformation)     )3   )3O2BA3  0 F   Rectaltemperature($C)*)3Oz݌ F8F8 Ќ   )3   )32BA3  0 F   Meanarterialpressure(mmHg)*)3I݌q F8F8 Ќ   )3   )32BA3  0 F   Heartrate*)3݌Z F8F8 Ќ   )3   )32BA3  0 F   Respiratoryrate*)3ڶ݌CF8F8 Ќ   )3   )3t2BA3  0 F   Fractionofinspiredoxygen*)3t݌,|F8F8 Ќ   )3   )3D2BA3  0 F   Partialpressureofoxygenintheblood*)3Do݌eF8F8 Ќ   )3   )3 2BA3  0 F   ArterialpH*)3 K݌NF8F8 Ќ   )3   )32BA3  0 F   Serumsodium(mMol/L)*)3 ݌7F8F8 Ќ   )3   )32BA3  0 F   Serumpotassium(mMol/L)*)3պ݌ F8F8 Ќ   )3   )3w2BA3  0 F   Serumcreatinine(Mol/L)*)3w݌ F8F8 Ќ   )3   )3H2BA3  0 F   Hematocrit*)3Hs݌F8F8 Ќ   )3   )32BA3  0 F   Whitebloodcellcount(total/mm3in1000's)*)32݌F8F8 Ќ   )3   )32BA3  0 F   GlasgowComaScore*)3݌tF8F8 Ќ  AdmissionSource  F  )3   )32BA3  0 F   EmergencyRoom)3 ݌0F8F8 Ќ   )3   )32BA3  0 F   4SW)3ο݌iF8F8 Ќ   )3   )3Z2BA3  0 F   4E)3Z݌ RF8F8 Ќ   )3   )32BA3  0 F   4W)3;݌ ;F8F8 Ќ   )3   )32BA3  0 F   4NE)3݌!$ F8F8 Ќ   )3   )3}2BA3  0 F   3W)3}݌" !F8F8 Ќ   )3   )332BA3  0 6   4NW)33^݌!6(#6(# Ќ   )3   )32BA3  0 6   3SW)3݌k"6(#6(# Ќ   )3   )32BA3  0 6   CoronaryCareUnit)3݌T#6(#6(# Ќ   )3   )3g2BA3  0 6   3E)3g݌= $6(#6(# Ќ   )3   )32BA3  0 6   Admissionfromanotherlocation)3H݌& v%6(#6(# Ќ  AdmissionDiagnosis#1   H'  )3   )3"2BA3  0 6   Postoperative)3"M݌ 2(6(#6(# Ќ   )3   )32BA3  0 6   Acutehypercapnicrespiratoryfailure)3݌  )6(#6(# Ќ   )3   )32BA3  0 6   Trauma)3݌ *6(#6(# Ќ   )3!   )3v2BA3  0 6   Drugoverdose)3v݌ +6(#6(# Ќ   )3"   )372BA3  0 6   Ketoacidosis)37b݌ ,6(#6(# Ќ   )3#   )32BA3  0 6   Suddencessationofheartorlungs)3"݌o -6(#6(# Ќ   )3$   )32BA3  0 6   Otherdiagnosis#1)3݌X .6(#6(# Ќ  AdmissionDiagnosis#2  *z0  )3%   )32BA3  0 6   Carotidendarectomy)3݌d16(#6(# Ќ   )3&   )32BA3  0 6   Nothingfilledin)3݌M26(#6(# Ќ   )3'   )3Q2BA3  0 6   Abdominalaorticaneurysmrepair)3Q|݌636(#6(# Ќ   )3(   )3%2BA3  0 6   Motorvehicleaccident)3%P݌46(#6(# Ќ   )3)   )32BA3  0 6   Lobectomy)3݌56(#6(# Ќ   )3*   )32BA3  0 6   Aortobifemoralbypass)3݌66(#6(# Ќ   )3+   )3u2BA3  0 6   Pneumonia)3u݌76(#6(# Ќ   )3,   )322BA3  0 6   Acutepulmonaryedema)32]݌s86(#6(# Ќ   )3-   )32BA3  0 6   Otherdiagnosis#2)3&݌\96(#6(# Ќ  AdmissionDiagnosis#3  .~;  )3.   )32BA3  0 6   Nothingfilledin)3݌h<6(#6(# Ќ   )3/   )32BA3  0 6   Lungcancer)3݌ Q=6(#6(# Ќ   )30   )3w2BA3  0 6   Postoperative)3w݌ :>6(#6(# Ќ   )31   )382BA3  0 6   Ischemicfoot)38c݌!#?6(#6(# Ќ   )32   )32BA3  0 6   Otherdiagnosis#3)3$݌ " @6(#6(# Ќ  DUUhUUhUU  hhh D.UUhUUhUU  .5 XX*Identifiesnonbinaryvaluedvariables(Note:fractionofinspiredoxygenisaonesidedcontinuous_variables,_Ԁtherefore, $@ thehigh/lownodeexperimentsdidnotrequireanadditionalnodeforthisvariable).#X X52#  R% A Table2:PerformanceofANNsusingweighteliminationandnoweighteliminationcomparedtotheconstantpredictor(CP)andtheminimumdistanceclassifier(MDC)forthePOSTOPpatientdatabase  * % > ddd Xdd Xdd X(#(#,A $$ ,$$ ,X$$ ,$$ ,X$$ ,$$ ,$$ +  #kk $$#3 XX ANNarchitecture !  !Maxtest  setCCR* P (%) 9/! a" 9CP  (%)#X X3#3 XX  9/!P " 9Performance   improvement P  overCP(%)#X X3#3 XX  9/! a " 9MDC   (%)#X X3#3 XX  9/!P" 9Performance  improvement P overMDC#X X3t#3 XX Ԁ(%)#X X38#3 XX  9/! a" 9ROC**  curves ;1'P" $$  $$;Besttwolayernetwork !  j ! 9/! j" 9 9/! j" 9 9/! j" 9 9/! j" 9#X X3~#3 XX  9/! j" 9 >4' j" $$  $$>򀀀withweightelimination(51:1)#X X3o#3 XX  0&# s 090.5 # s 1.20#X X3O#3 XX  [Q! 4" fffffQ@71.1fffffQ@[71.1#X X3#3 XX  |rB# s" fffffQ@71.1 fffffQ@ ffffff3@19.4ffffff3@|19.4#X X3#3 XX  |rB# s" ffffff3@19.4 ffffff3@ fffffU@86.1fffffU@|86.1#X X3F#3 XX  {qB# s " fffffU@86.1 fffffU@ @4.4@{4.4#X X3#3 XX  YOA# s!" @4.4 @ Y0.9182 # s" #X X3#3 XX 0.0213 >4' 4#" $$  $$>򀀀withoutweightelimination(51:1) 0& =$ 088.8#X X3#3 XX   =% 0.70 [Q! &" fffffQ@71.1fffffQ@[71.1 |rB ='" fffffQ@71.1 fffffQ@ 333331@17.7333331@|17.7 |rB =(" 333331@17.7 333331@ fffffU@86.1fffffU@|86.1#X X3l#3 XX  {qB =)" fffffU@86.1 fffffU@ @2.7@{2.7 YOA =*" @2.7 @ Y0.9165#X X3&#3 XX   =+ 0.0194 >4' ," $$  $$>Bestthreelayernetwork#X X3J#3 XX  0& - 0 9/! ." 9 9/! /" 9 9/! 0" 9 9/! 1" 9 9/! 2" 9 >4' 3" $$  $$>򀀀withweightelimination(51:2:1)#X X3#3 XX  0& 4 091.8  5 1.15#X X3#3 XX  [Q! 6" fffffQ@71.1fffffQ@[71.1#X X3#3 XX  |rB 7" fffffQ@71.1 fffffQ@ 333334@20.7333334@|20.7#X X3(#3 XX  |rB 8" 333334@20.7 333334@ fffffU@86.1fffffU@|86.1#X X3#3 XX  {qB 9" fffffU@86.1 fffffU@ @5.7@{5.7#X X3 #3 XX  YOA :" @5.7 @ Y0.9301#X X3l #3 XX   ; 0.0195 >4' <" $$  $$>򀀀withoutweightelimination(51:2:1)#X X3 #3 XX  0& = 090.5  > 0.90#X X3 #3 XX  [Q!K ?" fffffQ@71.1fffffQ@[71.1 |rB @" fffffQ@71.1 fffffQ@ ffffff3@19.4ffffff3@|19.4 |rB A" ffffff3@19.4 ffffff3@ fffffU@86.1fffffU@|86.1 {qB B" fffffU@86.1 fffffU@ @4.4@{#X X3m #3 XX 4.4 YOA C" @4.4 @ Y#X X3#3 XX 0.9212  D 0.0195#X X3'#7-+K E" $$   73 XX *CCR=correctclassificationrate 0E **ROC=receiveroperatingcharacteristic AF #X X3#  G Table3:PerformanceofweighteliminationANNsusingregulardatapresentationorhigh/lownodes  datapresentationcomparedtotheconstantpredictor(CP)andtheminimumdistanceclassifier  (MDC)forPOSTOPpatientcases  * 7 B ddA $$ A $$ X$$ X$$ X$$ X$$ $$ % >(#(#, $$ ,$$ ,9$$ ,$$ ,$$ , $$ ,d$$ ,$$ +  #kk $$#3 XX ANNarchitecture !  !No.of  input P variables 9/! a" 9Maxtest  setCCR* P  (%) 9/! a " 9CP   (%)#X X3#3 XX  9/!P " 9Performance   improvement P overCP(%)#X X3#3 XX  9/! a" 9MDC  (%)#X X3#3 XX  9/!P" 9Performance  improvement P overMDC#X X3q#3 XX Ԁ(%)#X X35#3 XX  9/! a" 9Area  under P ROC**  a curves ;1' "" $$  $$;Besttwolayernetwork !  + ! 9/! +" 9 9/! +" 9 9/! +" 9 9/! +" 9 9/! +" 9#X X3{#3 XX  9/! +" 9 >4' + " $$  $$>#X X3#3 XX 򀀀withweightelimination PF 4!  I@51I@P51 XN@ 4""  I@51 I@ X#X X3#3 XX 90.5#X X3#3 XX   4# 1.20#X X3#3 XX  [Q! $" fffffQ@71.1fffffQ@[71.1#X X3C#3 XX  |rB 4%" fffffQ@71.1 fffffQ@ ffffff3@19.4ffffff3@|#X X3#3 XX 19.4#X X3#3 XX  |rB 4&" ffffff3@19.4 ffffff3@ fffffU@86.1fffffU@|86.1#X X3#3 XX  {qB 4'" fffffU@86.1 fffffU@ @4.4@{#X X3#3 XX 4.4#X X3g#3 XX  YOA 4(" @4.4 @ Y0.9182#X X3#3 XX   4) 0.0213 >4' *" $$  $$>򀀀withweighteliminationand  + Ѐhigh/lownodes#X X3R #3 XX  PFo ,  @P@65@P@P65 XN@ -"  @P@65 @P@ X#X X37!#3 XX 89.5  . 0.85#X X3#"#3 XX  [Q!o /" fffffQ@71.1fffffQ@[71.1#X X3"#3 XX  |rB 0" fffffQ@71.1 fffffQ@ ffffff2@18.4ffffff2@|18.4#X X3(##3 XX  |rB 1" ffffff2@18.4 ffffff2@ fffffU@87.1fffffU@|87.1#X X3##3 XX  {qB 2" fffffU@87.1 fffffU@ 333333@2.4333333@{2.4#X X3$#3 XX  YOA 3" 333333@2.4 333333@ Y0.9204  4 0.0207#X X3l%#3 XX  >4'o 5" $$  $$>Bestthreelayernetwork#X X3-&#3 XX  0&x 6 0 9/!x 7" 9#X X3&#3 XX  9/!x 8" 9 9/!x 9" 9#X X3q'#3 XX  9/!x :" 9 9/!x ;" 9#X X3%(#3 XX  9/!x <" 9 >4'x =" $$  $$>#X X3(#3 XX 򀀀withweightelimination  > Ѐ(2hiddennodes)#X X3)#3 XX  PFB ?  I@51I@P51 XN@ @"  I@51 I@ X#X X3*#3 XX 91.8#X X3 +#3 XX   A 1.15 [Q!B B" fffffQ@71.1fffffQ@[71.1#X X3T+#3 XX  |rB C" fffffQ@71.1 fffffQ@ 333334@20.7333334@|#X X3,#3 XX 20.7 |rB D" 333334@20.7 333334@ fffffU@86.1fffffU@|86.1#X X3,#3 XX  {qB E" fffffU@86.1 fffffU@ @5.7@{#X X3-#3 XX 5.7 YOA F" @5.7 @ Y0.9301  G 0.0195#X X3O.#3 XX  >4'B H" $$  $$>#X X3/#3 XX 򀀀withweighteliminationand K I Ѐhigh/lownodes#X X3/#3 XX Ԁ(8hidden  \J Ѐnodes)#X X30#3 XX  PFK  @P@65@P@P65 XN@K L"  @P@65 @P@ X91.5 K M 1.00#X X30#3 XX  [Q! \N" fffffQ@71.1fffffQ@[71.1 |rBK O" fffffQ@71.1 fffffQ@ ffffff4@20.4ffffff4@|20.4#X X31#3 XX  |rBK P" ffffff4@20.4 ffffff4@ fffffU@87.1fffffU@|87.1 {qBK Q" fffffU@87.1 fffffU@ @4.4@{#X X32#3 XX 4.4#X X33#3 XX  YOAK R" @4.4 @ Y#X X3:4#3 XX 0.9480 K S 0.0146#X X34#7-+ \T" $$   73 XX *CCR=correctclassificationrate T **ROC=receiveroperatingcharacteristic sU #X X3t5#  VX Table4:ComparisonofANNtestresults  *? Add $$  $$ 9$$ 9$$ $$  $$ d$$ d$$ 7 B(#(#,  , ,e ,r ,F +  3 " 33 XX Database @ /!"  @#Variables#X X3.8#3 XX  @ /!"  @MaxCCR*(%) @ /!"  @ASE**atMaxCCR @ /!"  @Approx#Epochs G='"    GAllvariables(demographics,APACHE  IIvariables,admissionsource, c admissiondiagnoses) YO!$t "  I@51I@Y51#X X38#3 XX  zp@ "  I@51 I@ 333333V@88.8333333V@z88.8 |rB " 333333V@88.8 333333V@ RQ?0.38RQ?|0.38 {qB " RQ?0.38 RQ? x@394x@{394#X X3;#3 XX  g]G " x@394  x@ gVariableswithlargestweightsafter  e weightelimination XN! &"  @6@X6#X X3<#3 XX  yo? e"  @6 @ V@90.5V@y90.5 |rB e" V@90.5 V@ {Gz?0.32{Gz?|0.32 {qB e" {Gz?0.32 {Gz? @`@130@`@{130#X X3># XX WMK e" @`@130   @`@ W#X X?#3 XX *CCR=correctclassificationrate   **ASE=averagesquarederror p  #X X3s@#  1  m19)%`|0k `.EGkkKm (#(#                    (#(#Figure1:Comparisonofcorrectclassificationrate(CCR)forconstantpredictorandANNusing  POSTOPdatabase    (#(#                    (#(#m<9)%`|0 `.E>V(Xm߰Figure2:Comparisonofcorrectclassificationrate(CCR)forconstantpredictorandANNusing  NONPOSTOPdatabase#X6X XX p#