Gradient boosting: nastavitev hiperparametrov

1 Priprava podatkov

Izbrane spremenljivke (p < 0.10):

  1. HospitalizationBeforeSurgeryMo
  2. Age
  3. BMI
  4. CardiogenicSchockYN
  5. Diabetes
  6. DiabetesPerOsTherapie
  7. DiabetesOnInsuline
  8. PerifernoArterijskoObolenje
  9. ExtracardiacArteriopathy
  10. PsychoSyndrome
  11. TherapyRelevantPsychoSyndrome
  12. PreoperativeInfectionYN
  13. KongestiveHeartFailure
  14. EjectionFractionEF
  15. EF50
  16. AtrialFibrillationYN
  17. ChronicLungDiseaseYN
  18. CockcraftGaultIndexPreop
  19. ACEInhibitors
  20. IABPPreoperatively
  21. DurationOfTheOperation
  22. NumberOfGrafts
  23. PericardDrainage
  24. RethoracotomyYN
  25. CoagulationDisorder
  26. Cardioversion
  27. SumOtherInfectYN
  28. AcuteKidneyFailure
  29. TotalDrainage
  30. NumberOfPlasmaUnits
  31. Transfusion
  32. MoreThan2UnitsOfErythrocytes
  33. RespiratoryFailureYN
  34. ProlongedMechanicalVentilation
  35. Reintubation
  36. NumberOfReintubations
  37. Tracheotomy
  38. AorticClampingTime
  39. BypassOperationTime
  40. AorticCalcificatio
  41. LeukocytesFirstPostoperativeDa
  42. LeukocytesSecondPostoperativeD
  43. HbPreop12GDl
  44. HbPreoperativelyGDl
  45. PoorGlycemicControlPrediabetes
  46. ITA
  47. BIMA
  48. SaEtAlCreatinine226MgDdlOrPost
  49. TiesselFibrinGlueMl
  50. GFRLaurisProdop60NotNormal
  51. GFRLaurisPostop1stDay60NotNorm
  52. GFRLaurisPostop2ndDay60NotNorm
  53. PleuralEffusion

Target: DSWI01 | Predictors: 53 | Samples: 4023

Izvedeno skaliranje s standardizacijo numeričnih spremenljivk in “one-hot encoding” za kategorijske spremenljivke.

2 Nastavitev hiperparametrov

Najboljša nastavitev:

  • model__learning_rate: 0.2
  • model__max_depth: 4
  • model__min_samples_leaf: 3
  • model__min_samples_split: 2
  • model__n_estimators: 400
  • model__subsample: 0.7

Best roc_auc: 0.937

Top 10 kombinacij (CV):

params_str mean_test_accuracy mean_test_f1 mean_test_roc_auc
model__learning_rate=0.2, model__max_depth=4, model__min_samples_leaf=3, model__min_samples_split=5, model__n_estimators=400, model__subsample=0.7 0.955 0.748 0.937
model__learning_rate=0.2, model__max_depth=4, model__min_samples_leaf=3, model__min_samples_split=2, model__n_estimators=400, model__subsample=0.7 0.955 0.748 0.937
model__learning_rate=0.05, model__max_depth=4, model__min_samples_leaf=3, model__min_samples_split=5, model__n_estimators=400, model__subsample=0.7 0.953 0.731 0.935
model__learning_rate=0.05, model__max_depth=4, model__min_samples_leaf=3, model__min_samples_split=2, model__n_estimators=400, model__subsample=0.7 0.953 0.731 0.935
model__learning_rate=0.1, model__max_depth=4, model__min_samples_leaf=3, model__min_samples_split=5, model__n_estimators=400, model__subsample=0.7 0.953 0.737 0.935
model__learning_rate=0.1, model__max_depth=4, model__min_samples_leaf=3, model__min_samples_split=2, model__n_estimators=400, model__subsample=0.7 0.953 0.737 0.935
model__learning_rate=0.2, model__max_depth=4, model__min_samples_leaf=1, model__min_samples_split=2, model__n_estimators=400, model__subsample=0.7 0.951 0.728 0.935
model__learning_rate=0.05, model__max_depth=4, model__min_samples_leaf=1, model__min_samples_split=2, model__n_estimators=400, model__subsample=0.7 0.951 0.717 0.932
model__learning_rate=0.2, model__max_depth=4, model__min_samples_leaf=1, model__min_samples_split=5, model__n_estimators=400, model__subsample=1.0 0.956 0.752 0.932
model__learning_rate=0.1, model__max_depth=4, model__min_samples_leaf=1, model__min_samples_split=2, model__n_estimators=400, model__subsample=0.7 0.953 0.733 0.932

CV setup: StratifiedKFold with 5 folds (shuffle=True, random_state=1974).