Skip to content

Final submission: temporal evaluation + risk prediction + tests + exa…#1118

Open
leeethinh wants to merge 1 commit intosunlabuiuc:masterfrom
leeethinh:emdot-temporal-eval
Open

Final submission: temporal evaluation + risk prediction + tests + exa…#1118
leeethinh wants to merge 1 commit intosunlabuiuc:masterfrom
leeethinh:emdot-temporal-eval

Conversation

@leeethinh
Copy link
Copy Markdown

Name: Thinh Le
NetID: tle91
Email: tle91@illinois.edu

Type of contribution: Standalone Task + Evaluation Utility

Paper: Zhou et al., Evaluating Model Performance in Medical Datasets Over Time
https://www.helen-zhou.com/assets/pdf/emdot/paper.pdf

Description:
This PR implements a temporal evaluation pipeline inspired by the EMDOT framework.
It evaluates models by training on past data and testing on future data, simulating
real-world deployment conditions.

The implementation includes:
Temporal risk prediction task for MIMIC-style data.
Temporal evaluation pipeline (train on past, test on future).
Random baseline for comparison.
Ablation study across multiple time splits.
Synthetic, fast unit tests.
Example scripts demonstrating usage.

Files to review:

  • pyhealth/tasks/temporal_risk_prediction.py
  • pyhealth/tasks/temporal_evaluation.py
  • tests/test_temporal_risk_prediction.py
  • tests/test_temporal_evaluation.py
  • examples/synthetic_temporal_shift_demo.py
  • examples/synthetic_temporal_mortality_logistic.py
  • examples/mimic4_temporal_mortality_logistic.py
  • docs/api/tasks/pyhealth.tasks.temporal_evaluation.rst
  • docs/api/tasks/pyhealth.tasks.temporal_risk_prediction.rst

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant