Repository navigation
Add validated CT/SEG tensor and patient-split tutorial - #116
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
Hi @denis-samatov thanks for this proposal. @fedorov and I discussed this and we really liked the idea and think it could be very useful for IDC users, however we were concerned that the original notebook had too much validation code that distracted from the main purpose of the tutorial (loading paired CTs and SEGs), and too little explanation of the key concepts. We also felt that conflating the cohort selection and image loading parts was unhelpful. Rather than go backwards and forwards on this, I decided to propose a simpler version that Claude created using your initial notebook, by distilling the main points, removing much of the distracting validation code, and adding some text explanation to guide the reader through the concepts used. But the concept is your own. I pushed this to the PR already. Perhaps you could look through it and see what you think and whether this would be useful to new IDC users? |
|
Hi @CPBridge, thank you for taking the time to simplify this, and
thanks @fedorov for the feedback. I reviewed the version at `ca09520`
and agree that it is a better introduction for new IDC users. The
explanations of spatial slice ordering, SEG channels, geometry
matching and tensor dimensions make the main workflow much easier to
follow. I support this narrower scope, with patient splitting left as
next-step guidance.
I also executed the discovery/download cells and all subsequent code
cells locally using idc-index 0.13.4, highdicom 0.28.2 and PyTorch
2.14.1+cpu. The query selected KiTS-00061, KiTS-00038 and KiTS-00110,
downloading 56,547,222 bytes. All three pairs loaded and passed
through the Dataset/DataLoader successfully. I checked matching
CT/mask affines and spatial shapes, binary nonempty mask channels, HU
values against explicit modality-only conversion, and the normalized
intensity range. I also inspected the first-pair overlay. Notebook
schema validation and the repository's standards check for this
notebook passed. This was a local run; I have not verified Google
Colab.
I found three small documentation points:
- The README entry and PR title/body still describe the previous
version, including `volume_geometry_index`, source-reference
validation, split manifests and the removed behavior tests. They
should be updated to match the simplified notebook, for example
**“Loading CT images and DICOM segmentations into PyTorch tensors.”**
- The SEG introduction says there are two variants. DICOM also
supports FRACTIONAL SEG. Perhaps: “This example uses BINARY
segmentations; DICOM also supports FRACTIONAL and LABELMAP
representations.”
- The `match_geometry()` explanation should include cropping alongside
padding, flipping and axis permutation. The point about no
interpolation is correct.
I found no execution blocker on the selected cohort. The latest “Check
Notebook Standards” workflow is currently `action_required`, so it
still needs approval before GitHub can run it for this revision.
Overall, I am happy with this direction and think the revised tutorial
will be useful to new IDC users.
Best regards,
Denis
…On Wed, 21 Oct 2026 14:04:13 -0700, Chris Bridge ***@***.***> wrote:
CPBridge left a comment (ImagingDataCommons/IDC-Tutorials#116)
Hi @denis-samatov thanks for this proposal. @fedorov and I discussed this and we really liked the idea and think it could be very useful for IDC users, however we were concerned that the original notebook had too much validation code that distracted from the main purpose of the tutorial (loading paired CTs and SEGs), and too little explanation of the key concepts. We also felt that conflating the cohort selection and image loading parts was unhelpful.
Rather than go backwards and forwards on this, I decided to propose a simpler version that Claude created using your initial notebook, by distilling the main points, removing much of the distracting validation code, and adding some text explanation to guide the reader through the concepts used. But the concept is your own.
I pushed this to the PR already. Perhaps you could look through it and see what you think and whether this would be useful to new IDC users?
--
Reply to this email directly or view it on GitHub:
#116 (comment)
You are receiving this because you were mentioned.
Message ID: ***@***.***>
|
|
Thanks @denis-samatov good points/suggestions. I applied them in the last commit |
fedorov
left a comment
There was a problem hiding this comment.
This looks really nice, thank you for working on this @denis-samatov and @CPBridge!
Add a self-contained tutorial for turning a small public C4KC-KiTS cohort into validated CT/SEG tensors and a patient-disjoint train/test manifest.
Related to #115. This is a concrete implementation for review; the proposal's scope and notebook placement have not yet been confirmed by a maintainer.
Changes
Validation
Repeated locally on 13 October 2026, Python 3.12.14:
Execution outputs are cleared in the submitted notebook to keep the diff small; its cell sources were checked against the fully executed copy. Google Colab execution has not been verified.
Review focus
Would this fit advanced_topics, or would collections_demos be preferable? Feedback on the validation boundary and the four-patient teaching example would be useful. This convenience sample is not a model-quality benchmark; enhanced CT, irregular grids and resampled segmentations are intentionally outside scope.
AI assistance was used for implementation and local validation.