FIX: annotation onsets usable directly with get_data and plot#13893
Closed
SanskaarUndale21 wants to merge 3 commits intomne-tools:mainfrom
Closed
FIX: annotation onsets usable directly with get_data and plot#13893SanskaarUndale21 wants to merge 3 commits intomne-tools:mainfrom
SanskaarUndale21 wants to merge 3 commits intomne-tools:mainfrom
Conversation
raw.annotations[i]["onset"] previously returned time in meas_date-relative reference (including first_samp offset), but get_data(tmin=...) and plot(start=...) expect time in raw.times reference (0-indexed from first_samp). For FIFF files with first_samp != 0 this caused misaligned data extraction. Fix: tag annotations with _raw_first_time in set_annotations, then subtract it in Annotations.__getitem__ so the returned onset is always in raw.times reference. Slice access propagates the tag. crop_by_annotations no longer needs its manual - self.first_time correction. Also fix events_from_annotations chunk_duration branch which iterated over annot["onset"] and passed it to time_as_index with the wrong origin. Closes mne-tools#13890
- Remove double-correction in _brainvision.py: annot["onset"] now returns raw.times-relative value, so subtracting raw.first_time was applying the offset twice - Fix _raw_first_time comment in set_annotations to match the actual attribute name being set - Make Annotations.append() aware of _raw_first_time so callers can pass onsets in raw.times reference (consistent with __getitem__)
- Revert Annotations.append() change: the previous commit incorrectly added _raw_first_time to stored onsets, breaking tests that already pass meas_date-relative values to append() - Fix EpochAnnotationsMixin.get_annotations_per_epoch: it computed onset relative to epoch t_zero using annot["onset"] (now in raw.times reference) subtracted from epoch_tzeros (meas_date-relative), causing a _raw_first_time-sized offset. Use the raw onset array directly to keep both sides in the same reference frame. - Add towncrier changelog entry for this fix - Add Sanskaar Undale to contributor list
|
Hello! 👋 Thanks for opening your first pull request here! ❤️ We will try to get back to you soon. 🚴 |
There was a problem hiding this comment.
Pull request overview
This PR fixes a long-standing time-reference mismatch where raw.annotations[i]["onset"] could not be used directly with user-facing APIs like raw.get_data(tmin=...) and raw.plot(start=...) when first_samp != 0, by returning onsets in raw.times (0-based-from-first_samp) reference.
Changes:
- Tag
Raw-attachedAnnotationswith_raw_first_timeinBaseRaw.set_annotations()and use it inAnnotations.__getitem__to returnann["onset"]inraw.timesreference. - Remove now-redundant manual
first_timecorrections in callers (e.g.,crop_by_annotations, BrainVision export) and adjustevents_from_annotations(chunked events) and epoch-annotation alignment logic to use consistent internal vs public reference frames. - Add a changelog entry and contributor name.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
mne/io/base.py |
Tags annotations with _raw_first_time; updates crop_by_annotations onset handling. |
mne/export/_brainvision.py |
Stops subtracting raw.first_time from per-annotation onset before time_as_index. |
mne/annotations.py |
Adjusts Annotations.__getitem__ to return raw-times onsets; updates epoch-annotation and chunked-events computations to avoid double offsets. |
doc/changes/names.inc |
Adds contributor reference for changelog attribution. |
doc/changes/dev/13892.bugfix.rst |
Documents the bugfix in the development changelog. |
Comments suppressed due to low confidence (1)
mne/annotations.py:524
Annotations.__getitem__now subtracts_raw_first_timefrom the returnedonsetbut still returnsorig_time=self.orig_time. This makes the per-item contract inconsistent: the returned onset is no longer “seconds after orig_time” (per the class docstring), so users doingannot['orig_time'] + annot['onset']will get the wrong absolute time. Consider also adjusting the returnedorig_time(e.g., shift by_raw_first_timewhen not None) or otherwise ensure theonset/orig_timepair stays consistent.
out_keys = ("onset", "duration", "description", "orig_time")
# When attached to a Raw object, subtract _raw_first_time so that
# ann["onset"] is in raw.times reference (usable directly with
# raw.get_data(tmin=...) and raw.plot(start=...)).
_raw_ft = getattr(self, "_raw_first_time", 0.0)
out_vals = (
self.onset[key] - _raw_ft,
self.duration[key],
self.description[key],
self.orig_time,
)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What the problem is
When you read back
raw.annotations[0]["onset"], the value you get is not directly usable withraw.get_data(tmin=...)orraw.plot(start=...). For files wherefirst_samp != 0(basically all FIFF files), the onset comes back with afirst_timeoffset baked in, butget_dataandplotboth expect time inraw.timesreference which starts at 0.So if you do something like:
...you get data from completely the wrong part of the recording. For the sample dataset,
ann["onset"]comes back as ~46 instead of 3, soget_datafetches from 46 seconds in rather than 3.Fixes #13890.
Why it happens
set_annotationsinternally adds_first_time(=first_samp / sfreq) to stored onsets whenorig_time is None, so annotations line up withmeas_date. The problem is__getitem__hands that raw stored value straight back to the user, with no adjustment. Meanwhile everything that uses annotations internally (_sync_onset,_annotations_starts_stops, the viz code) all already compensate for this offset. The user never gets a chance to.Worth noting that
crop_by_annotationsalready had a manualannot["onset"] - self.first_timecorrection in it, confirming this was a known mismatch.What this PR does
Tags the stored
Annotationsobject with_raw_first_timeinsideset_annotations, then subtracts it inAnnotations.__getitem__before returning to the caller. This meansann["onset"]comes back inraw.timesreference, which is what every user-facing API expects.raw.annotations[i]["onset"]now returns the onset inraw.timesreference__iter__goes through__getitem__)- self.first_timecorrection fromcrop_by_annotationschunk_durationbranch ofevents_from_annotationswhich was building onsets fromannot["onset"]and then passing them totime_as_indexwith the wrongorigin_mne_annots2pybv_eventsin BrainVision export which was manually subtractingfirst_timefromannot["onset"](would double-subtract with the new behavior)EpochAnnotationsMixin.get_annotations_per_epochto use the raw onset array directly when computing annotation position relative to epoch t_zero (since that calculation requires both sides in the same internal reference frame)The internal
annotations.onsetarray is untouched, so all the internal machinery keeps working exactly as before.Testing
Ran the full annotation test suite (
mne/tests/test_annotations.py) and the raw I/O test suite (mne/io/tests/test_raw.py) locally. Verified bothtest_chunk_durationcases withfirst_samp=10000pass, andtest_crop_by_annotationspasses for all parameter combinations.Also verified the exact reproduction case from the issue:
Workaround until this lands