Skip to content

Thread safety and cache integrity improvements#911

Open
sreckoskocilic wants to merge 7 commits intofman-users:mainfrom
sreckoskocilic:thread_safety_cache_codescan
Open

Thread safety and cache integrity improvements#911
sreckoskocilic wants to merge 7 commits intofman-users:mainfrom
sreckoskocilic:thread_safety_cache_codescan

Conversation

@sreckoskocilic
Copy link
Copy Markdown

Summary

  • fs_cache.py rewrite — full lock hierarchy documented and enforced: Cache._lockCacheItem._children_lockCacheItem._attr_locks_lock → per-attr RLock. Generation counter detects invalidation during query() and retries. New mutate() and clear_attr() methods for atomic cache operations
  • CachedIterator concurrency fix — split into _lock (protects items/counts) and _source_lock (serializes source advancement). remove()/append() use proper count clamping. get_next() re-scans after acquiring _source_lock to avoid missed items
  • mother_fs.py_remove_from_parent / _add_to_parent use cache.mutate() instead of get-then-modify pattern; _clear_parent_stat invalidates parent's stat and icon cache on add/remove; notify_file_added clears child cache before notification; iterdir always wraps in CachedIterator
  • record_files.py — missing _m_files[url] = file_ assignment when file passes filter but isn't in _m_files yet
  • Event systemadd_callback / remove_callback / trigger protected by Lock; trigger snapshots callback list before iteration
  • DirectoryPane._broadcast — iterates over list(self._listeners) snapshot to avoid mutation during iteration
  • file_under_cursor context manager — thread-local storage replaces monkey-patched method; try/finally ensures cleanup
  • icon_provider.pyLock around suffix cache dict; .messagestr(e) for FontError compatibility
  • model/__init__.py_MAX_VISITED cap, sourceModel() null guard, iterative parent traversal, shutdown() method (shared with critical crashes branch)
  • model/diff.py — 5 assert statements replaced with raise RuntimeError for production safety
  • qt/thread.pyExecutor cleanup: removed bare except, added KeyError guard on timer cleanup, deleteLater() ensures Qt event processing
  • fman/__init__.pyAPPDATA env var check on Windows; Canceled extends Exception not KeyboardInterrupt; listener iteration snapshots
  • fs.pyFileSystem docstring added; _file_changed_callbacks protected by lock with snapshot-then-iterate pattern; copy/move docstrings clarify they receive full URLs
  • 380+ lines of new teststest_fs_cache.py (14 tests), test_mother_fs.py (11 tests), test_fs.py (3 tests) covering cache concurrency, generation-based invalidation, CachedIterator thread safety, and event callback thread safety

Test plan

  • Navigate directories rapidly — verify no cache corruption or stale entries
  • Delete/add files while directory is loading — verify parent cache invalidation
  • Run test suite: all 246+ tests pass
  • Stress test: rapid directory switching with concurrent file operations

…paring two tuples, every equality check returns True

- master -> main branch fix
- prepare_trash now calls self.move_to_trash instead of self.delete, so plugin subclasses won't permanently delete files when the user expects trashing
- NotImplementedError - unrecognized platforms fail fast with a clear message instead of a cryptic NameError
- removed shadowing basename import
- get_column_widths uses range so plugin-added columns get their widths saved/restored
…it__ always runs, even on exception.

  - util/qt/__init__.py — Added missing c_void_p import from ctypes, fixing a macOS runtime crash.
  - table.py — Fixed off-by-one: bounds check now rejects len + 1 correctly.
  - widgets.py — Added null guard on _main_window before accessing it in state change handler.
…ved, so a/b/c/../../d correctly becomes a/d.

  - session.py — Removed dead _get_startup_message method (duplicated by _show_startup_messages)
- Add Cache._lock with generation counter to detect stale query results
- Use RLock per-attr in CacheItem for reentrant nested queries
- Split CachedIterator locking: _lock for items, _source_lock for source
- Always wrap iterdir results in CachedIterator for uniform cache ops
- Protect Event callbacks with Lock, snapshot before iteration
- Add _tasks_lock to Executor for safe quit/submit races
- Use thread-local for _override_file_under_cursor (was monkey-patching)
- Snapshot _listeners list before iteration in DirectoryPane._broadcast
- Lock notify_file_changed callback snapshot in FileSystem
- Add _cache_lock to IconProvider for thread-safe surrogate creation
- Cap _already_visited set at 512, convert recursion to iteration
- Add shutdown() to SortedFileSystemModel for clean signal disconnect
- Replace assert with RuntimeError in diff.py for production safety
- Fix missing _m_files update in record_files.py for filtered entries
- Change Task.Canceled base from KeyboardInterrupt to Exception
- Add APPDATA null guard on Windows
- 47 new concurrency and cache invalidation tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant