Skip to content

Handle failure during thread creation #2471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sentrivana merged 12 commits into from
Nov 2, 2023
Merged

Handle failure during thread creation #2471

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
58568fd
Handle failure during thread creation
antonpirker Oct 30, 2023
a24a1c4
Handling starting threads during startup
antonpirker Oct 31, 2023
60b5e32
typos
antonpirker Oct 31, 2023
7b03304
Merge branch 'master' into antonpirker/handle-creation-of-threads-dur…
antonpirker Oct 31, 2023
1392666
Merge branch 'master' into antonpirker/handle-creation-of-threads-dur…
sentrivana Nov 2, 2023
25b87a6
add comments
sentrivana Nov 2, 2023
807f09f
revert condition
sentrivana Nov 2, 2023
30bf13a
add profiler test
sentrivana Nov 2, 2023
842950f
add handling for gevent too
sentrivana Nov 2, 2023
3dcec2a
more tests
sentrivana Nov 2, 2023
d4302a4
also set thread to none
sentrivana Nov 2, 2023
02efaca
Merge branch 'master' into antonpirker/handle-creation-of-threads-dur…
sentrivana Nov 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sentry_sdk/metrics.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -348,7 +348,7 @@ def _ensure_thread(self):
try:
self._flusher.start()
except RuntimeError:
# Unfortunately at this point the interpreter is in a start that no
# Unfortunately at this point the interpreter is in a state that no
# longer allows us to spawn a thread and we have to bail.
self._running = False
return False
Expand Down
9 changes: 8 additions & 1 deletion sentry_sdk/monitor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,14 @@ def _thread():

thread = Thread(name=self.name, target=_thread)
thread.daemon = True
thread.start()
try:
thread.start()
except RuntimeError:
# Unfortunately at this point the interpreter is in a state that no
# longer allows us to spawn a thread and we have to bail.
self._running = False
return None
Comment thread
sentrivana marked this conversation as resolved.

self._thread = thread
self._thread_for_pid = os.getpid()

Expand Down
8 changes: 7 additions & 1 deletion sentry_sdk/profiler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -918,7 +918,13 @@ def ensure_running(self):
# can keep the application running after other threads
# have exited
self.thread = threading.Thread(name=self.name, target=self.run, daemon=True)
self.thread.start()
try:
self.thread.start()
except RuntimeError:
# Unfortunately at this point the interpreter is in a state that no
# longer allows us to spawn a thread and we have to bail.
self.running = False
return
Comment on lines +926 to +936
Copy link
Copy Markdown
Member

@szokeasaurusrex szokeasaurusrex Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find it slightly confusing here that the method is called ensure_running, but it is possible for the method to terminate without the thread running. We should have some way to indicate failure, perhaps by raising a custom exception when ensure_running cannot start the scheduler or by returning a value to indicate success/failure. We might instead or additionally consider renaming the function to indicate that we may fail to ensure that the scheduler is running and/or add a documentation comment.

If we make this change to indicate when the ensure_running fails, we should also make sure to handle failure appropriately in the calling code.

Copy link
Copy Markdown
Contributor

@sentrivana sentrivana Nov 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szokeasaurusrex Added docstrings to the three ensure_running functions.

I considered making the function return a bool, but in all three cases it already communicates success/failure by setting the self.(_)running variable, so it feels a bit redundant to make it return essentially the same information, especially since the calling places usually can't do much with the return value -- all code that depends on ensure_running being run checks the value of self.(_)running instead anyway. Which feels like an ok pattern to me.

Regarding the other points:

  • added some tests
  • added the same logic to the gevent scheduler (I checked and it would also attempt to spawn a thread)


def run(self):
# type: () -> None
Expand Down
10 changes: 9 additions & 1 deletion sentry_sdk/sessions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,9 +120,17 @@ def _thread():

thread = Thread(target=_thread)
thread.daemon = True
thread.start()
try:
thread.start()
except RuntimeError:
# Unfortunately at this point the interpreter is in a state that no
# longer allows us to spawn a thread and we have to bail.
self._running = False
return None
Comment thread
sentrivana marked this conversation as resolved.

self._thread = thread
self._thread_for_pid = os.getpid()

return None

def add_aggregate_session(
Expand Down