cc docs: clarify task cancellation behavior

Clarified the behavior of @ref engine::TaskBase::State, @ref engine::TaskBase::IsValid and cancellation propagation.
commit_hash:a3c45aee640d0d431e4c82bea088a113c4578c72

Author: Anton3

core/include/userver/engine/task/task_base.hpp

@@ -51,8 +51,34 @@ class [[nodiscard]] TaskBase {
         kQueued,     ///< awaits execution
         kRunning,    ///< executing user code
         kSuspended,  ///< suspended, e.g. waiting for blocking call to complete
-        kCancelled,  ///< exited user code because of external request
-        kCompleted,  ///< exited user code with return or throw
+
+        /// The task is cancelled and was finished without returning a value or throwing a user-provided exception.
+        ///
+        /// This can happen for two reasons:
+        ///
+        /// 1. When a non-critical task (see @ref Importance::kCritical, see @ref flavors_of_async) is cancelled before
+        ///    it starts running, the task functor is skipped, only destructor is executed. This can be interpreted
+        ///    as every non-critical task having an implicit cancellation point at its start.
+        ///    See more details and examples in @ref task_cancellation_before_start.
+        ///
+        /// 2. When a task is cancelled because of a call to @ref engine::current_task::CancellationPoint.
+        ///    This cancellation is implemented using a non-`std::exception`-based exception.
+        ///
+        /// In both cases, @ref engine::TaskWithResult::Get throws @ref engine::TaskCancelledException.
+        ///
+        /// Unintuitively, this status is not set when the task was cancelled after it started running,
+        /// which caused the user code to exit early.
+        ///
+        /// Use @ref TaskBase::CancellationReason instead to check whether the task was cancelled.
+        kCancelled,
+
+        /// Exited user code with return or throw.
+        ///
+        /// This includes cases where the task was cancelled after it started running,
+        /// which caused the user code to exit early.
+        ///
+        /// Use @ref TaskBase::IsFinished instead to check whether the task finished execution.
+        kCompleted,
     };
 
     /// Task wait mode
@@ -63,15 +89,19 @@ class [[nodiscard]] TaskBase {
         kMultipleAwaiters
     };
 
-    /// @brief Checks whether this object owns
-    /// an actual task (not `State::kInvalid`)
+    /// @brief Checks whether this object owns an actual task (not @ref State::kInvalid)
     ///
-    /// An invalid task cannot be used. The task becomes invalid
-    /// after each of the following calls:
+    /// An invalid task cannot be used. The task becomes invalid after each of the following calls:
     ///
     /// 1. the default constructor
-    /// 2. `Detach()`
-    /// 3. `Get()` (see `engine::TaskWithResult`)
+    /// 2. moving from this task object
+    /// 3. @ref engine::TaskWithResult::Get
+    /// 4. @ref concurrent::BackgroundTaskStorageCore::Detach
+    /// 5. @ref engine::DetachUnscopedUnsafe
+    ///
+    /// Notably, the task does *not* become invalid immediately after it finishes execution.
+    /// (That would always cause race conditions when trying to await a task.)
+    /// It means that some of the task's resources are held onto until the task object is invalidated or destroyed.
     bool IsValid() const;
 
     /// Gets the task State

scripts/docs/en/userver/intro.md

@@ -49,7 +49,7 @@ low-latency, then it may be fine to run all the code on the same task processor.
 ______
 ⚠️🐙❗ If you want to run code that uses standard synchronization primitives
 (for example, code from a third-party library), then this code should be run in
-a separate `engine::TaskProcessor` to avoid starvation of main task processors.
+a separate @ref engine::TaskProcessor to avoid starvation of main task processors.
 See @ref scripts/docs/en/userver/task_processors_guide.md for more info.
 ______
 
@@ -95,21 +95,21 @@ By engine::TaskProcessor:
 
 By shared-ness:
 
-* By default, functions return engine::TaskWithResult, which can be awaited
+* By default, functions return @ref engine::TaskWithResult, which can be awaited
   from 1 task at once. This is a reasonable choice for most cases.
 * Functions from `utils::Shared*Async*` and `engine::Shared*AsyncNoSpan`
-  families return engine::SharedTaskWithResult, which can be awaited
+  families return @ref engine::SharedTaskWithResult, which can be awaited
   from multiple tasks at the same time, at the cost of some overhead.
 
-By engine::TaskBase::Importance ("critical-ness"):
+By @ref engine::TaskBase::Importance ("critical-ness"):
 
-* By default, functions can be cancelled due to engine::TaskProcessor
+* By default, functions can be cancelled due to @ref engine::TaskProcessor
   overload. Also, if the task is cancelled before being started, it will not
   run at all.
 * If the whole service's health (not just one request) depends on the task
   being run, then functions from `utils::*CriticalAsync*` and
   `engine::*CriticalAsyncNoSpan*` families can be used. There, execution of
-  the function is guaranteed to start regardless of engine::TaskProcessor
+  the function is guaranteed to start regardless of @ref engine::TaskProcessor
   load limits
 
 By tracing::Span:
@@ -128,7 +128,7 @@ By tracing::Span:
       But beware! Using tracing::Span::CurrentSpan() will trigger asserts
       and lead to UB in production.
 
-By the propagation of engine::TaskInheritedVariable instances:
+By the propagation of @ref engine::TaskInheritedVariable instances:
 
 * Functions from `utils::*Async*` family (which you should use by default)
   inherit all task-inherited variables from the parent task.
@@ -270,76 +270,87 @@ Cancellation can occur:
 
   * by an explicit request;
   * due to the end of the task object lifetime;
-  * at coroutine engine shutdown (affects tasks launched via `engine::Task::Detach`);
+  * at coroutine engine shutdown (affects tasks launched via @ref engine::DetachUnscopedUnsafe);
   * due to the lack of resources.
 
-To cancel a task explicitly, call the `engine::TaskBase::RequestCancel()` or `engine::TaskBase::SyncCancel()` method. It cancels only a single task and does not directly affect the subtasks that were created by the canceled task.
+To cancel a task explicitly, call the @ref engine::TaskBase::RequestCancel or @ref engine::TaskBase::SyncCancel method.
+For details on what happens with subtasks when a task is cancelled, see @ref task_cancellation_outside_world.
 
-Another way to cancel a task it to drop the `engine::TaskWithResult` without awaiting it, e.g. by returning from the function that stored it in a local variable or by letting an exception fly out.
+Another way to cancel a task it to drop the @ref engine::TaskWithResult without awaiting it, e.g. by returning from the function that stored it in a local variable or by letting an exception fly out.
 
 @snippet core/src/engine/task/cancel_test.cpp stack unwinding destroys task
 
-Tasks can be cancelled due to `engine::TaskProcessor` overload, if configured. This is a last-ditch effort to avoid OOM due to a spam of tasks. Read more in `utils::Async` and `engine::TaskBase::Importance`. Tasks started with `engine::CriticalAsync` are excepted from cancellations due to `TaskProcessor` overload.
+In the example above, `child_task` is cancelled and awaited due to stack unwinding. In any case, the child task's
+execution is guaranteed to be finished once its @ref engine::TaskWithResult handle is destroyed.
+
+Tasks can be cancelled due to @ref engine::TaskProcessor overload, if configured. This is a last-ditch effort to avoid OOM due to a spam of tasks. Read more in @ref utils::Async and @ref engine::TaskBase::Importance. Tasks started with @ref engine::CriticalAsync are excepted from cancellations due to `TaskProcessor` overload.
 
 ### How the task sees its cancellation
 
 Unlike C++20 coroutines, userver does not have a magical way to kill a task. The cancellation will somehow be signaled to the synchronization primitive being waited on, then it will go through the logic of the user's function, then the function will somehow complete.
 
 How some synchronization primitives react to cancellations:
 
-  * `engine::TaskWithResult::Get` and `engine::TaskBase::Wait` throw `engine::WaitInterruptedException`, which typically leads to the destruction of the child task during stack unwinding, cancelling and awaiting it;
-  * `engine::ConditionVariable::Wait` and `engine::Future::wait` return a status code;
-  * `engine::SingleConsumerEvent::WaitForEvent` returns `false`;
-  * `engine::SingleConsumerEvent::WaitForEventFor` returns `false` and needs an additional `engine::current_task::ShouldCancel()` check;
-  * `engine::InterruptibleSleepFor` needs an additional `engine::current_task::ShouldCancel()` check;
-  * `engine::CancellableSemaphore` returns `false` or throws `engine::SemaphoreLockCancelledError`.
+  * @ref engine::TaskWithResult::Get and @ref engine::TaskBase::Wait throw @ref engine::WaitInterruptedException, which typically leads to the destruction of the child task during stack unwinding, cancelling and awaiting it;
+  * @ref engine::ConditionVariable::Wait and @ref engine::Future::wait return a status code;
+  * @ref engine::SingleConsumerEvent::WaitForEvent returns `false`;
+  * @ref engine::SingleConsumerEvent::WaitForEventFor returns `false` and needs an additional @ref engine::current_task::ShouldCancel check;
+  * @ref engine::InterruptibleSleepFor needs an additional @ref engine::current_task::ShouldCancel check;
+  * @ref engine::CancellableSemaphore returns `false` or throws engine::SemaphoreLockCancelledError.
 
 Some synchronization primitives deliberately ignore cancellations, notably:
 
-  * `engine::Mutex`;
-  * `engine::Semaphore` (use `engine::CancellableSemaphore` to support cancellations);
-  * `engine::SleepFor` (use `engine::InterruptibleSleepFor` to support cancellations).
+  * @ref engine::Mutex;
+  * @ref engine::Semaphore (use @ref engine::CancellableSemaphore to support cancellations);
+  * @ref engine::SleepFor (use @ref engine::InterruptibleSleepFor to support cancellations).
 
-Most clients throw a client-specific exception on cancellation. Please explore the docs of the client you are using to find out how it reacts to cancellations. Typically, there is a special exception type thrown in case of cancellations, e.g. `clients::http::CancelException`.
+Most clients throw a client-specific exception on cancellation. Please explore the docs of the client you are using to find out how it reacts to cancellations. Typically, there is a special exception type thrown in case of cancellations, e.g. @ref clients::http::CancelException.
 
+@anchor task_cancellation_outside_world
 ### How the outside world sees the task's cancellation
 
-The general theme is that a task's completion upon cancellation is still a completion. The task's function will ultimately return or throw something, and that is what the parent task will receive in `engine::TaskWithResult::Get` or `engine::TaskBase::Wait`.
+The general theme is that a task's completion upon cancellation is still a completion. The task's function will ultimately return or throw something, and that is what the parent task will receive in @ref engine::TaskWithResult::Get or @ref engine::TaskBase::Wait.
 
-If the cancellation is due to the parent task being cancelled, then its `engine::TaskWithResult::Get` or `engine::TaskBase::Wait` will throw an `engine::WaitInterruptedException`, leaving the child task running (for now), so the parent task will likely not have a chance to observe the child task's completion status. Usually the stack unwinding in the parent task then destroys the `engine::Task` handle, which causes it to be cancelled and awaited.
+If the cancellation is due to the parent task being cancelled, then its @ref engine::TaskWithResult::Get or @ref engine::TaskBase::Wait will throw an @ref engine::WaitInterruptedException, leaving the child task running (for now), so the parent task will likely not have a chance to observe the child task's completion status. Usually the stack unwinding in the parent task then destroys the @ref engine::TaskWithResult handle, which causes it to be cancelled and awaited.
 
 @snippet core/src/engine/task/cancel_test.cpp parent cancelled
 
+In the example above, `child_task` is cancelled and awaited due to stack unwinding. In any case, the child task's
+execution is guaranteed to be finished once its @ref engine::TaskWithResult handle is destroyed.
+
 If the child task got cancelled without the parent being cancelled, then:
 
-  * `engine::TaskWithResult::Get` will return or throw whatever the child task has returned or thrown, which is practically meaningless (because why else would someone cancel a task?);
-  * `engine::TaskBase::Wait` will return upon completion;
-  * `engine::TaskBase::IsFinished` will return `true` upon completion;
-  * `engine::TaskBase::GetStatus` will return `engine::TaskBase::Status::kCancelled` upon completion.
+  * @ref engine::TaskWithResult::Get will return or throw whatever the child task has returned or thrown, which is practically meaningless (because why else would someone cancel a task?);
+  * @ref engine::TaskBase::Wait will return upon completion;
+  * @ref engine::TaskBase::IsFinished will return `true` upon completion;
+  * @ref engine::TaskBase::GetStatus will return @ref engine::TaskBase::Status::kCancelled upon completion.
+
+@anchor task_cancellation_before_start
+### What happens to tasks that are cancelled before they start running
 
-There is one extra quirk: if the task is cancelled before being started, then only the functor's destructor will be run by default. See details in `utils::Async`. In this case `engine::TaskWithResult::Get` will throw `engine::TaskCancelledException`.
+There is one extra quirk: if the task is cancelled before being started, then only the functor's destructor will be run by default. See details in @ref utils::Async. In this case @ref engine::TaskWithResult::Get will throw @ref engine::TaskCancelledException.
 
-Tasks launched via `utils::CriticalAsync` are always started, even if cancelled before entering the function. The cancellation will take effect immediately after the function starts:
+Tasks launched via @ref utils::CriticalAsync are always started, even if cancelled before entering the function. The cancellation will take effect immediately after the function starts:
 
 @snippet core/src/engine/task/cancel_test.cpp critical cancel
 
 ### Lifetime of a cancelled task
 
-Note that the destructor of `engine::Task` cancels and waits for task to finish if the task has not finished yet. Use `concurrent::BackgroundTaskStorage` to continue task execution out of scope.
+Note that the destructor of @ref engine::Task cancels and waits for task to finish if the task has not finished yet. Use @ref concurrent::BackgroundTaskStorage to continue task execution out of scope.
 
-The invariant that the task only runs within the lifetime of the `engine::Task` handle or `concurrent::BackgroundTaskStorage` is the backbone of structured concurrency in userver, see `utils::Async` and `concurrent::BackgroundTaskStorage` for details.
+The invariant that the task only runs within the lifetime of the @ref engine::Task handle or @ref concurrent::BackgroundTaskStorage is the backbone of structured concurrency in userver, see @ref utils::Async and @ref concurrent::BackgroundTaskStorage for details.
 
 ### Utilities that interact with cancellations
 
 The user is provided with several mechanisms to control the behavior of the application in case of cancellation:
 
-  * `engine::current_task::CancellationPoint()` -- if the task is canceled, calling this function throws an exception that is not caught during normal exception handling (not inherited from `std::exception`). This will result in stack unwinding with normal destructor calls for all local objects. The parent task will receive `engine::TaskCancelledException` from `engine::TaskWithResult::Get`.
+  * @ref engine::current_task::CancellationPoint -- if the task is canceled, calling this function throws an exception that is not caught during normal exception handling (not inherited from `std::exception`). This will result in stack unwinding with normal destructor calls for all local objects. The parent task will receive @ref engine::TaskCancelledException from @ref engine::TaskWithResult::Get.
   **⚠️🐙❗ Catching this exception results in UB, your code should not have `catch (...)` without `throw;` in the handler body**!
-  * `engine::current_task::ShouldCancel()` and `engine::current_task::IsCancelRequested()` -- predicates that return `true` if the task is canceled:
-    * by default, use `engine::current_task::ShouldCancel()`. It reports that a cancellation was requested for the task and the cancellation was not blocked (see below);
-    * `engine::current_task::IsCancelRequested()` notifies that the task was canceled even if cancellation was blocked; effectively ignoring caller's requests to complete the task regardless of cancellation.
-  * `engine::TaskCancellationBlocker` -- scope guard, preventing cancellation in the current task. As long as it is alive all the blocking calls are not interrupted, `engine::current_task::CancellationPoint` throws no exceptions, `engine::current_task::ShouldCancel` returns `false`.
-    **⚠️🐙❗ Disabling cancellation does not affect the return value of `engine::current_task::IsCancelRequested()`.**
+  * @ref engine::current_task::ShouldCancel and @ref engine::current_task::IsCancelRequested -- predicates that return `true` if the task is canceled:
+    * by default, use @ref engine::current_task::ShouldCancel. It reports that a cancellation was requested for the task and the cancellation was not blocked (see below);
+    * @ref engine::current_task::IsCancelRequested notifies that the task was canceled even if cancellation was blocked; effectively ignoring caller's requests to complete the task regardless of cancellation.
+  * @ref engine::TaskCancellationBlocker -- scope guard, preventing cancellation in the current task. As long as it is alive all the blocking calls are not interrupted, @ref engine::current_task::CancellationPoint throws no exceptions, @ref engine::current_task::ShouldCancel returns `false`.
+    **⚠️🐙❗ Disabling cancellation does not affect the return value of @ref engine::current_task::IsCancelRequested.**
 
 
 ----------