[enhancement] crypto: Add buffer-based overloads for zero-allocation operations

[cherninkiy]

Add a description

Description

The current cryptographic API in userver::crypto (e.g., crypto::PrivateKey::GetPemString, crypto::CmsSigner::Sign) heavily uses std::string for both input and output data. While this approach is simple and convenient, it results in unpredictable heap allocations on every call.

This creates serious problems for high‑load microservices built on userver, especially with the advent of post‑quantum cryptography (PQC), where key and signature sizes can reach tens of kilobytes. Frequent allocations on the hot path of signing/verification operations can lead to unpredictable latency, increased allocator pressure, and memory fragmentation — contradicting userver’s philosophy of writing predictable and efficient asynchronous applications.

Concrete example

The current GetPemString() method returns std::optional<std::string>. Even for a single PEM blob (e.g., a large PQC private key), this causes at least one heap allocation per call. Under thousands of requests per second, this becomes a bottleneck.

Proposed solution

Introduce a set of overloaded methods that take a pre‑allocated output buffer via std::span<std::byte> (or std::span<char>). This allows users to manage memory themselves, reuse buffers, and completely eliminate dynamic allocation overhead on the hot path.

API changes

1. PrivateKey

   • Existing method:

     std::optional<std::string> GetPemString(std::string_view password) const;

   • New overload:

     bool GetPemString(std::span<char> out_buffer, std::string_view password, size_t& out_len) const;

   Returns true on success and writes the exact output size to out_len. If the buffer is too small, it returns false (or optionally writes the required size to out_len without copying).

2. crypto:: Certificate, crypto::ComputeHash (hash.cpp) and crypto::ComputeHmac (signers.cpp) – similar overloads accepting std::span<char> for output.

3. crypto::CmsSigner – an overload for Sign that takes an output buffer.

Usage example

// Old (allocation):
std::optional<std::string> pem_opt = private_key.GetPemString("pass");
if (pem_opt) {
    // use pem_opt.value()
}

// New (zero‑allocation if buffer is reused):
std::vector<char> buffer(4096); // reusable buffer
size_t written = 0;
if (private_key.GetPemString(buffer, "pass", written)) {
    std::string_view pem(buffer.data(), written);
    // use pem
}

Migration plan (stranger pattern)

To keep the change safe and backward‑compatible, we propose a gradual migration:

1. Add the new overloads while keeping the old methods unchanged.
2. Deprecate the old methods in a future major release (add [[deprecated]] with a message pointing to the new API).
3. Remove the old methods after one or two major releases, giving the community enough time to migrate.

Expected benefits

· Performance: zero heap allocations on the hot crypto path.
· Predictability: latency is not affected by allocator contention.
· Future‑proof: ready for large PQC keys and signatures without hidden overhead.