Does a const-qualified member function promise not to modify the object? How is this legal?

constcast conversion - cppreference:

const_cast makes it possible to form a reference or pointer to non-const type that is actually referring to a const object or a reference or pointer to non-volatile type that is actually referring to a volatile object. Modifying a const object through a non-const access path and referring to a volatile object through a non-volatile glvalue results in undefined behavior.

_{[Emphasis mine]}

This means that your code as currently written may or may not be undefined behavior, depending on how you actually use (call) it.

LazyValue foo{};
foo.value(); // ok
const LazyValue& bar = foo;
bar.value(); // ok, reference refers to non-const foo
const LazyValue baz{};
baz.value(); // UB, casting away const-ness

As already mentioned in your posts comments, there exists the mutable keyword which allows you to modify members through a const method.

mutable is used to specify that the member does not affect the externally visible state of the class (as often used for mutexes, memo caches, lazy evaluation, and access instrumentation).

_{[Emphasis mine]}