Is std::map::operator[] thread-safe if it doesn't insert?

First, let’s reiterate that unsafe multithreaded access to C++ standard library containers is undefined behavior. Simultaneous read operations are safe for multithreading, but simultaneous writes or a read and write occurring simultaneously are unsafe for multithreading.

Why does unsafe multithreaded use of an std::unordered_map crash more often than unsafe multithreaded use of a std::map?

Additionally, do not use operator[] on std::(unordered)map. It is the most convenient syntax, but is not often what you actually want.
The std::map subscript operator is a convenience, but a potentially dangerous one
The operations for reading and writing single elements for C++ standard library maps
A simplified overview of ways to add or update elements in a std::map

The simplest way to ensure safety is to have all threads refer to the object using a const& like std::asconst, since const methods are guaranteed to be thread safe (or at least I'm not aware of any counterexamples). This of course only refers to the method itself, not what you do with the return, e.g. std::vector::at(sizet) const is safe to call, but not safe to assign to dereferenced pointer.

If you try to do that with std::undordered_map::operator[], you will see the code simply does not compile, because it is not marked const. You can consider it a counterintuitive defect in the standard that it inserts a default-constructed element if none is found, but that is what we are stuck with and the best we can do is avoid it unless that is exactly what we need (e.g. to implement a counter).

Even though you might think that "logically" the operator should be "effectively const" if you know the value is already there, the standard poses no such restriction. Compiler implementers try to squeeze out every bit of performance out of the letter of the standard, and it would be perfectly legal for them to use this call to e.g. to rehash all the data to remove tombstones.

Just stick with at() const - it does exactly what you want.