Answer to: How to choose between brute force and efficient solution that has overhead?

In practice you have to implement both algorithms and measure in order to determine the "break even"-point. In theory it could be calculated if you know the cost of each individual operation, but this gets incredibly complex. For example some operations become slower if the dataset exceed the size of a memory page. The reason big-O analysis is feasible is because we ignore the cost of individual operations and just look at the growth in number of operations. The recommended approach is to always implement the simplest algorithm first and then only optimize if performance turns out to be a problem with realistic input. It might never be an issue. The fancy solution is to support both algorithms, and select the optimal one based on the input size. But you should only do this if it gives a measurable real-world benefit, since it gives you twice as much code to debug and maintain.