How can I require a delegate/lambda to be static?

and then I also need to ensure that asyncDelegate is free from side-effects.

In general case you can't do that because even if the delegate is a static one and it operates on truly immutable data it still can do anything like modifying global state. Unless you are willing to go down the analyzing expression trees road (i.e. accept Expression and visiting it in search of side-effects) then there is not much you can do with stuff like:

static class MyActionHolder
{
    private static int Counter;
    public static void Act(P p, R r)
    {
        Counter++; // modify global state in non-thread-safe manner
        // or any other side-effect like
        // Console.WriteLine($"{p}: {r}"); 
    }
}

Rider had an attribute to mark methods as Pure - PureAttribute, but not sure how it was enforcing it and how you can enforce it on the action.

to process an instance of P and produce an instance of R

Should not it be Func then, not Action?

A compile-time solution would be great

If you want to ensure only the static part, then you can look into writing your own Roslyn code analyzer and check in it if the passed delegate either marked with static or is invocation of a static method (will try to make a demo for this on weekend).