C thread deadlock debug regarding a thread synchronization assignment

This constraint ...

• The main thread can't end before all other threads end

... is straightforward. The main thread just needs to join all the others (which presumably it also created) before it terminates.

This constraint ...

• At a given moment, at most 6 threads can be running

... is also fairly straightforward. One approach would be to initialize a counting semaphore with a count of 6, then ensure that each thread decrements it before entering the "running" state, and increments it again after leaving the the running state. There are other alternatives.

This constraint ...

• thread 10 can end only when 6 others threads are running (including itself)

... is the tricky one. It requires you to maintain a count of the number of threads currently running, at least until thread 10 has terminated, to reliably build up a stable of exactly five other threads, by preventing that many from terminating until thread 10 has done, and then letting all running threads proceed.

Your approach seems to be to reserve a special permit to run for thread 10, plus five more for the other threads to share. You allow threads to take permits to run from the appropriate pool on a first-come, first-served basis, blocking thread 10 from terminating until all five other permits have been taken, and blocking all other permitted threads from terminating until thread 10 has terminated.

That's workable, though reserving a permit exclusively for thread 10 does limit your throughput more than need be, and requiring that it terminate before any of the others is a bit inefficient. You have some implementation problems, however:

• most importantly, the semwait(&semt2) performed by the "kidnapped" threads is wrong. It will prevent any of them from allowing thread 10 to proceed to termination, which is necessary for thread 10 to release them, so there's a definite deadlock. The latter wait on semkidnap is the only one you need here.

• also significantly, your handling of variable p2tcount is not thread safe. You have data races there, giving your program undefined behavior. Making that variable atomic would fix the data races, but that is neither necessary nor sufficient, because you also have a higher-level race condition between threads testing p2tcount < 5 and other threads incrementing p2tcount. This can also deadlock you even after the previous issue is fixed, because you could get more than five threads waiting for semkidnap.

• your threadt210 should not have to signal two semaphores five times each to let other threads proceed. I think the idea must have been that the kidnapped threads would each receive one of each of those, but that can't work as you intended because the kidnapped threads must proceed into the running state before thread 10 can let them go, but when it then does so, it needs to avoid letting any of the other waiting threads go instead, lest you end up with more than six threads running at once. Sticking close to your current design, thread 10 would post only to semkidnap (five times), and let the kidnapped threads post to semt2 after they terminate to let other threads run.

Overall, your threadbarrier might work better like this:

// initialize semaphore counts as:
// semmutex 1
// semt210 0
// semt2 0
// semkidnap 0
void thread_barrier(void arg) {
    struct threadarg *threadinfo = (struct threadarg *) arg;
    // with p2tcount now being accessed only once, making it atomic
    // would do away with the need for semmutex
    semwait(&semmutex); // begin critical region for access to p2tcount
    int myindex = p2tcount++;
    sempost(&semmutex); // end critical region
    if (myindex < 5) {
        // get 5 threads running and keep them that way until t10 exits
        info(BEGIN, threadinfo->processn, threadinfo->threadn);
        sempost(&semt210); // unlock t10 once (5 times needed)
        semwait(&semkidnap);
    } else {
        // 34 potential other threads will wait here
        semwait(&semt2);
        info(BEGIN, threadinfo->processn, threadinfo->threadn);
    }
    // All threads other than t10 pass through here
    info(END, threadinfo->processn, threadinfo->threadn);
    sempost(&semt2);
    return NULL;
}

The corresponding other thread might look more like this:

void thread_t210(void arg){
    info(BEGIN, 2, 10);
    // wait for five threads to signal they have started
    for (int i = 0; i < 5; i++){
        semwait(&semt210);
    }
    info(END, 2, 10);
    // unblock the kidnapped threads
    for(int i = 0; i < 5; i++){
        sempost(&semkidnap);
    }
    return NULL;
}

That does add a fourth(!) semaphore, which overall is at least one more than you really need.

---

If it were me, however, I would design it a bit differently. I would use a counting semaphore initialized with value 6 to enforce the running thread limit, and I would implement this termination logic: a running thread T is permitted to terminate if and only if one of the following is true:

• thread 10 has already terminated
• there are five other threads running and none of them is thread 10 (T itself might or might not be thread 10)

You need to serialize evaluation of the latter condition, in the sense that once you determine that it allows T to terminate, you must not test it for any other thread until T actually has terminated.

Implementation left as an exercise, noting that it is somewhat awkward to implement this approach without condition variables.

I know you have decided to use semaphores (although you indicated in the comments that you are not averse to mutexes and therefore condition variables), but here is an example using a global thread counter, a mutex and a condition variable. I find the approach with mutexes and condition variables much more intuitive, although a semaphore basically does the same (similar) thing under the hood (in particular, you have the freedom to define the critical sections in which an exclusive right to global or shared data is granted).

#include 
pthreadmutext mtx = PTHREADMUTEXINITIALIZER;
pthreadcondt cnd  = PTHREADCONDINITIALIZER;
int numthreads = 0; //global thread counter (number of running threads)
void* threadproc(void arg)
{
    int idx = ((int*) arg);
    //enter critical section
    pthreadmutexlock(&mtx);
        //test if the threshold for the maximum number of threads 
        //has been reached
        while (!(numthreads < 6)) { //spurious wakeups can occur, don't use 'if'
            //if so, wait until signaled by another thread
            pthreadcondwait(&cnd, &mtx); //unlocks mutex
        }
        //increase the thread count
        ++numthreads;
    //leave critical section
    pthreadmutexunlock(&mtx);
    //do something
    //enter critical section
    pthreadmutexlock(&mtx);
        //test if this thread is the special one
        if (idx == 10) {
            //if so, ensure that enough threads are running
            //(at least 6, including this one, must be running)
            while (numthreads < 6) { //5 + 1 (self) == 6 total threads
                //if not, wait until signaled by another thread
                pthreadcondwait(&cnd, &mtx); //unlocks mutex
            }
        }
        //decrease the thread count
        --numthreads;
    //leave critical section
    pthreadmutexunlock(&mtx);
    //signal another waiting thread
    pthreadcondsignal(&cnd);
    return NULL;
}
int main()
{
    struct threadinfot {
        pthreadt id; //native thread handle
        int idx;      //thread index (required for special case)
    } threadlst[40];
    //ensure all threads starts waiting
    pthreadmutexlock(&mtx);
    //create all threads
    for (int i=0; i < 40; ++i) {
        threadlst[i].idx = i;
        pthreadcreate(
            &threadlst[i].id,
            NULL,
            &threadproc,
            &threadlst[i].idx
        );
    }
    //release lock
    pthreadmutexunlock(&mtx);
    //wait for all threads to terminate
    for (int i=0; i < 40; ++i) {
        pthreadjoin(thread_lst[i].id, NULL);
    }
}