[問題] 用pthread_cond_wait來切換thread

[wtchen (沒有存在感的人)]

開發平台(Platform): (Ex: VC++, GCC, Linux, ...) 
Linux + Raspberry Pi 1 + gcc 4.8

額外使用到的函數庫(Library Used): (Ex: OpenGL, ...) 
pthread, sys/mman

問題(Question)：
我希望用有效的thread管理來避免不必要的context switch
int main()裏面有個loop是每loop一次 sleep 5ms
在這5ms裏面可以用來處理被喚醒的thread: pid_1~4
程式最後會顯示每個loop耗費的時間 (完美結果會是5000，單位us)

(為了把scheduling的影響減到最小，已經把priority設到最高，也做了mlockall)

餵入的資料(Input)：


預期的正確結果(Expected Output)：
我輸出的結果每個loop時間為5200-5700us不等
請問還有無再優化的可能？



程式碼(Code)：(請善用置底文網頁, 記得排版) 

好讀程式碼：
https://gist.github.com/gnitnaw/6129098182bfd1f7c607

或以下：
#include <stdio.h>
#include <string.h>
#include <pthread.h>
#include <time.h>
#include <unistd.h>
#include <sys/mman.h>

#define NLOOP   30
int global_thread = 0;
pthread_mutex_t mutex;
pthread_cond_t cond[5];

void _usleep(int micro)
{
    struct timespec req = {0};
    req.tv_sec = 0;
    req.tv_nsec = micro * 1000L;
    nanosleep(&req, (struct timespec *)NULL);
}

void* Thread(void* x) {
    int i;
    int *X = (int*) x;
    for (i=0; i<5; ++i) {
        pthread_mutex_lock(&mutex);
        printf("This thread should sleep %d000 microsecond\n", *X);
        while (global_thread != *X) {
            printf("Thread %d wait\n", *X);
            pthread_cond_wait(&cond[*X], &mutex);
        }
        printf("Thread %d Enter!\n", *X);
        global_thread = 0;
        pthread_mutex_unlock(&mutex);
        _usleep(*X * 1000);
    }
    printf("Thread %d Exit! \n", *X);
    pthread_exit(NULL);
}

int main(void) {

    struct sched_param sp;
    memset(&sp, 0, sizeof(sp));
    sp.sched_priority = sched_get_priority_max(SCHED_FIFO);
    //sp.sched_priority = 49;
    sched_setscheduler(0, SCHED_FIFO, &sp);

    mlockall(MCL_CURRENT | MCL_FUTURE);

    int i;

    int a=1, b=2, c=3, d=4;
    float dt[NLOOP];
    struct timespec tp1, tp2;
    unsigned long startTime, procesTime;
    pthread_t pid_1, pid_2, pid_3, pid_4;

    for (i=0; i<5; ++i) {
        pthread_cond_init(&cond[i],NULL);
    }
    pthread_mutex_init(&mutex,NULL);

    pthread_create(&pid_1,NULL,Thread, (void*)&a);
    pthread_create(&pid_2,NULL,Thread, (void*)&b);
    pthread_create(&pid_3,NULL,Thread, (void*)&c);
    pthread_create(&pid_4,NULL,Thread, (void*)&d);

    sleep(5);
    puts("============================================");
    clock_gettime(CLOCK_REALTIME, &tp1);
    startTime = tp1.tv_sec*1000000000 + tp1.tv_nsec;
    for (i=0; i<NLOOP; ++i) {
        puts("");
        printf("LOOP %d\n", i);
        global_thread = i%5;
        if (global_thread <5) pthread_cond_signal(&cond[global_thread]);
        _usleep(5000);
        clock_gettime(CLOCK_REALTIME, &tp2);
        startTime = tp1.tv_sec*1000000000 + tp1.tv_nsec;
        procesTime = tp2.tv_sec*1000000000 + tp2.tv_nsec - startTime;
        dt[i] = (float)procesTime /1000.0;
        tp1 = tp2;
    }

    pthread_join(pid_1, NULL);
    pthread_join(pid_2, NULL);
    pthread_join(pid_3, NULL);
    pthread_join(pid_4, NULL);

    for (i=0; i<NLOOP; ++i) {
        printf("%d, %f\n", i, dt[i]);
    }

    return 0;

}

補充說明(Supplement)：


--
※ 發信站: 批踢踢實業坊(ptt.cc), 來自: 86.200.210.189
※ 文章網址: https://www.ptt.cc/bbs/C_and_CPP/M.1446561206.A.F56.html
推 LiloHuang: RPi 我不熟，也許你可以試試看用 select 來暫停 5ms
→ LiloHuang: 大概像是 select(0, NULL, NULL, NULL, &timeout);
→ wtchen: 我聽說nanosleep是高精度timer所以才用
→ wtchen: 至於select我還真的不熟
→ LiloHuang: select() 根據我的經驗可靠度比較好，你還是實驗看看
→ LiloHuang: 你的 gcc 版本還滿新的，也可以用 C++11 的 sleep_for
推 wa120: 有可能會因為global_thread=0-4的時候
→ wa120: 剛好執行到Thread global_thread=0被蓋過去
→ wtchen: 被蓋掉正常，我是故意這樣搞
我要的效果就是這樣：(以下為部份執行結果)

LOOP 0                                  // 不管其他thread

LOOP 1
Thread 1 Enter!                         // pid_1醒來
This thread should sleep 1000 microsecond
Thread 1 wait                           // pid_1睡

LOOP 2
Thread 2 Enter!                         // pid_2醒
This thread should sleep 2000 microsecond
Thread 2 wait                           // pid_2睡

LOOP 3
Thread 3 Enter!                         // pid_3醒
This thread should sleep 3000 microsecond
Thread 3 wait                           // pid_3睡

LOOP 4
Thread 4 Enter!                         // pid_4醒
This thread should sleep 4000 microsecond
Thread 4 wait                           // pid_4睡


最後顯示每個loop用掉的時間:

0, 5137.000000
1, 5192.000000
2, 5180.000000
3, 5200.000000
4, 5366.000000
5, 5173.000000
6, 5462.000000
7, 5198.000000
8, 5193.000000
9, 5198.000000
10, 5170.000000
11, 5240.000000
12, 5203.000000
13, 5199.000000
14, 5285.000000
15, 5174.000000
16, 5200.000000
17, 5429.000000
18, 5201.000000
19, 5339.000000
20, 5243.000000
21, 5203.000000
22, 5403.000000
23, 5638.000000
24, 5225.000000
25, 5184.000000         // 所有的thread已經結束，不需要再context switch
26, 5183.000000
27, 5171.000000
28, 5181.000000
29, 5173.000000

如果每個loop用掉5.2ms我可以接受(nanosleep看起來準度就是這樣)
可是差到5.6ms就....
剛剛試過用select好像更糟
※ 編輯: wtchen (86.200.210.189), 11/03/2015 23:31:04
推 wa120: 我是想說 有很小的機率會發生
→ wa120: global_thread = i%5; global_thread為2
→ wa120: 執行到下一行
→ wa120: pthread_cond_signal(&cond[global_thread]);
→ wa120: global_thread為0
→ wtchen: global_thread=0的話就變wait跳出thread回到main
→ wtchen: cond[0]也喚醒不了任何thread(我故意的)
推 yvb: 同樣程式在 PC 上跑, 看到的數值都是落在 50xx 範圍內...
→ yvb: 另外, sched_get_priority_max() 是最大值, 其priority最低.
→ wtchen: 我用sched_get_priority_max() + real time linux kernel
→ wtchen: 出來的結果是最好的
推 yvb: 改用 sched_get_priority_min() 或其它數值反而變差?
→ wtchen: 我改成49會變差