python - Run time of multiple processes at the same time

Question

I'm trying to understand what happens when I fire up several same processes on a multi-core processor and how actually it should behave. I'm running everything on laptop with Windows 10, i7-10510U (4 cores, MT 8 logical cores)

I've started analyzing the problem when Python's multiprocessing (3.8.5) was giving me some unexpected behavior. My expectation is that if I run 3/4 processes on a 4 core machine (let's ignore the logical cores for now) I should get at least 3x times speed up. However, what I saw that it was getting a less than 2x speed up. I did some testing with trivial problems. I used a simple hashing to make a time consuming program:

import hashlib
from timeit import default_timer as timer

start = timer()
for i in range(1,10000000):
    hashlib.sha224(b"Nobody inspects the spammish repetition").hexdigest()
end = timer()

print(f'proj time: {end - start}') 

When I run this as one process I get about 8s. If I fire up 4 separate processes manually at the same time the timing on each goes up to about 14s, hence the speed up is drastically impacted by the processing time of a single process. I thought that maybe something with the Python runtime (shouldn't, cause all are separate processes!) so tried some similar simple code in C++:

#include <iostream>
#include <string>
#include <chrono>
#include <iomanip>

int main()
{
    int noOfLoops = 300000000;
    auto t1 = std::chrono::high_resolution_clock::now();
    std::string sv = "Nobody inspects the spammish repetition";
    std::string a;
    for (int i = 0; i < noOfLoops; i++) {
        a=std::hash<std::string>{}(sv);
    }
    auto t2 = std::chrono::high_resolution_clock::now();

    float duration = std::chrono::duration_cast<std::chrono::milliseconds>(t2 - t1).count() / 1000.0;

        std::cout << std::fixed << std::setprecision(3) << duration << std::endl; 
        system("pause");
}

Running this yields about 10s on my pc. But again, firing this up 4 times makes the run time on single up go up to 15s.

Is there something I'm missing? Is it a expected behavior from your side?

question from:https://stackoverflow.com/questions/65951933/run-time-of-multiple-processes-at-the-same-time

Answer 1

Waitting for answers