Saturday 15 March 2014

python - Why threading increase processing time? -


I was working on multitasking a basic 2-D DLA simulation Diffusion Limited Aggregation (DLA) occurs when you have The particles have to walk uniformly and have to walk together, when they mix the current total.

In simulation, I have 10.000 particles in each step in a random direction. I use a pool of workers and a queue to eat them. I feed them with a list of particles and the workers execute the method .updatePositionAndgregate () on each particle.

If I have a worker, then I get it 10.000 particles, if I have two workers, then I feed them with a list of 5.000 particles, if I have 3 workers, then I 3.333 feed with a list of particles, etc. and etc.

I'll show you some code for the worker

  class worker (thread): "" "The workers class here to process a list of particles and collect them To try. " "Df __int __ (self, name, particle):" "" worker and its events "" thread .__ init __ (self, name = name) self.daemon = True self.particles = particle self.start ( ) Def run (self): "" "The worker has started after its construction and is waiting to feed them with a list of particles to take action on them." "While correct: particle = self Wake up with .particles.get () # print self.name + ':' + str (LAN (self portals)) + 'particle' + '\ n' # Process of particles containing C. For particles in particle: particle.updatePositionAndAgregate () self.particles.task_done () # print self.name + ': is done' + '\ n'  

and in the main thread :

  Create # workers for workers (for workers in range (0, num_threads) q_ = num_threads: worker ("worker_" + str (i), worker questions) # we All particles are created unless some particles are created while some_condition (): # feed worker. For starters (datetime.datetime.now), (0, num_threads) for range: j = i * len (particle) / num_threads = (I + 1) * len (particle) / num_threads # function Karta thread feeding # Print "Main: Feeding" + Employee name. + Name + "+ str (lane (worker page) + 'particle \ n' worker QU.Pot (particle [J: K]) # Waiting for workers of all laborers () Worker Guidelines .append ((datetime.datetime.now () - startWorker) .total_seconds () Print amount (workers are given) / lanes (workers are given)  

Therefore, I print the average time to end the work of the workers To wait I did some experimenting with different thread number.

  | Num Threads | Average Worker Period (s). | ------------- | ------------------------------- | | 1 | 0.147835636364 | | 2 | 0.228585818182 | | 3 | 0.258296454545 | | 10 | 0.294294636364 | I am really surprised that why workers have to increase in processing time, I thought at least 2 workers will decrease in processing time, but it dramatically increases from 14. 0.23 Why can you explain to me?  

EDIT: So, the interpretation is Python Threading implementation, what is a path, so can I be real multitasking?

This is happening because threads do not execute at the same time Due to Python GIL (Global Interpreter Lock), only one thread can execute at a time.

When you create a new thread, it leaves everything except the thread. When it stops, the other is executed. Spawning threads require a lot of time.

Speaking friendly, code <100> is not any type of code using 100 thread, it is slower than code using Python 10 threads ( If more thread means more efficiency and more speed, which is not always right.)

Here's an exact quote:

CPython implementation details < / Em>:

Due to the Global Interpreter Lock in CPython, only one thread executes the Python code at a time. Can (even if some performance-oriented library can overcome this limitation). If you want your app to make better use of computational resources of multi-core machines, you are advised to use the multiprocessing or concurrent.futures.ProcessPoolExecutor . However, threading is still a suitable model if you want to run more than one I / O-bound work together.


No comments:

Post a Comment