python - Numpy: Multiprocessing a matrix multiplication with pool -

I am trying to calculate the dot product with pool

  pool = The pool (8) X = np.array ([2,3,1,0]) y = np.array ([1,3,1,0]) print np.dot (x, y) #works Print Pool. Map (NP.DOT, X, Y) # ValueError below terror: The truth value of an array with more than one element is ambiguous. Also try either A.any () or a.all ()  

  ne.evaluate ('dot (x, y)') Type Error: 'Variable Node' object is not scalable  

What do you do Unfortunately, the way you are trying to do it is not possible in it, and it is not possible either in a simple way.

To make things even worse, multiprocessing for Python 2.7. The pool document is completely false and is essentially contained for it Pool.map : this builtin Map is not equal to . Builtin map can take many logic theaters to pass the function, while Pool.map can not ... it is known and not fixed or in the dotstring For documented, Pool.map Obviously, there is a partial fix in Python 3, which ...

Honestly, however, multiplication to speed up numerical code The module is not very useful. For example, for long discussion of such situations where many numpy operations were slow when was done through multi-processing.

However, there is another issue here: you can not just parallel this. Map takes lists / iterators of arguments and in turn applies a function for each. It is not what you want: In this case, try the map (np.dot, x, y) , and note that what you get is just X and Y The product of each element is a list, it is easy to run a function multiple times in a parallel dot product. It is difficult to make that call parallel to a call because it requires the function to be parallel. In this case, this usually means to rewrite the function.

except for np.dot actually parallel already , if you have a version of egg with blas or atlas ( np.__ config___ Show () ). You do not really need any work in that case: np.dot (x, y) should already use all your cores without any work!

I should take care that this is limited to some dtypes ; Float is usually the most supported. For example, on my computer, clear difference between float and int :

 : [= 1] See: a = np.matrix (np.random.randint (0,10, size = (1000,1000)), dtype = 'int' in [20]: b = a.astype ('float') [ 23]:% timeit np. Dot (A, A) 1 loop, 3: 6.91s per loop in best [24]:% timeit np.dot (b, b) 10 loops, best 3: 28.1 ms per loop  

For numexpr (more useful in asking questions, it is helpful to indicate that you are not using it briefly), only a limited set of supported functions; Check the document for a list You get this error because dot is not a supported function. Since you are working with 1D arrays, and the dot is very easy to define, the following will work: ne.evaluate ('sum (x * y)') . I doubt, however, that you are planning to use only 1D arrays

If you really want to parallel things to a large extent, then I recommend using IPython , And this, which is contrary to the multiplication of Python, is actually useful for numerical work. As an extra bonus, it can also parallel across computers. However, this kind of similarity is usually only useful for things that take a little time per run; If you want to use all your core for ordinary things, then it is best to hope that you have the support of the majority of the work that you want to use to use.