multithreading - Multiple access to mmap objects in python -

I have a number of maps for memory (like mmap objects). Each file should be opened multiple times during their processing. It works fine, if there is only one thread, however, when I try to run the work in parallel, a problem arises: different threads can not use the same file together Problem this sample Pictured by:

  Import mmap, threading class MMAP readings (threading thread): def __init __ (self): threading. Thread .__ init __ (self) def run (self): in category for (10000): content = mmap_object.read (). If not content: print (error reading 'mmap object') opened ('my_dummy_file.' ('My_dummy_file.txt', 'r') as F: Mmap_object = mmap.mmap (f.fileno (), 0, prot) as F: f.write ('Hello World' as t: 'txt', 'w') = Mmap.PROT_READ) Thread = [] I (64) in category: threadsappend (MmapReading ()) threads [i] .damemon = true threads [i] threading.enumerate () Start for threads (if:) Thread! = Threading Courant_thread (): Thread .join () print ('MMAP reading tested!')  

Every time I run this script, I get about 20 error messages.

Is there a way to overcome this problem, the other 64 copies made of each file ies (which would consume too much memory in my case)?

search (0) is not always done before jumping into another thread And read () .

1. Say for reading the thread 1, the end of the file to read; The search for (0) has not been executed yet.
2. Then the thread is read 2. The file indicator in Mmap is still at the end of the file. reads () hence '' returns.
3. Error identification code has been triggered because content is '' . Instead of using reading <> , you can use laminate to get the same result. Change:

     content = mmap_object.read (). With decoded ('utf-8') mmap_object.seek (0)  

     content = mmap_object [:]. Decode ('utf8')  

   content = mmap_object [: mmap_object.size ()] also works.

   Locking is another way, but in this case it is unnecessary. If you want to try it, you can use the global threading.lock object and send it to MmapReading immediately. Store the Lodge object in an instance variable self.lock . Then self.lock.acquire () before calling for reading / and calling self.lock.release () later. You will experience such a very noticeable performance penalty.

     Threading import lock class MMAP readings (threading thread): def __init __ (self, lock): self.lock = lock threading.Thread .__ init (define) def run (manually ): For category (10000): self.lock.acquire () mmap_object.seek (0) content = mmap_object.read () Decode ('utf-8') self Lock.release () if not content: print (error in reading 'mmap object') = Lock (for lock) I (in range): threads.append (MmapReading (lock)). . .  

   Note that I have changed the order of reading and searching; It makes more sense to search for the first to move the file pointer at the beginning of the file.