java - What causes a MongoDB / GridFS MD5 hash mismatch? -

I'm testing some of MongoDB GridFS as a binary store and I had to face during my tests, On rare occasions, the exception is thrown into the driver (in my case, in Java) when I recognize that the file was actually stored properly, it is the belief that the Java driver was calculated as the MD5 Comparison of materials that make it Mongo The file of DB is stored with the MD5 given back by the MD5 command.

This is an exception document and I can work to handle it properly:

My question is why does this happen (obviously a binary Mismatched, but there is nothing

I can understand that this system occurred during the failure but it appears randomly (though not very often).

Finally there was some time to check this properly, It is known!

By default, the Mongo Java driver writes for any and everybody to worry about writing any / without any unknown, which writes to all mongodb This means that writeing Mongode is an incomprehensible process. "The driver," says the driver, "when you have the chance, then store all this stuff" and Mongo eventually stores it, but immediately return a response to this application

The Mongo GridFile file object Java driver is a method called valid that can be called after the file is stored. The methodology is used by MD5, which was compiled in memory by the Java driver because the stream was divided into sections and it was sent to the mongode, and after that it has a Mongo file MD5 compares with the results of the command. When a file MD5 request is issued to Mongo, it takes the provided ID and calculates the MD5 based on the content of various binary array fragmentes which are present in the context of the provided ID (which Together, represents an entity that is stored in gridfs).

The driver, who had returned from the mouth to see the consistency of the MD5 and MD5 computed, was basically not worried about writing a default, it was returning control to the application and Immediately after the validate / filemd5 command, which currently exists on the current state of its contents. It is possible (and in fact it seems like this) that it was computing MD5 for the MongodiB ID before everything was actually written in memory and so it was a bad MD5 sometimes.

In addition, it is relevant what was seen in the debugging details - after the exception I added logging to query the file that was stored and its content was read, and at that time Always had the right material and the perfect MD5.

The solution is that Mongo driver interaction with Gridfox should at least be used by a written concern of ACKNOWLEGED, which means that at least in order to write data, Must stay in memory I have manually set it for my tests and have enabled it for gridfox archive on add-start-up because it has not yet appeared in spring-data as a configuration option, apart from this also noticeable It is that the new Mongo-3 .XA Java driver has a default written concern of ACKNOLDGDD, once I am using it, Can Iaping be withdrawn ideologically?

This is why this is why it is sometimes not seen locally, it is possible that my machine has available memory and processing power that demands the need to store the material adequately. By the time the code was called , everything was already there and therefore the MD5 came back was correct. Build agents running several types of concurrent tests are under heavy load and are probably more limited resources and therefore do not always end up processing content before the Moongodb filemand5 command is executed.

In fact, for sure verification, I had previously seen 3-4 trials (about 2000 in) due to this issue on every construction failure and I have now run 15 full test sets on the build agents. And it has not seen it once.

The actual code I am using to improve:

  private static final string GRID_FS_FILES_COLLECTION_NAME = "fs.files"; Private static final string GRID_FS_CHUNKS_COLLECTION_NAME = "fs.chunks"; @Atoward Protected Monooperation Mongo Operation; MongoOperations.getCollection (GRID_FS_FILES_COLLECTION_NAME) .setWriteConcern (WriteConcern.ACKNOWLEDGED); MongoOperations.getCollection (GRID_FS_CHUNKS_COLLECTION_NAME) .setWriteConcern (WriteConcern.ACKNOWLEDGED);