Sunday 15 March 2015

sqlite - Python 3: Best string compression method to minimize the size of a sqlite3 db -


I recently created a script that parses multiple web proxy logs into a clean sqlite3 db file, for me Is working very well ... the size of the file with a hassle is pressed to use this format (a sqlite3 db) and the dragon handles it natively as a winner, so my question is this .. What string is the best way to compression, which I used to do Can new use when the file size zlib only concern? Base-n? Klingon?

Any advice will help me load, then string compression for characters compliant for the URL.

There is a SQLite extension to provide compression here.

This extension provides a function that can be called on the individual field.

Here is some example page from text

Make a test table

sqlite> table test (name varchar (20), surname varchar ( 20));

Insert some text into the test table in the test table, you can also insert binary material and put it in the Blob field

sqlite> Insert test values ​​(mycompress ('This is a sample text'), mycompress ('this is a sample text'));

It does not show anything because our data is in binary format and select Compressed

sqlite> from Test *;

The following works, it dissolves the data

sqlite> from the test select myuncompress (name), myuncompress (surname);


No comments:

Post a Comment