.net - Is there a way of directely loading a textfile into the sqlite database without processing inside the code it but still using c#? -

I am currently working on a program that reads data from a huge (40,000 lines) * .txt file Lines look like this:

4, Barberdorf, 505,552,15758 99232,378,0 5, der + Lazette + Macht + Das + Leach + Aus, 484,458,1576458064,5757,0

In general:

  $ id, $ name, $ x, $ y, $ player, $ points, $ rank  

The following is the work that was written to get this data into the SQLite database:

  void ThreadMethod () {string sql = ""; SQLiteConnection m_dbConnection; M_dbConnection = New SQLiteConnection ("Data Source = Villages .db; version = 3;"); M_dbConnection.Open (); SQLiteCommand command = new SQLiteCommand (sql, m_dbConnection); Try ({StreamReader sr = New StreamReader ("village.txt")) {String Line; Regex regex = new reggae (","); string [] substrings = regex.Split (line); int i = 0; string [] strVillage = new string [7]; foreacha (strings match in substrings) {strvillage [i ++] = match;} SLL = "villages (village name, village name, exchord, yoke, player name, village (+ + Strvillage [0] + ", + + strvillage [1] +" ',' + strvillage [2] + "," + swell [3] + + ", + + + + + Swilez [4] + "',' + Stewellage [5] +", "+ strowelgele [6] +");; command = New SQLiteCommand (SQL, M_D beaconation); command.ExecuteNonQuery (); Var line = System .iofile.readlines ("VillageT.T.T."); System.IO.File.WriteAllLines ("village.txt Take lines, lines (lines - 1) .oir ());}}} hold (ex before) {message box. Show (ex.Message, "error", MessageBoxButtons.OK, MessageBoxIcon.Error); } M_dbConnection.Close ();}  

This works, but it is really slow I hope you can help me improve performance. Best relationship!

An immediate and marked performance improvement can be done using a compiled regex instead of creating a new one Examples for the line are the method in which the class is:

  Public Class ClassThatThisMethodIsIn {Personal Stable Regex regex = New Regex (",", RegexOptions.Compiled); // the rest of the code goes here)  

and extract the line in this method:

  Regex regex = new regenges (",");  

If you are the only reason for splitting on a comma that you are using Regex, then delete the Regex completely and use the string instead. Sample:

  var substrings = line.Split (',');  

Further performance can be achieved by batching the inclusion in INSERT ... Choose multiple rows at one time, instead of putting the union statement and one for each row.