Saturday 15 January 2011

ruby - Logstash - Grok - Exception on multiple lines -


I am trying to parse through a grok filter, very few exceptions, so I wrote a grok filter Rubular.com, to parse each exception of each type. The filter is:

  grok {match = & gt; {Message = & gt; "^ (& Lt; year & gt; \ d {4}?) - (, \ d {1,2}?) - (& lt ;? day & gt; \ {1, 2}) \ w (& lt; hour & gt; \ d {2}): ( d; {2}): (& lt; second & gt; \ {2}) ( ,) [0-9] * (*) (& lt?? Log_level> (ERROR | INFO)) (? & Lt; exception & gt; (. * \ N ^ axis. * \ N. * \ N. * \ N. * \ N. * \ N. * \ N. * \ N. *) | (Com. * Trying. * \ Ncom. *. * * *. * Java . *) | (Com. * \ N ^ org. * \ N ###. * Non Valid \ n \ N * * ###. * \ N ^ ###. * \ N ^ ###. * ) | *. (*. * Trying. * \ N ^ com. * Service Exception. * \ N ### Error Query. * \ N \ n * \ n ^ ### * \ n *) | .. ..... (* Try * \ n ^ com * ServiceException * \ n ^ ### * \ n ^ ###. *). (* *. * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ================= ================================================ * \ N ^ org * * \ n ### *) | .. (* \ n ^ Java * CORBA * \ n *) | .... (* \ n ^ Java * *) | | (Com. * \ N ^ com *) | ........ (* null \ n ^ axis * \ n * \ n * \ n * \ n * \ n * \ n * \ n * \ n ... * \ N. * \ N. *). (. * \ N)) "}}  

The problem is that you can see as many or in the exception field, and for a lot of things \ n take a carriage return. That, what I think, logostas can read only one line at a time and can not match with multiple lines (hence, this pattern is working perfectly on the price, it is not in logsta). How can I properly filter?

You can multiline before the grok, for example the Java exception:

  multiline {type =>% showtype pattern = & gt; "(^ \ S)" What is & gt; Previous}  

Then it will add all the lines that start from the whitspace, and then you can use the grok filter.

Oh, and you can mutate to avoid the symbols '\ n' after multiline

  Mutate {gsub = & gt; ["message", "\ n", ""]}  
< P> After that you are ready to filter multiline messages.


No comments:

Post a Comment