elasticsearch - Logstash json filter for only one level -


i use following filters configuration:

filter {   if [type] == "client-log" {     grok {       match => { "message" => "%{combinedapachelog}" }     }     urldecode{       field => "request"     }     mutate {       gsub => [ "request", '/log/', '' ]     }     json {       source => "request"     }   } }

it works fine when request 1 level json object. if gets more 1 level logstash getting parse error. advice?

request example:

{    session_id: "123",    message: {        id: 1221       //....    }    //.... }

error:

{:timestamp=>"2016-09-07t15:53:34.712000+0100", :message=>"error parsing json", :source=>"request", :raw=>"{\"session_id\":\"8d078da0-74f9-11e6-8d31-6925e76dde0e\",\"level\":\"debug\",\"methodname\":\"fetchexternaldata\",\"class\":\"fetchexternaldata\",\"linenumber\":78,\"message\":\"{\"0\":\"fetchexternaldata dofetch assets \",\"1\":\"http://lab6services:8080/alerts\",\"2\":{\"method\":\"post\",\"headers\":{},\"body\":\"{\\\"results\\\":{\\\"types\\\":[\\\"alerts\\\"],\\\"format\\\":\\\"table\\\"},\\\"filter\\\":{\\\"title\\\":\\\"new alerts\\\",\\\"filtertype\\\":\\\"property\\\",\\\"operator\\\":\\\"range\\\",\\\"field\\\":\\\"createdat\\\",\\\"type\\\":\\\"alert\\\",\\\"values\\\":[{\\\"value\\\":\\\"07/09/2016 15:49:44\\\"},{\\\"value\\\":\\\"07/09/2016 15:51:44\\\"}]},\\\"aggregate\\\":null}\"}}\",\"version\":\"1.2.0\",\"user\":\"user\",\"timestamp\":\"2016-09-07t12:51:44.953z\"}", :exception=>#<logstash::json::parsererror: unexpected character ('0' (code 48)): expecting comma separate object entries  @ [source: [b@51b925ff; line: 1, column: 161]>, :level=>:warn}

log line:

127.8.4.1 - - [07/sep/2016:15:54:07 +0100] "get /log/%7b%22session_id%22:%228d078da0-74f9-11e6-8d31-6925e76dde0e%22,%22level%22:%22debug%22,%22methodname%22:%22fetchexternaldata%22,%22class%22:%22fetchexternaldata%22,%22linenumber%22:78,%22message%22:%22%7b%220%22:%22fetchexternaldata%20dofetch%20assets%20%22,%221%22:%22http://lab6services:8080/alerts%22,%222%22:%7b%22method%22:%22post%22,%22headers%22:%7b%7d,%22body%22:%22%7b%5c%22results%5c%22:%7b%5c%22types%5c%22:%5b%5c%22alerts%5c%22%5d,%5c%22format%5c%22:%5c%22table%5c%22%7d,%5c%22filter%5c%22:%7b%5c%22title%5c%22:%5c%22new%20alerts%5c%22,%5c%22filtertype%5c%22:%5c%22property%5c%22,%5c%22operator%5c%22:%5c%22range%5c%22,%5c%22field%5c%22:%5c%22createdat%5c%22,%5c%22type%5c%22:%5c%22alert%5c%22,%5c%22values%5c%22:%5b%7b%5c%22value%5c%22:%5c%2207/09/2016%2015:49:44%5c%22%7d,%7b%5c%22value%5c%22:%5c%2207/09/2016%2015:52:22%5c%22%7d%5d%7d,%5c%22aggregate%5c%22:null%7d%22%7d%7d%22,%22version%22:%221.2.0%22,%22user%22:%22user%22,%22timestamp%22:%222016-09-07t12:52:22.928z%22%7d http/1.1" 200 0 "http://localhost:3000/main.worker.js" "mozilla/5.0 (macintosh; intel mac os x 10_11_6) applewebkit/537.36 (khtml, gecko) chrome/52.0.2743.116 safari/537.36"

--- edit ---

i want parse first level of 'resquest'. how can prevent filter parse nested json elements?


-

Comments

Post a Comment

Popular posts from this blog

serialization - Convert Any type in scala to Array[Byte] and back -

i have following question: i have variable value in program declared value. i want convert value byte array.. how can serialize byte array , back? found examples related other types such double or int, not any. this should need. it's pretty similar how 1 in java. import java.io.{bytearrayinputstream, bytearrayoutputstream, objectinputstream, objectoutputstream} object serialization extends app { def serialise(value: any): array[byte] = { val stream: bytearrayoutputstream = new bytearrayoutputstream() val oos = new objectoutputstream(stream) oos.writeobject(value) oos.close stream.tobytearray } def deserialise(bytes: array[byte]): = { val ois = new objectinputstream(new bytearrayinputstream(bytes)) val value = ois.readobject ois.close value } println(deserialise(serialise("my test"))) println(deserialise(serialise(list(1)))) println(deserialise(serialise(map(1 -> 2)))) println(deserialise(seri
Read more

matplotlib support failed in PyCharm on OSX -

any idea how fix ? how can debug might problem ? not seem : https://youtrack.jetbrains.com/issue/py-15520 i in console: python 2.6.9 (unknown, jul 14 2015, 19:46:31) [gcc 4.2.1 compatible apple llvm 6.0 (clang-600.0.39)] on darwin >>> import matplotlib.pyplot plt matplotlib support failed the backend : >>> import matplotlib >>> print(matplotlib.rcparams['backend']) macosx
Read more

python - Matplotlib: TypeError: 'AxesSubplot' object is not callable -

i new programming please pardon me if rookie mistake. i writing python script have defined functions create subplots of distributions. i have slider widget end user change value of variables(mu , sigma) , see effect on distribution. here's code import numpy np print np.__version__ import scipy print scipy.__version__ scipy.stats import norm, lognorm, uniform import matplotlib print matplotlib.__version__ import matplotlib.pyplot plt matplotlib.widgets import slider, button, radiobuttons, checkbuttons #####calculating mean , standard deviation##### global mu_a1 mu_a1 = 1 global mu_b1 mu_b1 = 10 global mu_c1 mu_c1 = -13 global sigma_a1 sigma_a1 = 0.14 global sigma_b1 sigma_b1 = 1.16 global sigma_c1 sigma_c1 = 2.87 mu_x01 = -11 sigma_x01 = 1.9 #####_____##### #####generating random data##### a1 = 0.75*mu_a1 + (1.25 - 0.75)*sigma_a1*np.random.sample(10000) b1 = 8*mu_b1 + (12 - 8)*sigma_b1*np.random.sample(10000) c1 = -12*mu_c1 + 2*sigma_c1*np.random.sample(10000)
Read more