![]() ![]() I tried forcing the resulting encoded value (result of event.sprintf) to be ASCII-8BIT but that didn't help. This also broke how logstash-codec-plain works because originally it relied on a pure Ruby implementation that doesn't force a different encoding on the value. Turns out in Logstash 5.0 there was breaking change where they moved the core Event object from native Ruby to Java for performance reasons. The event.sprintf function always produces a UTF-8 string even if the logstash-codec-plain configuration parameter charset is not UTF-8 and even if the event value is not UTF-8. The culprit is the logstash-codec-plain plugin which invokes event.sprintf. I have uncovered the root cause of my problem. ![]() In my separate script, I tried opening the file with the a+ mode like that in the output file plugin and that still seemed to workĪny ideas where the problem is and how to fix it?.Set the file output to flush 0 had no impact.Specfiying the charset to be BINARY had no impact.That works fine so that tells me that the issue is with the output part of the pipeline. The files being written seem to be binary but corrupt.Īs a separate test, I wrote the base encoded attachments to file and then in a separate script simply read the file, base 64 decoded and then wrote to another file. ![]() My issue I think lies with either the output file plugin or the codec plain plugin. In the end I want a PDF file and a image file given the filename from the email.Īll the attachments come in Base64 encoded and so I have a filter step to decode. Those two attachments get split into two events - one for the PDF and one for the image. For example an email could have attachments of a PDF and an image. The gist of what's going on above is taking the email messages, grabbing the attachments and splitting them up into individual events and then writing each event (attachment) into their respective file. Here is a snippet of my pipelines.yaml: - pipeline.id: email-write-attachments I'm stuck on writing the attachments to the filesystem. My issue is that the files are not getting written correctly so not readable. I'm trying to use Logstash to ingest emails and write the messages into Elasticsearch and write attachments onto the filesystem (maybe S3 later). ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |