Logstash pipeline tester
Code is community submitted, community supported, and recognized as ‘Use At Your Own Risk’.
Short Description
A tool that makes developing logstash pipelines much much easier.
Problem solved by this Code Snippet
Oh. The problem... Have you ever tried to write a logstash pipeline? Did you suffer hair loss and splitting migraines? So did I.
Presenting, logstash pipeline tester which gives you a web interface where you can paste raw logs, send them to the included logstash instance and see the result directly in the interface. The included logstash instance is also configured to automatically reload once it detects a config change.
How to use this Code Snippet
TLDR;
- Don't do this, read the manual or checkout the video below
- Still here? Ok then! 🙂
- Install docker
- Clone the repo
- Run these commands in the repo root folder:sudo docker-compose build # Skip sudo if running Windows sudo docker compose up # Skip sudo if running WindowsGo to http://localhost:8080 on your PC/Mac
- Pick a pipeline and send data
- Edit the pipeline
- Send data
- Rince, repeat
Version info
v1.0.0
- Docker containers no longer runs as root
- Vulnerability fix:https://github.com/epacke/logstash-pipeline-tester/releases/tag/v1.0.0
Video on how to get started:
https://youtu.be/Q3IQeXWoqLQ
Please note that I accidentally started the interface on port 3000 in the video while the shipped version uses port 8080. It took me roughly 5 hours and more retakes than I can count to make this video so that mistake will be preserved for the internet to laugh at. 🙂
The manual:
https://loadbalancing.se/2020/03/11/logstash-testing-tool/
Code Snippet Meta Information
- Version: Check GitHub
- Coding Language: NodeJS, Typescript + React
Full Code Snippet
v1.0.25 Adding TCP retries when sending data to logstash. This makes the unit tests more robust.
v1.23 Better logging, error handling and more.
- Adding Pino as logging library, adding env BACKEND_ENDPOINT to make development easier
- Adding instructions on how to start a development environment
- Adding environment variable BACKEND_ENDPOINT in the logstash container to allow configuring logstash output to use the express web app instead of the docker name resolution
- Defining backend endpoint with environment variable and updating README.md
- Adding more detailed error message when backend is unable to contact logstash
- Upgrading React to 18.3.1
- build(deps-dev): bump rollup from 2.79.1 to 2.79.2 in /pipeline-ui/frontend by @dependabot in #253
- [Snyk] Security upgrade express from 4.21.0 to 4.21.1 by @epacke in #254
- [Snyk] Upgrade nodemon from 3.1.4 to 3.1.7 by @epacke in #255
- build(deps): bump cookie and express in /pipeline-ui/frontend by @dependabot in #256
- build(deps): bump cross-spawn from 7.0.3 to 7.0.6 in /pipeline-ui/frontend by @dependabot in #258
- Fixing postcss@^7.0.35 depnendency vulnerability by @epacke in #260
v1.0.18 released with minor patches. 🙌
A whole bunch of dependencies updated:
https://github.com/epacke/logstash-pipeline-tester/releases/tag/v1.0.14
A LOT of vulnerability patches. Tested and released v1.0.12.
Bumping Material packages and axios:
https://github.com/epacke/logstash-pipeline-tester/releases/tag/v1.0.9
v1.0.8 Bunch of security fixes. 🙌
v1.0.5 released which fixes an issue for Windows users with default git client settings. Also patches a bunch of vulnerabilities.
v1.0.4 released with a bunch of vulnerability patches:
https://github.com/epacke/logstash-pipeline-tester/releases/tag/v1.0.4
Another bump, this time semver. Now it should be calm for a while.
Kind regards,
Patrik