simplifying rsyslog JSON generation

With RESTful APIs, like for example ElasticSearch, you need to generate JSON strings. Rsyslog will soon do this in a very easy to use way. The current method is not hard either, but often looks a bit clumsy. The new way of doing things will most probably be part of the 8.33 release.You now can define a template as follows: template(name="outfmt" type="list" option.jsonf="on") { property(outname="@timestamp" name="timereported" dateFormat="rfc3339" format="jsonf") property(outname="host" name="hostname" format="jsonf") property(outname="severity" name="syslogseverity-text" caseConversion="upper" format="jsonf") property(outname="facility" name="syslogfacility-text" format="jsonf") ...
Read more

experimental debian rsyslog packages

We often receive requests for Debian packages. So far, we did not package for recent Debian, as the Debian maintainer, Michael Biebl, does an excellent job. Other than us, he is a real expert on Debian policies and infrastructure.Nevertheless, we now took his package sources and gave the Suse Open Build Service a try. In the end result, we now seem to have usable Debian packages (and more) available at:    https://software.opensuse.org//download.html?project=home%3Argerhards&package=rsyslogI would be very interested in your feedback on the first incarnation of this project. Is it useful? Is it something we should continue? Do you have any problems with the packages? Other suggestions? Please let us know.Please node: should we decide that the project is worth keeping, the above URL will change. However, it we will give sufficiently advance notice. The current version is not suggested for production systems, at least not without trying it out on test-systems first!
Read more

docker group security risk

The Docker doc spells out that there are security concerns of adding a user to the docker group. Unfortunately, they do not precisely give what the concern is. I guess that is a "security-by-obscurity" approach trying to avoid bad things. Practice show this isn't useful: the bad guys know anyways, and the casual user has a bad time understanding the actual risk involved.It is considerable, so let me explain at least one risk (I have not tried exhaustively check security issues):  The containers are usually defined to run as root user. This permits you to bypass permission checks on the host.Let’s assume a $USER is inside the docker group and otherwise has just installed docker. So he can run$ docker run -v/etc:/malicious -ti --rm alpine# cd /malicious# vi sudoers.... edit, write ...# ctl-DAs such, the user can modify system config that he could not access otherwise. It’s a real risk. If you have a one-person “personal” machine/VM where the user has sudo permissions in any case … I’d say it’s no real issue.The story is a different one on e.g. a CI machine.  It's easy to inject bad code into public pull requests, and so it'll run on the...
Read more

rsyslog 8.31 – an important release

Today, we release rsyslog 8.31. This is probably one of the biggest releases in the past couple of years. While it also offers great new functionality, what really important about it is the focus on further improved software quality.Let's get a bit down on it. First let's mention some important new features:mmongodb has greatly been enhanced - among others, it now uses the current, state-of-the art client library and supports TLS and MongoDB replicasets... and more. Special thanks go to Jérémie Jourdin and Hugo Soszynski of Advensomprog has been greatly improved and now provides full access to all of rsyslog's action capabilities. A big thanks goes to Joan Sala.the KSI signature subsystem has been upgraded and now operates faster than ever. Thanks to Allan Park for his work.a seamingly small but important capability has been added to mmanon: it now supports embedded IPv4 adresses in IPv6. This is vital to achieve perfect privacy. Thanks to Jan Gerhards for adding this.... and of course a couple of smaller additions (albeit not less important).And then we have a set of several hundred commits concerned with improved software quality:testbench dynamic tests have been extendedcoverage of different...
Read more

The clang thread sanitizer

Finding threading bugs is hard. Clang thread sanitizer makes it easier. The thread sanitizer instruments the to-be-tested code and emits useful information about actions that look suspicious (most importantly data races). This is a great aid in development and for QA. Thread sanitizer is faster than valgrind's helgind, which makes it applicable to more use cases. Note however that helgrind and thread sanitizer sometimes each detect issues that the other one does not.This is how thread sanitizer can be activated:install clang package (the OS package is usually good enough, but if you want to use clang 5.0, you can obtain packages from http://apt.llvm.org/)export CC=clang // or CC=clang-5.0 for the LLVM packagesexport CFLAGS="-g -fsanitize=thread -fno-omit-frame-pointer"re-run configure (very important, else CFLAGS is not picked up!)make clean (important, else make does not detect that it needs to build some files due to change of CFLAGS)makeinstall as usualIf you came to this page trying to debug a rsyslog problem, we strongly suggest to run your instrumented version interactively. To do so:stop the rsyslog system servicesudo -i (you usually need root privileges for a typical rsyslogd configuration)execute /path/to/rsyslogd -n ...other options...here "/path/to" may not be required and often is just "/sbin"...
Read more

Automating Coverity Scan with a complex TravisCI build matrix

This is how you can automate Coverity Scan using Travis CI - especially if you have a complex build matrix:create an additional matrix entry you will exclusively use for submission to Coveritymake sure you use your regular git master branch for the scans (so you can be sure you scan the real thing!)schedule a Travis CI cron job (daily, if permitted by project size and Coverity submission allowance)In that cron job, on the dedicated Coverity matrix entry:cache Coverity's analysis tools on your own web site, download them from there during Travis CI VM preparation (Coverity doesn't like too-frequent downloads)prepare your project for compilation as usual (autoreconf, configure, ...) - ensure that you build all source units, as you want a full scanrun the cov-int tool according to Coverity instructionstar the build resultuse the "manual" wget upload capability (doc on Coverity submission web form); make sure you use a secure Travis environment variable for your Coverity tokenyou will receive scan results via email as usual - if you like, automate email-to-issue creation for newly found defectsActual Sample Code: rsyslog uses this method. Have a look at it's .travis.yml (search for "DO_COVERITY"). The scan run can be found
Read more

Time for a better Version Numbering Scheme!

The traditional major.minor.patchlevel versioning scheme is no longer of real use:users want new features when they are ready, not when a new major version is craftedthere is a major-version-number-increase fear in open source development, thus major version bumps sometimes come very random (see Linux for example)distros fire the major-version-number-increase fear because they are much more hesitant to accept new packages with increased major versionIn conclusion, the major version number has become cosmetic for many projects. Take rsyslog as an example: when we switched to rsyslog scheduled releases, we also switched to just incrementing the minor version component, which now actually increments with each release - but nothing else. We still use the 8.x.0 scheme, where x is more or less the real version number. We keep that old scheme for cosmetic reasons, aka "people are used to it".Some projects, most notably systemd, which probably invented that scheme, are more boldly: the have switched to a single ever-increasing integer as version (so you talk of e.g. version 221 vs 235 of systemd). This needs some adaption from the user side, but seems to be accepted by now.And different other approach, known from Ubuntu, is to use "wine-versioning": just use the date. So we have 14.04, 16.04,...
Read more

Busy at the moment…

Some might have noticed that I am not as active as usual on the rsyslog project. As this seems to turn out to keep at least for the upcoming couple of weeks, I'd like to give a short explanation of what is going on. Starting around the  begin of June I got involved into a political topic in my local village. It's related to civil rights, and it really is a local thingy, so there is little point in explaining the complex story. What is important is that the originally small thing grew larger and larger and we now have to win a kind of election - which means rallies and the like. To make matters a little worse (in regard to my time...) I am one of the movement's speakers and also serve as subject matter expert to our group (I am following this theme for over 20 years now). To cut a long story short, that issue has increasingly eating up my spare time and we are currently at a point where little is left.Usually, a large part of my spare time goes into rsyslog and related projects. Thankfully, Adiscon funds rsyslog development, and so...
Read more

Introducing new team member

Good news: we have some new folks working on the rsyslog project. In a small mini-series of two blog postings I'd like to introduce them. I'll start with Jan Gerhards, who already has some rsyslog-related material online.Jan studies computer science at Stuttgart University. He has occasionally worked on rsyslog for a while. This summer, he does a two-month internship at Adiscon. He also plans to continue to work on rsyslog and logging-related projects in the future. Right now, he looks into improving the mmanon plugin. He is also trying to improve debug logging, which I admit is my personal favorite (albeit it looks like this is the lower-priority project ;-)).
Read more
Page 1 of 212