Home » Security Bloggers Network » Announcing the community-oriented osquery fork, osql

Announcing the community-oriented osquery fork, osql

by Mike Myers on April 18, 2019

For months, Facebook has been heavily refactoring the entire osquery codebase, migrating osquery away from standard development tools like CMake and integrating it with Facebook’s internal tooling. Their intention was to improve code quality, implement additional tests, and move the project to a more modular architecture. In practice, the changes sacrificed support for a number of architectures, operating systems, and a variety of useful developer tools that integrate well only with the standard build system preferred by the open-source C++ community.

Worse still, the project’s new inward focus has greatly delayed the review of community contributions — effectively stalling development of features or fixes for the needs of the community — without a clear end in sight. Lacking a roadmap or predictable release cycle, user confidence in the project has fallen. Enterprises are postponing their planned osquery deployments and searching for alternative solutions.

Many of the most secure organizations in the world have already invested in making osquery the absolute best endpoint management solution for their needs. Being forced to look elsewhere would be a waste of their investment, and leave them relying on less effective alternatives. That is why we are announcing the community-oriented osquery fork: osql.

What are the goals of osql?

With osql, we are committed to restoring the community’s confidence in the osquery project, to making the development process more open and predictable, and to reviewing and accepting community contributions more quickly. Our goal is to restore direct community participation.

An open and transparent development process

In the immediate term, osql will be maintained as a “soft-fork.” We will closely track Facebook’s upstream updates without diverging from the codebase. Plenty of completed work is simply waiting upstream, in Pull Requests. We prepared a workflow through which the osql project can accept Pull Requests that the community deems stable enough to be shipped, but which have been ignored by the upstream maintainers. The community can pick and choose its priorities from those contributions, and incorporate them into the next release of osql.

Sponsorships Available

The osql organization on GitHub will be a hub for community projects

Continuous Integration, Continuous Delivery

We’ve also integrated a much-needed public CI using Azure Pipelines, which will build and run tests at each commit. Find the results here. The CI will help us build, test, and release faster and more frequently. We are committing to release a new osql binary (package installer) on a regular monthly cadence. We will communicate the changes that users can expect in the next release. They will know when to expect it, and that the version they download has passed all tests.

Determine if the latest code is building for all platforms, at a glance

Restoring standard tool support for developers

We rewrote the build system from scratch to return it to CMake, the C++ community’s de-facto standard for building projects. This effort was non-trivial, but we believe it was central to preserving the project’s compatibility with open-source toolchains. The libraries and tools that represent the foundation of modern C++ development, such as Boost or the LLVM/Clang compiler toolchain, all support CMake natively. The most-used third party libraries use CMake as well, making it quite easy to include them in a CMake-based project.

Developers benefit from built-in CMake support in their IDEs. Visual Studio, VS Code, CLion and QtCreator can all easily open a project from its CMakeLists file, enabling a precise view of the project’s structure and the outputs of its build process. They’ll also regain the convenience of CMake-supporting static analyzer frameworks, like Clang’s scan-build, which helps discover critical bugs across an entire project.

By re-centering everything around a CMake build process, we made osql a more developer-friendly project than upstream osquery. If you would like to see for yourself and begin contributing to osql, check out the build guide.

Work conveniently in the Visual Studio Code IDE, with CMake integration

What’s next for osql

Our work is just beginning! We plan to continue improving the automation of osql releases. Initially, osql releases will be unsigned binaries/packages. The next priority for the project is to implement a secure code-signing step into the CI procedure, so that every release is a binary signed by the “osql” organization.

The osquery project’s build process used to allow you to choose whether to download or to build third-party dependencies, thanks to easily modifiable Homebrew formulas. Not only that, you could also choose from where these dependencies were downloaded. That is no longer true for osquery currently, but we will restore that ability in osql (a task made easier thanks to CMake).

We also plan to extend the public CI for osql to enable it to test PRs opened against upstream osquery. This will help the community review those PRs, and provide a kind of quality assurance for their inclusion in a future release of osql.

In the longer term, thanks to CMake’s support for building on various platforms, it will be possible for osql to be built for whatever new systems that the community demands.

Want More? Let’s Talk

When we originally ported osquery to Windows, we didn’t imagine it would become so big, or that it would outgrow what Facebook alone could maintain. A whole community of organizations now deploy and depend on osquery. That’s why we’ve launched osql, the community-oriented osquery fork. If you are part of this community and are interested in porting to other platforms, need special features from the project, or want some customization done to the core, join our osquery/osql support group or contact us!