Introducing Unminify

Shape Security is proud to announce the release of Unminify, our new open source tool for the automatic cleanup and deobfuscation of JavaScript.

Example

Given

function validate(i){var _=["no","ok"];return log(i),isValid(i)?_[1]:_[0]}

Unminify produces

function validate(i) {
  log(i);
  if (isValid(i)) {
    return 'ok';
  } else {
    return 'no';
  }
}

Installation and usage

Unminify is a node.js module and is available on npm. It can be installed globally with npm install -g unminifyand then executed as unminify file.js, or executed without installation as npmx unminify file.js. It is also suitable for use as a library. For more, see the readme.

Unminify supports several levels of transformation, depending on how carefully the original semantics of the program need to be tracked. Some transformations can alter some or all behavior of the program under some circumstances; these are disabled by default.

Background

JavaScript differs from most programming languages in that it has no portable compiled form: the language which humans write is the same as the language which browsers download and execute.

In modern JavaScript development, however, there is still usually at least one compilation step. Experienced JavaScript developers are probably familiar with tools like UglifyJS, which are designed to transform JavaScript source files to minimize the amount of space they take while retaining their functionality, allowing humans to write code they can read without sending extraneous information like comments and whitespace to browsers. In addition, UglifyJS transforms the underlying structure (the abstract syntax tree, or AST) of the source code: for example, it rewrites if (a) { b(); c(); } to the equivalent a&&(b(),c()) anywhere such a construct occurs in the source. Code which has been processed by such tools is generally signicantly less readable; however, this is not necessarily a goal of UglifyJS and similar minifiers.

In other cases, the explicit goal is to obfuscate code (i.e., to render it difficult for humans and/or machines to analyze). In practice, most tools for this are not significantly more advanced than UglifyJS. Such tools generally operate by transforming the source code in one or more passes, each time applying a specific technique intended to obscure the program’s behavior. A careful human can effectively undo these by hand, given time propotional to the size of the program.

Simple examples

Suppose our original program is as follows:

function validate(input) {
  log(input);
  if (isValid(input)) {
    return 'ok';
  } else {
    return 'no';
  }
}

UglifyJS will turn this into

function validate(i){return log(i),isValid(i)?"ok":"no"}

and an obfuscation tool might further rewrite this to
function validate(i){var _=[“no”,”ok”];return log(i),isValid(i)?_[1]:_[0]}

State of the art

There are well established tools like Prettier for formatting JavaScript source by the addition of whitespace and other non-semantic syntax which improves readability. These undo half of what a tool like UglifyJS does, but because they are intended for use by developers on their own code rather than for analysis of code produced elsewhere, they do not transform the underyling structure. Running Prettier on the above example gives

function validate(i) {
  var _ = ["no", "ok"];
  return log(i), isValid(i) ? _[1] : _[0];
}

Other tools like JSTillery and JSNice do offer some amount of transformation of the structure of the code. However, in practice they tend to be quite limited. In our example above, JSTillery produces

function validate(i)
    /*Scope Closed:false | writes:false*/
    {
        return log(i), isValid(i) ? 'ok' : 'no';
    }

and JSNice produces

function validate(i) {
  var _ = ["no", "ok"];
  return log(i), isValid(i) ? _[1] : _[0];
}

Unminify

Unminify is our contribution to this space. It can undo most of the transformations applied by UglifyJS and by simple obfuscation tools. On our example above, given the right options it will fully restore the original program except for the name of the local variable input, which is not recoverable:

function validate(i) {
  log(i);
  if (isValid(i)) {
    return 'ok';
  } else {
    return 'no';
  }
}

Unminify is built on top of our open source Shift family of tools for the analysis and transformation of JavaScript.

Operation

The basic operation of Unminify consists of parsing the code to an AST, applying a series of transformations to that AST iteratively until no further changes are possible, and then generating JavaScript source from the final AST. These transformations are merely functions which consume a Shift AST and produce a Shift AST.
This processes is handled well by the Shift family, which makes it simple to write and, crucially, reason about analysis and transformation passes on JavaScript source. There is very little magic under the hood.

Unminify has support for adding additional transformation passes to its pipeline. These can be passed with the --additional-transform transform.js flag, where transform.js is a file exporting a transformation function. If you develop a transformation which is generally useful, we encourage you to contribute it!

*** This is a Security Bloggers Network syndicated blog from Shape Security Blog authored by michaelficarra. Read the original post at: https://blog.shapesecurity.com/2018/03/02/introducing-unminify/