Parsing YAML is a minefield

Not long ago I was researching the behavior of YAML parsing libraries as part of my master’s thesis. Since there is a YAML specification , I wanted to assess whether the parsing libraries listed on the official yaml.org website comply with the current specification. My idea is hugely inspired by Nicolas Seriot’s Parsing JSON is a minefield ¹, which I definitely recommend reading.

Back when I chose the topic, I did not really know what I was stumbling into.

After conducting my research, I found that no two YAML parsers I examined produced identical results across the complete set of YAML documents I created and tested. I also found parser behavior that could enable Denial-of-Service (DoS) attacks in applications that process untrusted YAML input, particularly when no additional safeguards are in place.

#Why Specification Compliant YAML Parsing Matters

YAML finds many applications nowadays. Recently I found out that Elasticsearch supports HTTP requests using YAML instead of JSON.² Now, if someone were to send a maliciously crafted HTTP request using YAML, and the process hangs, or even worse, consumes all resources on the system, this could lead to the service being unavailable.

Or imagine the scenario where systems exchange data using YAML and they use different YAML parsing libraries. In a hypothetical scenario, we might have a YAML document that looks as follows:

users:
 - alice:
     admin: False
     role: Assistant
 - bob:
     admin: true
     role: Administrator

System A then sends this YAML document to system B. System B parses the YAML document with another YAML parsing library. The YAML parsing library of system B does not interpret False as a boolean but rather as a string (i.e., "False"). System B then goes on and checks the admin status of user Alice with the following function:

def check_admin_status(admin_status):
	if admin_status:
	  print("You're an admin")

Since False is being interpreted as a string instead of a boolean value, system B erroneously grants Alice administrative privileges.

Now, this is a hypothetical scenario which may or may not depict reality. However, I think everybody can see the problem with parsing libraries that deviate from the specification. My research showed that no two YAML 1.2 parsing libraries produce the same results given the same set of YAML documents. Furthermore, during my research I stumbled upon YAML documents that, on certain YAML 1.2 parsing libraries, lead to excessive resource consumption or even crash the whole process by a non-catchable stack overflow error.

#How the parsers were tested

First, I needed to choose which YAML parsing libraries I wanted to test. I selected YAML 1.2 libraries because the newest specification is YAML 1.2.2. The following parsing libraries have been chosen for my research:

My approach was to build a YAML Test Suite. Only after I had finished my own test suite did I discover that something similar already existed.¹⁹ That was not necessarily a bad thing: I learned a lot while building it and refreshed some rusty programming skills along the way. The existing YAML Test Matrix I found had its latest published matrix over 4 years ago, so building a new setup also gave me the opportunity to test a more recent selection of parser libraries.

Back to the topic, I created a Python script which is the heart of the YAML Test Suite:

• It scans the directory for YAML test case documents;
• Triggers the parsing process for each YAML parsing library with each YAML test case document;
• Accumulates the results;
• Outputs the results as an HTML file for visualization.

The entire parsing process is carried out within what I referred to in my thesis as standalone applications. These standalone applications are executables that have just one job: parse a given YAML document; if successful, return exit code 0; if not, return exit code 1. It’s as easy as that. To avoid that my YAML Test Suite gets stuck, because a parsing library fails to finish parsing, a 3 second timeout was set. If a standalone application fails to parse a file within 3 seconds, the result gets flagged as TIMEOUT. When an exit code other than 0 or 1 is being returned, the standalone application likely crashed due to an unknown error.

The Python script then checks the exit code and whether the result is expected according to the YAML 1.2.2 specification. Moreover, stdout and stderr are always being logged in a separate log file for later analysis.

A quick overview of all components can be seen in the following depiction.

One of the most important aspects of my research was building my own YAML test case documents. In the end I wrote 245 YAML test case documents by reading the YAML 1.2.2 specification. While I was reading the YAML 1.2.2 specification, I was thinking about what syntax should be accepted by a specification compliant parser and what not. However, these test cases do not cover the whole YAML 1.2.2 specification, as this was way beyond the scope. In other words, my YAML Test Suite can not determine the complete compliance to the YAML 1.2.2 specification. However, it can discover YAML parsing libraries that are not compliant. In order for my YAML Test Suite to distinguish what is valid according to the specification, and what is just rubbish that I came up with, I was prefixing the filenames of each YAML test case document with valid_ or invalid_.

#Specification compliance

After running my YAML Test Suite, I quickly saw that none of the 16 parsing libraries behaved in accordance with the YAML 1.2.2 specification for all 245 test cases. Every library had at least one test case where its behavior differed from what I expected based on the specification. This can make YAML parsing less predictable, particularly when different implementations are used across systems.

#Do all YAML parsers have the same behavior?

Expectedly, no. Across my complete test set, no two parser libraries produced identical results. Considering the complexity of YAML, this did not surprise me too much. The authors of the official yaml-grammar repository put it quite nicely: “[f]ully comprehending the YAML grammar is quite an undertaking for most mortals. Creating a fully compliant parser has proven almost impossible.”²⁰

In practice, this means that YAML libraries have implementation-specific differences that developers need to be aware of. A document that works with one parser may not behave in exactly the same way with another. This becomes particularly relevant when different systems exchange YAML while using different parser implementations.

#Denial of Service

To top it off, I have also found various YAML documents that lead to a DoS in the parsing library. One observation that I have made is that 13 out of 16 parsing libraries were breaching the 3-second timeout on a YAML document called valid_1000000_sequences.yml. As the name implies, the file consists of 1,000,000 sequences. The file looks as follows:

- 0: 'Test 0'
- 1: 'Test 1'
- 2: 'Test 2'
- 3: 'Test 3'
- 4: 'Test 4'
[...]
- 999995: 'Test 999995'
- 999996: 'Test 999996'
- 999997: 'Test 999997'
- 999998: 'Test 999998'
- 999999: 'Test 999999'

This file has a size of 23 MB. And well, I mean, it’s kinda obvious that the bigger the YAML file, the longer the parsing time (in Austria we’d say “no na ned”).

Nonetheless, this is another prime example of why unrestricted YAML parsing can become dangerous. Only the SnakeYAML Engine warns from this YAML document by throwing an error and refuses to parse this document, which is a neat feature. The YAML 1.2.2 specification however does not really prepare for such a scenario since there is no mention of a recommended maximum size.

A, in my opinion, far more concerning DoS-vector is hidden inside HsYAML and YamlReference. By adding square brackets (JSON arrays), the execution time of these two parsing libraries doubles. The following example illustrates shows how such a YAML document could look like:

[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]

In one example I was able to reach an execution time of ~34 minutes on HsYAML and ~48 minutes on YamlReference with a YAML document that contains 25 opening and 25 closing square brackets and nothing else. In the following figure you can see the execution times in relation to the opening and square brackets. A non-linear pattern can be observed.

The root cause is a backtracking implementation that repeatedly tries to resolve the input as a flow sequence while the parse remains incomplete. The parser, however, does not see the end of that sequence but rather the beginning of a nested sequence. So the parser recursively calls this backtracking function again, trying to resolve the new nested sequence. This cycle is being repeated for as long as there are opening square brackets, which leads to a massive allocation of memory. This slows down the process and consumes a huge amount of resources. What makes this particularly perfidious is that just a small YAML document (50 square brackets have a size of only 50 bytes) can cause a lot of damage.

But HsYAML and YamlReference are not the only examples that have problems with nesting in YAML documents because it gets even worse. YamlDotNet has a similar issue with JSON arrays, with the only distinction that the process does not hang but rather dies due to an uncatchable stack overflow error. If, for example, an application wants to parse untrusted YAML input, an attacker would be able to kill the whole process without the chance to catch the error.

The parsing library yaml-rust also runs into a stack overflow on a specific YAML document that includes huge nesting. While it can process square brackets well enough for it to not crash, it cannot handle nested sequences like the shortened version in the following example:

- - - [...] - - - test

The result is a stack overflow error, which is also not catchable in Rust.

#What this means for developers

Since no two parsing libraries parse YAML in the same way, it must be used with caution in machine-to-machine setups where YAML is being used to exchange data. The YAML parsing libraries must be tested thoroughly for their compatibility (e.g., by using a test suite like the one I created) to identify compatibility mismatches between these parsing libraries and take actions to avoid such inputs.

Extra precautions are also useful when YAML is accepted from untrusted sources. Depending on the application and parser, safeguards such as input-size limits, execution timeouts, and limits on nesting depth can reduce the impact of unusually expensive inputs. Some libraries provide configuration options for this purpose. For example, YamlDotNet provides a WithMaximumRecursion option that can be used to limit nesting depth. The appropriate mitigation ultimately depends on the parser and the environment in which it is used.

#Conclusion and Further Reading

The results presented here are only an excerpt from my master’s thesis, and there is considerably more to explore when it comes to YAML parser behavior, specification compliance, and security. If you are interested in the methodology and findings in more detail, you can read my master’s thesis .

You can inspect the results of my YAML Test Suite here ²¹. In addition, I intend to release the test suite itself so that others can reproduce the tests, extend them, and investigate compatibility between different YAML implementations with their own sets of YAML test cases.

Overall, the results show that parsing YAML is less predictable than one might expect from a serialization format with a formal specification. None of the libraries examined in this research behaved fully in accordance with the YAML 1.2.2 specification for all the tested documents, and no two libraries produced the same results across the complete test set. This does not necessarily mean that YAML itself is unsuitable for exchanging data, but it does suggest that developers should not assume identical behavior across different implementations.

The security-related findings point in a similar direction. Several of the tested parsers exhibited problematic behavior when processing unusually large or deeply nested documents, ranging from excessive processing time to stack overflows. The exact impact depends on the parser, its configuration, and the environment in which it is used. Nevertheless, applications that process YAML from untrusted sources may benefit from additional safeguards such as input-size limits, timeouts, and restrictions on nesting depth.

Perhaps the most important takeaway is therefore not that YAML should be avoided, but that its parser behavior should be treated as an implementation detail worth understanding. When YAML is used across system boundaries or with untrusted input, testing the particular libraries involved and being aware of their limitations can help avoid unexpected compatibility and security issues.