How to Set a POST Variable From a File When Using Curl

If you read through the curl man page, you’ll learn all about using the -d or --data options (and their variants ) to send a POST request. For example, you can:

• Specify form fields and their values as key/value pairs in the form <key>=<value>
• Set the request’s body from the contents of a file, by prefixing the name of the file with the at symbol @
• have curl URL encode a value by prefixing it with =, such as key==value.

But, how do you set the value of a POST parameter from a file’s contents? That’s something I’ve been struggling with – up until today.

To show what I mean, let’s say that I’m trying to test a small, simple contact form in one of my web apps. The form is nothing special. It just lets anyone send a message, along with their contact details so that it can be replied to with a bit of personalisation.

The form has three fields:

1. A text field for the sender’s name, named name
2. A text field for the sender’s email address, named email
3. A text area for the message, named message

If your message was quite short and not grammatically complex, you could run the following command:

curl \
    --data "name=Matthew Setter" \
    --data "email=matthew@example.org" \
    --data "message=Here is my message to your business" \
    https://localhost:8080/form

Here, the command uses the --data parameter for each of the three form fields. Nothing special. Sure.

But, what if the message was quite long and detailed? Hey, some of us feel the need to write War and Peace sometimes. To attempt to send such a message directly from the command line would be just asking for trouble – unless you were a command-line guru, which very few of us are.

A more sane (not to mention practical) choice would be to save the message to a text file, then set the value of the message parameter from the contents of the file. The question, though, is how?

It’s not all that clear from the curl documentation (well, not to me before quite some searching, anyway). However, there are two ways to do so.

Use the data-urlencode option

The first approach is to use the --data-urlencode option. With this option, you specify the parameter in the form key@filename. Quoting the documentation:

This makes curl load data from the given file (including any newlines), URL-encode that data and pass it on in the POST. The name part gets an equal sign appended, resulting in name=urlencoded-file-content.

Let’s assume that you saved your message in a file named message.txt in a directory named data, in the same directory where you run the curl command from. You could then run the following command:

curl \
    --data "name=Matthew Setter" \
    --data "email=matthew@example.org" \
    --data-urlencode message=@data/message.txt \
    http://localhost:8080/form

Note: you cannot use the --data or -d options.

Use a curl variable

The other far more interesting way (to me at least), is by using a curl variable, setting the variable’s contents from the file’s contents, then using the --expand-data option.

Again, quoting the curl documentation;

The contents of a variable can be referenced in a later command line option when that option name is prefixed with “–expand-”, and the name is used as “{{name}}” (without the quotes).

Here’s how we’d do that:

curl \
    --variable message@data/message.txt \
    --data "name=Matthew Setter" \
    --data "email=matthew@example.org" \
    --expand-data message={{message}} \
    http://localhost:8080/form

The command starts by setting a variable named message with the contents of data/message.txt. then, on the second last line, using the --expand-data option, it uses the contents of that variable as the value for the message parameter.

Use the -F and --form options

This one should (perhaps) have been more obvious (as I was reading through the documentation) as:

…this (-F and --form) lets curl emulate a filled-in form in which a user has pressed the submit button. This causes curl to POST data using the Content-Type multipart/form-data according to RFC 2388.

Given that, you can use -F and --form to upload a file, similar to the --data-urlencode option, which was introduced earlier. However, when using these options we need to prefix the value with <, not @.

curl \
    --form "name=Matthew Setter" \
    --form "email=matthew@example.org" \
    --form "message=<data/message.txt" \
    https://localhost:8080/form

If you prefix the value with @ when using -F or --form, then you’ll upload the file itself, not its contents.

And, you can also use read the key’s contents from STDIN by using - instead of a file path, as in the following example:

cat data/message.txt | curl \
    --form "name=Matthew Setter" \
    --form "email=matthew@example.org" \
    --form "message=<-" \
    https://localhost:8080/form

It’s a little more complicated, and only works on UNIX/Linux systems as far as I know, but it’s worth being aware of.

That’s how to set a POST request parameter with the contents of a file

Perhaps it’s just me who didn’t know how to do this, or was too impatient to read the curl documentation slowly and carefully enough. However, if you’ve been wanting to know how to test a POST request with curl, and set one or more of the form parameters from the contents of a file, now you know three ways to do so.