Sr. Web Developer
mediabistro.com
US-NY-New York

Justtechjobs.com Post A Job | Post A Resume

Using cURL with PHP
Ian Gilfillan
cURL is one of the most powerful PHP extensions. It stands for Client URL, and allows you to communicate with other servers using a wide range of protocols. Perhaps that sounds fairly uninteresting, but give it some more thought. Other servers and other protocols? At some stage in a novice developer's career, there comes a time to break out of the local server, and cURL is the first thing you should consider. I first used cURL for quite a simple task - processing a file on an FTP server. Later for dealing with merchant payments to create a transparent way of handling credit card authentication. And later still as a convenient way to get data from Amazon.com. The possibilities are almost limitless. I haven't even scratched the surface of what cURL can do, but for certain it's a powerful library.


libcurl (the library behind the PHP cURL extension) currently supports a wide range of protocols, including HTTP, HTTPS, FTP, TELNET, FILE, LDAP, DICT, GOPHER and HTTPS, as well as HTTPS certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, and user:password authentication. There are alternatives for some of these, such as streams, sockets or the FTP extension, but cURL is the master of them all - they are less flexible, and don't perform as well.


Unfortunately cURL does not come with PHP by default. This tutorial assumes you know how to install extensions (or even better, know someone who does). PHP needs at least CURL 7.0.2-beta or higher, PHP 4.2.3, requires at least CURL 7.9.0 or higher, PHP 4.3.0 needs CURL 7.9.8 or higher and PHP 5.0.0 needs at least version 7.10.5 or greater. The PHP/cURL page will have details for more recent versions of PHP. You can read the PHP manual for more about installing Windows extensions, and extensions in a Unix environment.


A first cURL script
Let's dive right in with a simple script. 


<?php
//curl1.php

// initialise the cURL session, passing an optional URL
$ch = curl_init('http://www.example.co.za/recipient.php');

//Execute the session
curl_exec($ch);

//Close the cURL session
curl_close($ch);
?>


The recipient script is: 


<?php
// recipient.php

print "Greetings, visitor from  {$_SERVER['REMOTE_ADDR']}";
?>


The curl1.php script connects to the URL supplied (recipient.php), handing over control to that script. Not particularly useful, but it introduces you to three of the main functions: curl_init(), which returns a cURL handle (hence the results are usually assigned to a variable called $ch), curl_exec(), which executes the session you've set up, and curl_close(), which frees all resources associated with the session. curl_exec() returns a boolean in the above context, but most often you will ask it to return the output from the call, which we look at later.


A simple cURL script to look up the meaning of a word
Let's look at a more practical use. One of the protocols cURL supports is dict. In this example, we'll create a simple tool to return the definition of a word from dict.org's database, connecting to dict.org with the dict protocol. Using something similar you can create an online dictionary, or integrate a word search into other applications. This article is not about using forms to submit values to a PHP script - I assume you already know how to do that, and can build upon this skeleton, taking proper care to validate all variables. 


<html>
<body>
<form action="curl2.php" method="POST">
Please enter a word: 
<input type="text" name="word">
<input type="submit">
</form>
</body>
</html>

<?php
// curl2.php

//get the word submitted from the form
$word = addslashes($_POST['word']);

// validate the word
if ($word != '') {

    // initialise the session, this time with no URL
    $ch = curl_init();
    
    // Set the URL, which includes $word, and is of the dict protocol
    curl_setopt($ch, CURLOPT_URL, "dict://dict.org/d:($word)");
    
    // Return the output from the cURL session rather than displaying in the browser.
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    
    //Execute the session, returning the results to $definition, and close.
    $definition = curl_exec($ch);
    curl_close($ch);

    //display the results - I'll leave the formatting to you
    print "Definition of $x: $definition";
}
?>


The new function introduced here is curl_setopt(). This takes a cURL handle, an option and a value for this option as arguments. $ch is of course the handle, while we introduce two of the options, CURLOPT_URL and CURLOPT_RETURNTRANSFER. CURLOPT_URL takes the URL for the script to fetch, and can also be set by passing the argument to curl_init(), as we did in the previous example. CURLOPT_RETURNTRANSFER is also one you'll probably be using most of the time - setting it to true returns the results as a string for processing in your script, rather than just simply displaying it, as in the previous example.
We'll be introducing a few more curl_setopt() options in the following examples, but there are a little too many for one tutorial. You can read all about the others in the PHP documentation and in the cURL documentation.


Accessing password-protected pages
The next scripts assume that the example.co.za/protected directory requires a username and password to access, and demonstrate the two simple ways to pass the authentication data to the remote server. Firstly, by using the CURLOPT_USERPWD option: 


<?php
// curl3.php

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.example.co.za/protected/recipient.php');

// Username and password, separated by a colon.
curl_setopt($ch, CURLOPT_USERPWD, 'iang:iang');

curl_exec($ch);
curl_close($ch);

?>


There's also an alternative way to handle the password protection - simply put the username and password in the URL, in the same way as you can do in a browser. 


<?php
//curl4.php

$ch = curl_init();

// Username and password, always separated by a colon after the protocol, with an amphersand 
leading into the URL.
curl_setopt($ch, CURLOPT_URL, 'http://iang:iang@www.example.co.za/protected/recipient.php');

curl_exec($ch);
curl_close($ch);

?>


Returning file info from the remote server
cURL allows you to return some potentially useful information, such as the HTTP_CODE (for example, 200 for success, 403 for forbidden, 500 for internal server error. See the full list on the w3.org site), various times (time to connect, resolve the name, complete the transfer and so on), size of the download, or upload, and size of all header or requests. The full list is displayed in the sample output, below. 


<?php
//curl5.php

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://iang:iang@www.example.co.za/protected/recipient.php');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_exec($ch);

// Return the results as an associative array.
$info = curl_getinfo($ch);

// Loop through the array, showing all key=>value pairs.
foreach ($info as $key=>$value) {
    print "$key -> $value<br>";
}
curl_close($ch);
?>


curl_getinfo() with no arguments returns an associative array with all data. Below is the sample output: 


// curl5.php output

url -> http://iang:iang@www.example.co.za/protected/recipient.php
content_type -> text/html; charset=iso-8859-1
http_code -> 200
header_size -> 244
request_size -> 138
filetime -> -1
ssl_verify_result -> 0
redirect_count -> 0
total_time -> 0.004161
namelookup_time -> 5.5E-05
connect_time -> 0.000253
pretransfer_time -> 0.000336
size_upload -> 0
size_download -> 21
speed_download -> 5046
speed_upload -> 0
download_content_length -> 0
upload_content_length -> 0
starttransfer_time -> 0.004037
redirect_time -> 0
Posting variables to a remote script
The next script shows how to POST values to a remote script. This is potentially very powerful, as it allows you to script your interaction with a remote form. 


<?php
//curl6.php

// populate the array containing the values to be posted.
$postfields = array();
$postfields['field1'] = urlencode('value1');
$postfields['field2'] = urlencode('value2');

$ch = curl_init();

// Follow any Location headers
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);

curl_setopt($ch, CURLOPT_URL, 'http://www.example.co.za/recipient2.php');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

// Alert cURL to the fact that we're doing a POST, and pass the associative array for POSTing.
curl_setopt($ch, CURLOPT_POST, 1);

curl_setopt($ch, CURLOPT_POSTFIELDS, $postfields);

$output = curl_exec($ch);
curl_close($ch);
print $output;
?>


The POST fields to be sent can either be placed into an associative array, as above (in which case it's sent as multipart/form-data) or as name-value pairs, separated by amphersands (in which case it's sent as application/x-www-form-urlen-coded). Note that the former enables one way of uploading files.


The CURLOPT_FOLLOWLOCATION is another option you'll commonly use, as it ensures any Location headers are followed (as happens for example when you leave the trailing slash off an URL). The script we POST to is listed below, and simply prints the results. 


<?php
// curl7.php

foreach ($_POST as $key=>$value) {
    print "$key: $value<br>";
}
?>


The output from curl6, as expected, reads as follows: 


field1: value1
field2: value2


To give you a taste of the power of this feature, you can use it to get results from any website accepting POST variables. A Google search? Paypal balances? Here's an example that returns the Amazon results of a search for books on cURL, also demonstrating the alternative way to use POST variables. 


<?php
// curl8.php

$ch = curl_init();
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);

// Optionally set a timeout
curl_setopt($ch, CURLOPT_TIMEOUT, 30);

curl_setopt($ch, CURLOPT_URL, 'http://www.amazon.com/exec/obidos/search-handle-form/002-0565257-5012066');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "url=index%3Dbooks&field-keywords=CURL");
$output = curl_exec($ch);
curl_close($ch);
print $output;
?>


FTP'ing a file to a server
As a final example, we FTP a file to a server. No cURL tutorial would be complete without this, as it's a common use of cURL, although there are other ways of performing the same operation. 


<?php
// curl9.php

// Assign filename, and open file handle for the file we wish to send from the local server
$filename = '/path/localfile.txt';

$fh = fopen($filename, "r");

$ch = curl_init();

//URL and filename with authentication details for the FTP server
curl_setopt($ch, CURLOPT_URL, 'ftp://username:password@ftpserver.co.za/path/newfile.txt');

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

// Alert cURL for the fact that we're uploading a file
curl_setopt($ch, CURLOPT_UPLOAD, 1); 

// File to FTP
curl_setopt($ch, CURLOPT_INFILE, $fp); 

// Use ASCII mode for the FTP transfer
curl_setopt($ch, CURLOPT_TRANSFERTEXT, 1); 

// Supply the expected file size, in bytes, of the file to upload.
curl_setopt($ch, CURLOPT_INFILESIZE, filesize($filename));

$output = curl_exec($ch);
curl_close($ch);
print $output;
?>


We've only had a taste in this tutorial, but I hope you've grasped some of the potential of this powerful library, and are keen to explore further. Good luck!


Further resources