This is what I’m mostly using :

I’m creating a PHP function to connect using CURL:

function get_file_curl($url,$proxy_ip,$proxy_port,$loginpassw)
{

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_PROXYPORT, $proxy_port);
curl_setopt($ch, CURLOPT_PROXYTYPE, 'HTTP');
curl_setopt($ch, CURLOPT_PROXY, $proxy_ip);
curl_setopt($ch, CURLOPT_PROXYUSERPWD, $loginpassw);
$data = curl_exec($ch);
curl_close($ch);

return $data;
}

PHP cURL functions used

  • curl_init – initializes a cURL session.
  • curl_setopt – sets and option for a cURL transfer.
  • curl_exec – performs a cURL session.
  • curl_getinfo – gets information about the last transfer.
  • curl_error – returns a string containing the last error for the current session.
  • curl_close – close a cURL session.

curl_setopt options used

  • CURLOPT_URL – the URL to scrap.
  • CURLOPT_HEADER – inlude/exclude the header?
  • CURLOPT_RETURNTRANSFER – return the transfer as a string or output it out directly? Use 1, i.e. return.
  • CURLOPT_PROXY – the HTTP proxy to tunnel request through.
  • CURLOPT_HTTPPROXYTUNNEL – tunnel through a given HTTP proxy? Use 1, i.e. tunnel.
  • CURLOPT_CONNECTTIMEOUT – it’s obvious.
  • CURLOPT_REFERER – header to be used in a HTTP request.
  • CURLOPT_USERAGENT – “User Agent:” to be used in a HTTP request.

Have fun!

0 Shares:
Leave a Reply

Your email address will not be published. Required fields are marked *

20 − 19 =

This site uses Akismet to reduce spam. Learn how your comment data is processed.

You May Also Like

Pico – Flat-File CMS

Pico is an open source CMS application that is capable of very few things. It uses flat-files as the database and built with PHP. Simply, there is no setup and the app works instantly.