Hey guys,

OK, here's what I am trying to do: I need to log into another website, and download their reports. Seems easy right? Well, it is... because I've done it with numerous other websites.

However, this one is tricky. The only way I can download the report at this website is by having the correct "key" at the end of the URL. The key is generated by their server and carries over with each link that you click on their site once logged in. Sort of like a session ID, but it's not a session ID at all.

Anyway, this key is unique to the current cookie session for that user. If they were to log out, none of those links with that unique trailing "key" would work. They would have to log back in and use the new links. The trailing key is nothing but 10 numbers long.

So, I've devised a small curl script that does everything... it logs into the site, grabs that trailing "key" from a hidden text field, and it downloads the CSV report. However, when I look at the downloaded report, it shows a "This page does not exist" custom apache page from their server.

The problem is that it is either losing the cookie when it goes to download the report URL, or it logs out/back in when it goes to download the report URL.

Here's the code (I've removed the URL's):

PHP Code:

$graburl = **URL OF CSV REPORT**;
$cookiefile = tempnam("/tmp", "cookies"); 
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $loginurl);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiefile);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiefile);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "$usernamefield=$username&$passwordfield=$password");

ob_start();
$output = curl_exec($ch);
ob_end_clean();

***************ADD CODE***************
$link = '**URL OF REPORT PAGE TO FETCH THE KEY**';
curl_setopt($ch, CURLOPT_URL, $link);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$page = curl_exec($ch);

$sk = '%<input type="hidden" id="window_name" value="(.*)">%';
preg_match_all($sk, $page, $results, PREG_PATTERN_ORDER);
***************ADD CODE***************

if ($output == '1') {
curl_setopt($ch, CURLOPT_URL, "$graburl&session_key=".$results[1][0]);
$putcsv = "report.csv";
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiefile);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiefile);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
file_put_contents("**MY URL**".$putcsv."", "$result");
}
curl_close($ch); 


Note: This code works beautifully on other sites where I get reports from as well. It's just the extra code marked in "**ADD CODE**" and the "&session_key=".$results[1][0]" added to the graburl that is giving me problems.

It may have something to do with grabbing to URL's in the same curl session, I can't figure it out.

Any help is appreciated,

Thanks,
Corey

Just an update:

I've tried it with just one instance of the cURL cookie options and I've tried it with init(). EX:

PHP Code:

$ch = curl_init ("http://somedomain.com/"); 


Neither worked.

If any one has any suggestions or knows a better to do this, let me know.

Thanks,
Corey

I had a similar problem once.
After logging in to the site, and then visiting another page, it wouldn't stay logged in.

I fixed this by including a PHPSESSID=something at the end of each link I visited.

Originally Posted by Insensus

I had a similar problem once.
After logging in to the site, and then visiting another page, it wouldn't stay logged in.

I fixed this by including a PHPSESSID=something at the end of each link I visited.

Thanks for the reply.

I tried that and it didn't work. The thing is that it's not a PHP session that is being carried. It's some sort of key that comes from the server, almost like a session, but not a PHPSESSION.

Right now I'm at the point where I can login, go to the page where the reports are and grab the key but the problem is that I can't write the result because

PHP Code:

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 


is at 1 instead of 0.

At 1 it puts the result into a string so that it can grab out the session key but at 0 it can't grab the key but can write to the CSV file, so I'm trying to figure out a solution to that.

If you have any ideas let me know.

Thanks,
Corey