I'm not very familiar with cURL, and not sure if it's the best way to do this, but I've been trying it based on what I've read.

Basically, there's a form on a government website that calculates your legislative district that I need to spoof. (perfectly legal, the organization i'm writing this for has permission). You can submit fields with a url, because it's a GET submission. (Something like http://www.governmentwebsite.com/rep...55&city=Austin), so it's easy to submit the form from my site.
But, when you use this URL then the site takes the form info into a CGI program, and somehow redirects you to a page where it spits out the results. The results page has the information I need to get both in the page content AND in the URL, so if I could scrape either info from the page I would have what I need. But how do I do this???

I've read everything I could find about screen scraping, but if I use cURL to scrape this:
http://www.governmentwebsite.com/rep...55&city=Austin
it only pulls information from the first page, before I'm redirected to the results page. How do I follow the form submission through a redirect and then pull information from the page I end up at?? Or even, based on the above URL, get the results page URL that I will end up at after getting redirected around?

I really appreciate any help anyone can give me!!! I've been pulling my hair out for days over this problem!!

Try instantiating cURL with the option to view the headers.

Find the redirect code within the headers (or, possibly, embedded in JavaScript on the page - but unlikely).

Follow the URL referenced in the redirect code you grabbed from the headers with another cURL request.

Scrape from the results page - may need to set a higher timeout to ensure the page will have time to be generated.

Oh my god, thanks! I tried receiving headers before, but I must not have had the right combination of options set, and when I wrote the results to a database or file it would be blank. But your suggestion sent me looking in the right direction and I kept trying different things, and now it's working!!! Thanks for saving me a panic attack - I've got a deadline on Tuesday for the whole project!

Glad to be of service - keep on codin'!