Ok, I just don't understand how the pointer is gotten? If the pointer is like an array key that allows us to find that part in the file......Or, wait, is it more like the pointer just say's this is the file to work with but when you fgets then your opening the file into the memory?

The file pointer, as I understand it from C programming, is a pointer to a specific position in a file. When you read, it reads from the position held in the pointer, and when you write, you write from the position in the pointer (which is why we can test for EOF with feof() and have different write modes that place the pointer at the beginning or end of the file). The pointer itself doesn't hold any of the file data, it is, as you put it, "this is the file to work with".

When you use another function, like fgets, then you are reading X bytes from the pointer onwards into a variable (ie. memory).

For small and even moderately sized files the file() function is a whole lot easier tho. Combine that with the map_array() function and you're laughing.

Originally Posted by cptnwinky

Hey Oberon...Just for future reference this whole block of code...

PHP Code:

$fp = fopen("test.txt","r");

while(!feof($fp)) {
  $line=fgets($fp,1024);
  echo $line ."<br />";
}

fclose($fp); 


Can be replaced with this....

PHP Code:

$file = file('test.txt');
foreach($file as $r) {
    echo $r . '<br />';
} 


Originally Posted by cptwinky

So either way if you have a 2gb file it's going to be using all the memory and start paging to the hard drive.

No.

The first example will read 1024 bytes at a time, and only 1024 bytes (give or take a few more for other stuff not relevant).
It's reading the data, using it, then overwriting it on the next loop, do you see ? Doing it this way also allows a whole host of other options, seeking half way through the file so you don't have to read the whole thing in, stop reading the file once you've found what you want, etc. The second example won't allow this.

The second example, copies the whole file to memory, then starts looping it. And you probably won't be able to do that due to PHP's memory_limit ini option being exceeded.

So now you see my point ?


It seems you do not have a thorough enough understanding of fopen. The best place to clear this up is the PHP manual;
http://uk2.php.net/manual/en/function.fopen.php
http://uk2.php.net/manual/en/ref.filesystem.php

I got it tress, you tought me something I did not know before. Just please, try not to be condescending about it, use a smily in your post or something.

Well this is definitely a learning experience. Speaking of file sizes mine will be of a moderate size. And I stiil need help removing the zeros from '00000014'
I need something that will remove the zeros only the zeros leading up to the real number.
Number of zeroes leading up to that point could be any number in length.

Read this. The section "Converting to integer" may be of particular interest.

excellent! I knew there had to be an easy way of converting the str to int.

grr wrong topic. will tech me for having two windows open of the same site!! lol

OK my friends now for a toughy:
I want to increment a value associated with another value in a table.row
The script parses the flat file it checks each value in it against the values in the current db, if they already exist then it needs to add 1 to the number of times it has appeared. Actually it needs to increment two different values.
Any help?

UPDATE
Here is what I tried: (Surprised the server didn't explode!!)

PHP Code:

 if ($a>0) {
        $table = mysql_query("SELECT valid_cust_code FROM codes");
            $check=substr($fcode, 0, 11);
            while ($row = mysql_fetch_assoc($table)) {
            $dataBaseValue=stripslashes($row['valid_cust_code']);
            $currentTotal=(int)($row['current_points_earned']);
            $totalEarned=(int)($row['total_points_earned']);
                if ($dataBaseValue === $check) {
                $currentTotal++;
                $totalEarned++;        
                $sql = "INSERT INTO codes (current_points_earned, total_points_earned) VALUES ('$currentTotal','totalEarned')";
                mysql_query($sql);
                }
    else 
        {
        $currentTotal=1;
        $totalEarned=1;        
        $sql = "INSERT INTO codes (valid_cust_code, current_points_earned, total_points_earned) VALUES ('$fcode','$currentTotal','totalEarned')";
        mysql_query($sql);
        }
}
} 


Here is what I tried: This adds 14 values from a flat file 8,192 times. It checks each value against every value in the table and instead of updating it it just adds a new row. I'm frustrated and can't seem to figure out what I'm doing wrong.

PHP Code:

 while(!feof($fh)) {
$line = fgets($fh,1024);
list($fcode) = explode("/n", $line);
$a = (int)substr($line, 0, 1);
   if (substr($fcode, 0, 4) === "HEAD") {
      $sql = "INSERT INTO parsed_files (header_id) VALUES ('$fcode')";
      mysql_query($sql);
      }
   if ($a>0) {
        $check=substr($fcode, 0, 11);
        $table2 = mysql_query("SELECT valid_cust_code FROM codes");
            $row = mysql_fetch_assoc($table2);
            if ($row == NULL) {
            $currentTotal=1;
            $totalEarned=1;
            $sql = "INSERT INTO codes (valid_cust_code,current_points_earned,total_points_earned) VALUES ('$fcode','$currentTotal','$totalEarned')";
            mysql_query($sql);
            }    

        $table2 = mysql_query("SELECT valid_cust_code FROM codes");
            while ($row = mysql_fetch_assoc($table2)) {
            $dataBaseValue=stripslashes($row['valid_cust_code']);
            $currentTotal=stripslashes($row['current_points_earned']);
            $totalEarned=stripslashes($row['total_points_earned']);
                if ($dataBaseValue === $check) {
                $currentTotal++;
                $totalEarned++;
                $sql = "UPDATE codes SET current_points_earned = ".$currentTotal.", total_points_earned = ".$totalEarned." WHERE $dataBaseValue=$check";
                mysql_query($sql);
                }
        else 
            {
            $currentTotal=1;
            $totalEarned=1;
            $sql = "INSERT INTO codes (valid_cust_code,current_points_earned,total_points_earned) VALUES ('$fcode','$currentTotal','$totalEarned')";
            mysql_query($sql);
            }    
        }
    }
} 


Hallmarc,

What undesirable result are you getting? What flat file are you using for processing?

What happens when you try the above code? Do you get a php error or do the MySQL parts just not have an effect? If it's the latter case, try adding an or die() to your sql:

PHP Code:

$result= mysql_query("SQL QUERY HERE") or die(mysql_error()); 


Then if there is an error in your SQL you will see it in your browser when you run the script.

it should only add 14 rows to the table not 8,192. it is a .dat file. these values should only be added if they don't already exist in the table. If they already exist then both count values (current_earned_points, total_earned_points) should be incremented by one respectively.

I echo'ed the results from the script and it is adding them to the table 2,4,8,16,32 times until it gets through all 14 values in the file.

One step closer. I have it incrementing the values, yet ti still keeps adding unnecessary new rows.

PHP Code:

while(!feof($fh)) {
$line = fgets($fh,1024);
list($fcode) = explode("/n", $line);
$a = (int)substr($line, 0, 1);
   if (substr($fcode, 0, 4) === "HEAD") {
      $sql = "INSERT INTO parsed_files (header_id) VALUES ('$fcode')";
      mysql_query($sql);
      }
   if ($a>0) {
        $check=substr($fcode, 0, 11);
        $table2 = mysql_query("SELECT valid_cust_code FROM codes");
            $row = mysql_fetch_assoc($table2);
            if ($row == NULL) {
            $currentTotal=0;
            $totalEarned=0;
            $sql = "INSERT INTO codes (valid_cust_code,current_points_earned,total_points_earned) VALUES ('$fcode','$currentTotal','$totalEarned')";
            mysql_query($sql);
            }    

        $table2 = mysql_query("SELECT code_id,valid_cust_code,current_points_earned,total_points_earned FROM codes");
            while ($row = mysql_fetch_assoc($table2)) {
            $dataBaseValue=stripslashes($row['valid_cust_code']);
            $updateIdentifier=($row['code_id']);
            $currentTotal=stripslashes($row['current_points_earned']);
            $totalEarned=stripslashes($row['total_points_earned']);
                if ($dataBaseValue === $check) {
                $currentTotal++;
                $totalEarned++;
                echo $totalEarned;
                $sql = 'UPDATE codes SET current_points_earned = '.$currentTotal.', total_points_earned = '.$totalEarned.' WHERE code_id = '.$updateIdentifier.' LIMIT 1'; 
                mysql_query($sql);
                }
        else 
            {
            $currentTotal=1;
            $totalEarned=1;
            $sql = "INSERT INTO codes (valid_cust_code,current_points_earned,total_points_earned) VALUES ('$fcode','$currentTotal','$totalEarned')";
            mysql_query($sql);
            }    
        }
    }
} 


Merged your two relevant threads.

Without running the code, it would appear that you have INSERT operations in both conditional branches of the code when checking whether $a>0. You have significantly nested logic which I would recommend you break up into functions -- it would vastly increase the readability of your code.

Once that is done, put some debug before the INSERT operations providing a description of the use case (conditions by which you reached that point of the code) and an indication of the INSERT to be performed. That will clearly illustrate the point at which you are performing the extra INSERTs.

PHP Code:

   if ($a>0) { 
        $check=substr($fcode, 0, 11); 
        $table2 = mysql_query("SELECT valid_cust_code FROM codes"); 
            $row = mysql_fetch_assoc($table2); 
            if ($row == NULL) 
           { 
            $currentTotal=0; 
            $totalEarned=0; 
            $sql = "INSERT INTO codes (valid_cust_code,current_points_earned,total_point  s_earned) VALUES ('$fcode','$currentTotal','$totalEarned')"; 
            mysql_query($sql); 
            }     

        $table2 = mysql_query("SELECT code_id,valid_cust_code,current_points_earned,tota  l_points_earned FROM codes"); 
            while ($row = mysql_fetch_assoc($table2)) 
            { 
            $dataBaseValue=stripslashes($row['valid_cust_code']); 
            $updateIdentifier=($row['code_id']); 
            $currentTotal=stripslashes($row['current_points_earned']); 
            $totalEarned=stripslashes($row['total_points_earned']); 
                if ($dataBaseValue === $check) { 
                $currentTotal++; 
                $totalEarned++; 
                echo $totalEarned; 
                $sql = 'UPDATE codes SET current_points_earned = '.$currentTotal.', total_points_earned = '.$totalEarned.' WHERE code_id = '.$updateIdentifier.' LIMIT 1'; 
                mysql_query($sql); 
                } 
        else 
            { 
            $currentTotal=1; 
            $totalEarned=1; 
            $sql = "INSERT INTO codes (valid_cust_code,current_points_earned,total_point  s_earned) VALUES ('$fcode','$currentTotal','$totalEarned')"; 
            mysql_query($sql); 
            }     
        } 


the first insert is only run once, ever. After that is all down hill. I agree on breaking up the code into functions. The way it is now, there are too many instances where $dataBaseValue !== $check and the else block is used. Just haven't figured out how to do this yet.