Hi!

Im running a mail script on my server but when i shall send emails to all my users like 3000 i can only send like 200 emails then it returns with a error.
Exactly the same as this example:

I am working on a job board for a client and i am trying to do a mass email to all jobseekers(there are currently over 1000 jobseekers).
When i try to send out emails to just some of the jobseekers, it works fine. But when i try to send out emails to all the jobseekers, i am getting the following error:

Anyway when i contacted my server admin of the VPS he replied:

Thank you. Yes, I understand the issue. I also had a word with our system administrators and the reason for this has been found. The maximum simultaneously running user processes allowed by the system per VPS node are as follows:

regular 480

This is a restriction we would not change from a security and performance point of view.

You can either:
- re-write your script to send the messages in pack of 150 for instance.
- find a ready-to-use script to do so.

Well so now my question is. Does anyone knows how to rewrite this email function so it can send the mails in a loop, i mean send like 50 email at one time and then send another 50 emails?

Heres the script (emailtousers.php):

<?php
require_once("functions.php");
require_once("config.php");

$readyto=(isset($_POST["readyto"])?$_POST["readyto"]:"sa");
$readyto1=(isset($_POST["readyto1"])?$_POST["readyto1"]:"sa");
$kullan=(isset($_POST["kullan"])?$_POST["kullan"]:3);
$konu=(isset($_POST["konu"])?$_POST["konu"]:"Subject!");
$mailbolumu=(isset($_POST["mailbolumu"])?$_POST["mailbolumu"]:"Mail Body!");

$count=0;
block_begin("Email to Users");
?>
<center>
<TABLE width=20% class="lista">
<TR>
<TD align="center" class="header"><? echo "E-Mail to Users"; ?></TD></TR>
<TR><TD align="center" class="blocklist">
<?php
print("<form method=post action=admincp.php?user=".$CURUSER["uid"]."&code=".$CURUSER["random"]."&do=emailtousers>");
?>
<!-- <form method=post action="admincp.php?user=<?php echo $CURUSER["uid"]; ?>&code=<?php echo $CURUSER["random"]; ?>&do=emailtousers"> -->
<!-- <form method=post action="admincp.php?user=".$CURUSER["uid"]."&code=".$CURUSER["random"]."&do=emailtousers"> -->
</TD></TR>
<tr>
<td>
<b>Subject </b><input name="konu" type="text" id="konu" size="60" maxlength="60" />
</td>
</tr>
<tr>
<td>
<b>Mail</b><br />
<textarea name="mailbolumu" cols="57" rows="5" id="mailbolumu"></textarea>
</td>
</tr>
<tr>
<td align="center" class="blocklist">
User Group : <!Dropdown added by miskotes>

         <?
         print("<select name=\"kullan\">");
         print("<option value=0".($level==0 ? " selected=selected " : "").">".ALL."</option>");
         $res=mysql_query("SELECT id,level FROM users_level WHERE id_level>1 ORDER BY id_level");
         while($row=mysql_fetch_array($res))
         {
             $select="<option value='".$row["id"]."'";
             if ($level==$row["id"])
                $select.="selected=\"selected\"";
             $select.=">".$row["level"]."</option>\n";
             print $select;
         }
         print("</select>");
         ?>

<!End dropdown>

</td>
</tr>
<TR><TD align="center" class="lista">
<input type="submit" name="readyto" value="Send Email"><input name="readyto1" type="submit" id="readyto1" value="Send Activation Mail"></td></tr></form></table><br>
</center>
<?php
if ($readyto=="Send Email") {
?>
<TABLE width=100% class=lista cellpadding=0 cellspacing=0>
<TR><TD align="center" height="20px" class="block"><b>Emailed Users USER GROUP : <? echo ($kullan==0?"ALL":$kullan) ?></b></TD></TR></table>
<center>
<?php
if ($kullan==0) {
$q=mysql_query("SELECT users.id as id, username, random, email, language, downloaded, uploaded, level, UNIX_TIMESTAMP(joined) as joined, UNIX_TIMESTAMP(lastconnect) as lastconnect FROM users LEFT JOIN users_level ON users.id_level=users_level.id ORDER BY (uploaded / downloaded) ASC");
}
else
{
$q=mysql_query("SELECT users.id as id, username, random, email, language, downloaded, uploaded, level, UNIX_TIMESTAMP(joined) as joined, UNIX_TIMESTAMP(lastconnect) as lastconnect FROM users LEFT JOIN users_level ON users.id_level=users_level.id where (users.id_level='".$kullan."') ORDER BY (uploaded / downloaded) ASC");
}
while ($user=mysql_fetch_object($q)) {
if ($user) {
echo "<b>".$user->username."</b> emailed!<br>";
mail($user->email,$konu,$mailbolumu,"From: PantheraBits <email@siteemail.com>");
   $count++;
}
}

echo "<br><br> Found <b>".$count."</b> users and emailed!</b>";

}
if ($readyto1=="Send Activation Mail") {
if ($kullan==0) {
$q=mysql_query("SELECT users.id as id, username, random, email, language, downloaded, uploaded, level, UNIX_TIMESTAMP(joined) as joined, UNIX_TIMESTAMP(lastconnect) as lastconnect FROM users LEFT JOIN users_level ON users.id_level=users_level.id ORDER BY (uploaded / downloaded) ASC");
}
else
{
$q=mysql_query("SELECT users.id as id, username, random, email, language, downloaded, uploaded, level, UNIX_TIMESTAMP(joined) as joined, UNIX_TIMESTAMP(lastconnect) as lastconnect FROM users LEFT JOIN users_level ON users.id_level=users_level.id where (users.id_level='".$kullan."') ORDER BY (uploaded / downloaded) ASC");
}
while ($user=mysql_fetch_object($q)) {
if ($user) {
//echo "www.pantherabits.com/account.php?act=confirm&confirm=".$user->random."&language=".$user->language." <br>";
echo "<b>".$user->username."</b> emailed!<br>";
mail($user->email,"activation subject","Activation link       http://www.yoursite.com/account.php?act=confirm&confirm=".$user->random."&language=".$user->language,"From: pantherabits <email@yoursite.com>");
   $count++;
}
}
echo "<br><br> Found <b>".$count."</b> users and emailed!</b>";

}
   block_end();

?>

I would be very glad if anyone could help me with this one
Thanks in advice and sorry 4 my bad gramma.

Cheers

well, it sorta do this already, but too fast.
The easiest thing to do would be to add a sleep(10) (example value) to do a 10 second pause every 200 mails, but you might run across another issue: timeout on the running page.

Basically, when you start the process, it would keep running for a long time, and your browser might be frozen.

The cleanest way would be to setup another page, that would be called via a system call (cron job, or a shell php call, rather than through the web browser) but this would man that the current page would only store the mail in a specific table, and another process would send emails automatically later on.

Like this:
1) You write the text to be sent in the emails
2) When you hit the "send" button, it don't send emails, but write those in a db table, with the recipient.
3) A simple php script will be setup like this:
table schema (done by hand and memory after 10 hours of non-stop work, take with caution and verify it. I don't work often with mysql):

create table tmpMails(
  tmpEmailId int not null auto_increment,
  sent int(1,0) default 0,
  sentDt timestamp default now(),
  dest varchar(255) not null,
  subject varchar(255) not null,
  msg text,
  primary key (tmpEmailId)
);

PHP Code:

$q="select * from tmpMails where sent=0 limit 200";
$r=mysql_query($q);
while($o=mysql_fetch_object($r)){
  mail($o->dest, $o->subject, $o->msg);
  mysql_query("update tmpMails set sent=1, sentDt=now() where tmpEmailId={$o->tmpEmailId}");
} 


And this file is called by cron every minutes.

That way, every email that ends up in this table will be sent, by grapes of 200, every minutes. It limits your throughput to 12000 emails per hours, but as long as it's ok to you, it would be sufficient.

Yes thats a great idea i must say. But i do not know how i shall intigrate it to the script so it will work. I have a script called sanity.php who runs in a interval of 1800 sec and might in someway possible add the send code into the sanity? First when hitting the "confirm/send" button it will send out 200 emails, after that when sanity runs next-time it will send another 200 emails.

In someway u need to add some "DELETE FROM tmpMails" when all messages is sent, also doing this from sanity.php in someway.

What do u think mate? Any ideas?

Cheers

In the order:
yes
yes

But... 1800 seconds are 30 minute. 200 emails each 30 minutes means 400 emails per hours, 9600 emails per day.
Are you sure your queue won't grow too much? If not, fine, go with it.
Otherwise, it's a simple matter of setting a cron job.

With a bit more details:
Once the tmpEmails table is created, simply modify the php file up there to remplace the mail() call with an insert query (around line 77):

PHP Code:

    while ($user = mysql_fetch_object($q)) {
        if ($user) {
            echo "<b>" . $user->username . "</b> emailed!<br>";
            //mail($user->email, $konu, $mailbolumu, "From: PantheraBits <email@siteemail.com>");
            $q<<<SQL

insert into tmpEmails
(dest, subject, msg)
values
('{$user->email}','$konu','$mailbolumu')

SQL;
            mysql_query($q);
            $count++;
        }
    } 


And place the code I gave you sooner in the sanitize file.
It should do it nicely.
And, in the mail sending part of sanitize, don't forget to add the "From: PantheraBits <email@siteemail.com>" as 4th parameters of the mail() call.
Or add a field in the table, and insert this parameter into the table with the other datas.

Just a last warning: as I understand it, this will make all the emails you send to be delayed, not just the bulk emails.
If you want to differentiate them, you might need a 2 button procedure.
"bulk mailing", which does what we talked now, and "normal mailing" which call the mail() function directly.

Thanks for the fast reply mate, you are great

Yes u can easy add a cron for every 2 min and just point the cron for that php file right?
Like: /home/bits/public_html/include/emailtousers.php

Yes i need to add the line like this with my email adress:

$q="select * from tmpMails where sent=0 limit 200";
$r=mysql_query($q);
while($o=mysql_fetch_object($r)){
  mail($o->dest, $o->subject, $o->msg"From: PantheraBits <email@siteemail.com>");
  mysql_query("update tmpMails set sent=1, sentDt=now() where tmpEmailId={$o->tmpEmailId}");
}  

Yes ofcourse would i have 2 buttons who is Email Directly and another is Bulk-Email for massive sendings (over 200).


Ok here's some progress in the development of the Bulk Email system. Thanks for the help

Well the sql didn't work for me so i needed to make a little change in it to this;
Sql:

CREATE TABLE `tmpMails` (
  `tmpEmailId` int NOT NULL auto_increment,
  `sent` int(10) NOT NULL default '0',
  `sentDt` timestamp NOT NULL default 'now()',
  `dest` varchar(255) NOT NULL,
  `subject` varchar(255) NOT NULL,
  `msg` text,
  PRIMARY KEY (`tmpEmailId`)
) ENGINE=MyISAM AUTO_INCREMENT=80 DEFAULT CHARSET=latin1;

Then i added the "bulk" mail buttom and added some more code in emailtousers.php

PHP Code:

<?php
require_once("functions.php");
require_once("config.php");

//global $CURUSER;
// only for security, maybe someone try to access directly to the file...
if (!defined("IN_ACP"))
    die("No direct access!");
//When hit the confirm..
if ($action=="post")
{
$readyto=(isset($_POST["readyto"])?$_POST["readyto"]:"sa");
$readyto1=(isset($_POST["readyto1"])?$_POST["readyto1"]:"sa");
$readyto2=(isset($_POST["readyto2"])?$_POST["readyto2"]:"sa");
$kullan=(isset($_POST["kullan"])?$_POST["kullan"]:3);
$konu=(isset($_POST["konu"])?$_POST["konu"]:"Subject!");
$mailbolumu=(isset($_POST["mailbolumu"])?$_POST["mailbolumu"]:"Mail Body!");

$count=0;
//Normal Email Start
if ($readyto=="Send Email") {

if ($mailbolumu =="" || $konu =="")
{
      block_begin("Email to Users");
      err_msg(ERROR,"Oops something is empty. You need to fill all the fields!");
      block_end();
      stdfoot();
      exit();
   }
?>
<?php 
block_begin("Email Sent");
?>
<TABLE width=100% class=lista cellpadding=0 cellspacing=0>
<TR><TD align="center" height="20px" class="block"><b>Emailed Users USER GROUP : <? echo ($kullan==0?"ALL":$kullan) ?></b></TD></TR></table>
<center>
<?php
if ($kullan==0) {
$q=mysql_query("SELECT users.id as id, username, random, email, language, downloaded, uploaded, level, UNIX_TIMESTAMP(joined) as joined, UNIX_TIMESTAMP(lastconnect) as lastconnect FROM users LEFT JOIN users_level ON users.id_level=users_level.id ORDER BY (uploaded / downloaded) ASC");
}
else
{
$q=mysql_query("SELECT users.id as id, username, random, email, language, downloaded, uploaded, level, UNIX_TIMESTAMP(joined) as joined, UNIX_TIMESTAMP(lastconnect) as lastconnect FROM users LEFT JOIN users_level ON users.id_level=users_level.id where (users.id_level='".$kullan."') ORDER BY (uploaded / downloaded) ASC");
}
$counter = 0;
while ($user=mysql_fetch_object($q)) {
if ($user) {
echo "<b>".$user->username."</b> emailed!<br>";
mail($user->email,$konu,$mailbolumu,"From: PantheraBits <noreply@pantheraonline.com>");
   $count++;
  if(++$counter % 10 == 0) sleep(3);
}
}

echo "<br><br> Found <b>".$count."</b> users and emailed!</b>";
block_end();
stdfoot();
}
//Normal Email Ends
//Resend Validation Email Start
if ($readyto1=="Send Activation Mail") {
if ($kullan==0) {
      block_begin("Error Mate");
      err_msg(ERROR,"Are you really sure that u want to email ALL users a NEW activation link?!");
      block_end();
      stdfoot();
      exit();
//$q=mysql_query("SELECT users.id as id, username, random, email, language, downloaded, uploaded, level, UNIX_TIMESTAMP(joined) as joined, UNIX_TIMESTAMP(lastconnect) as lastconnect FROM users LEFT JOIN users_level ON users.id_level=users_level.id ORDER BY (uploaded / downloaded) ASC");
}
else
{
$q=mysql_query("SELECT users.id as id, username, random, email, language, downloaded, uploaded, level, UNIX_TIMESTAMP(joined) as joined, UNIX_TIMESTAMP(lastconnect) as lastconnect FROM users LEFT JOIN users_level ON users.id_level=users_level.id where (users.id_level='".$kullan."') ORDER BY (uploaded / downloaded) ASC");
}
block_begin("Email Sent");
while ($user=mysql_fetch_object($q)) {
if ($user) {
echo "<b>".$user->username."</b> emailed!<br>";
mail($user->email,"activation subject","Activation link       http://www.pantherabits.com/account.php?act=confirm&confirm=".$user->random."&language=".$user->language,"From: www.pantherabits.com <noreply@pantheraonline.com>");
   $count++;
}
}
echo "<br><br> Found <b>".$count."</b> users and emailed!</b>";
block_end();
stdfoot();
}
//Ressend Validation Email Ends
//Begin Bulk Mail
if ($readyto2=="Bulk Email") {

if ($mailbolumu =="" || $konu =="")
{ 
      block_begin("Email to Users");
      err_msg(ERROR,"Oops something is empty. You need to fill all the fields!");
      block_end();
      stdfoot();
      exit();
   }
?>
<?php 
block_begin("Email Sent");
?>
<TABLE width=100% class=lista cellpadding=0 cellspacing=0>
<TR><TD align="center" height="20px" class="block"><b>Emailed Users USER GROUP : <? echo ($kullan==0?"ALL":$kullan) ?></b></TD></TR></table>
<center>
<?php
if ($kullan==0) {
$q=mysql_query("SELECT users.id as id, username, random, email, language, downloaded, uploaded, level, UNIX_TIMESTAMP(joined) as joined, UNIX_TIMESTAMP(lastconnect) as lastconnect FROM users LEFT JOIN users_level ON users.id_level=users_level.id ORDER BY (uploaded / downloaded) ASC");
}
else
{
$q=mysql_query("SELECT users.id as id, username, random, email, language, downloaded, uploaded, level, UNIX_TIMESTAMP(joined) as joined, UNIX_TIMESTAMP(lastconnect) as lastconnect FROM users LEFT JOIN users_level ON users.id_level=users_level.id where (users.id_level='".$kullan."') ORDER BY (uploaded / downloaded) ASC");
}
while ($user=mysql_fetch_object($q)) {
if ($user) {
echo "<b>".$user->username."</b> emailed!<br>";
mysql_query("INSERT INTO tmpMails (dest, subject, msg) VALUES ('{$user->email}','$konu','$mailbolumu')");

   $count++;
}
}

write_systemlog("Bulkmail added to queue, $count emails is waiting to be sent");
echo "<br><br> Found <b>".$count."</b> users and emailed!</b>";
block_end();
stdfoot();
}
//End Bulk Mail

}
//Main Block..
if($action=="write")
{
block_begin("Email to Users");
?>
<center>
<TABLE width=20% class="lista">
<tr>
<td align=center class=lista>
Welcome to Mass Email system. If u shall send more than 100 emails u shall use the
bulk mail. That will put the emails to a loop and send 100 mail per loop.
That's lower the server load alot. Use Send Email if u shall send to just
a few members ect.. or to a moderator/admin group.</td>
</tr>
<TR>
<TD align="center" class="header"><? echo "E-Mail to Users"; ?></TD></TR>
<TR><TD align="center" class="blocklist">
<?php
print("<form method=post action=admincp.php?user=".$CURUSER["uid"]."&code=".$CURUSER["random"]."&do=emailtousers&action=post>");
?>
</TD></TR>
<tr>
<td>
<b>Subject </b><input name="konu" type="text" id="konu" size="60" maxlength="60" />
</td>
</tr>
<tr>
<td>
<b>Mail</b><br />
<textarea name="mailbolumu" cols="57" rows="5" id="mailbolumu"></textarea>
</td>
</tr>
<tr>
<td align="center" class="blocklist">
User Group : <!Dropdown added by miskotes>

         <?
         print("<select name=\"kullan\">");
         print("<option value=0".($level==0 ? " selected=selected " : "").">".ALL."</option>");
         $res=mysql_query("SELECT id,level FROM users_level WHERE id_level>1 ORDER BY id_level");
         while($row=mysql_fetch_array($res))
         {
             $select="<option value='".$row["id"]."'";
             if ($level==$row["id"])
                $select.="selected=\"selected\"";
             $select.=">".$row["level"]."</option>\n";
             print $select;
         }
         print("</select>");
         ?>

<!End dropdown>

</td>
</tr>
<TR><TD align="center" class="lista">
<input type="submit" name="readyto" value="Send Email"><input type="submit" name="readyto2" value="Bulk Email"><input name="readyto1" type="submit" id="readyto1" value="Send Activation Mail"></td></tr></form></table><br>
</center>
<?php
   block_end();
   stdfoot();
}
?>

Then i added this now as a test in my sanity.php, as i can manually run it every minute i can easy test with just this code inside sanity.php. Worked just fine, i added some write_systemlog(); and a delete query to delete the sent=1.
If i shall have this runned with a Cron i need to add msql connection into the code so it knows where to connect and wich database. Maybe u can help me with that mate?

PHP Code:

//Bulk Email Start
//begin deleteing of sended bulk mails
mysql_query("DELETE FROM tmpMails WHERE sent=1");
//ends
$q="SELECT * FROM tmpMails where sent=0 limit 100";
$r=mysql_query($q);
while($o=mysql_fetch_object($r)){
  mail($o->dest, $o->subject, $o->msg,"From: PantheraBits <noreply@pantheraonline.com>");
  mysql_query("UPDATE tmpMails SET sent=1, sentDt=now() WHERE tmpEmailId={$o->tmpEmailId}");
}
$hl=mysql_query("SELECT * FROM tmpMails WHERE sent=1 ORDER BY tmpEmailId DESC LIMIT 1");
while($hs=mysql_fetch_object($hl)){
    write_systemlog("Sended Bulk Email and last id was $hs->tmpEmailId","add"); }
//Bulk Email Ends 


Cheers