Piping Emails to CakePHP

Have you ever needed, or wanted, to pipe emails to a CakePHP application for automatic processing?  While there seems to be little information out there on how to do this, it turns out that it's quite easy in fact.

If you need to handle incoming emails in your CakePHP application, to process a support ticket or accept blog posts by email for example, then piping those emails to CakePHP is likely the way you would want to do so.  There is not a lot of information out there on how to set this up or how to pass that incoming email to CakePHP for processing.  However, with just a few steps and one 3rd party library you can have this up and running in no time.

Note: This article pertains to CakePHP v1.3.x so your mileage may vary with other versions of CakePHP.  This article is also targeted at a Unix/Linux server environment with Apache.  You may need to make some adjustments depending on your particular server environment.  This has not been tested with a Windows server and, as I don't use Windows servers regularly, I am unable to offer any advice on how to do this with a Windows server.

Getting Started

You will need the following items to set this up:

The first step, natch, is to get your CakePHP app setup and operating as desired.  If you are new to CakePHP I highly recommend that you start by reading the "CookBook" (the CakePHP documentation) at http://book.cakephp.org which will introduce you to CakePHP and walk you through setup and usage.  When you have your app setup and everything is running smoothly then you can proceed to add the email piping support to your app.

Download and Install the MIMEParser class files

If you haven't already, you will now need to download the MIMEParser PHP class and "install" it into your CakePHP app.  Installing the class is nothing more than copying the MIMEParser directory / files into the "/app/vendors/" directory (I placed the files in "/app/vendors/mimeparser/").  There are a few extra files, such as the class documentation, in the download archive but you only need the following two files for this process:

  1. mime_parser.php - This is the "workhorse" for the class and handles most of the email parsing functionality
  2. rfc822_addresses.php - This provides some additional functionality for handling email address according to the RFC822 specification.

Simply place those files into the "mimeparser" (or whatever you prefer to name it) directory in "/app/vendors/" and it is "installed".

Preparing CakePHP

When using CakePHP from the command line interface ("cli") you may run into some issues with the "path" environment variable on the server.  This is a special system setting which tells the server where to search for files / commands.  If you understand what that means, great!  If not, don't worry about it, you don't need to know how that bit works for the purposes of this article.

In either case, you will want to setup a "shell script" to take care of the necessary bits for you.  This is described in the CakePHP manual in the "Running Shells as cronjobs" section.  As directed in that article you will need to create a file with the following contents (don't worry if you're not sure what all this code means):

  1. #!/bin/bash
  2. TERM=dumb
  3. export TERM
  4. cmd="cake"
  5. while [ $# -ne 0 ]; do
  6. if [ "$1" = "-cli" ] || [ "$1" = "-console" ]; then
  7. PATH=$PATH:$2
  8. shift
  9. else
  10. cmd="${cmd} $1"
  11. fi
  12. shift
  13. done
  14. $cmd

Save this code in a file named "cakeshell" (note there is no extension on this filename) in the "/app/vendors" directory.  You will also need to make sure that this file is "executable" on the server (i.e. "chmod +x") so that it can be executed when the email is piped in (which we'll cover later in this article).  You may also need to make the "cake" shell itself (found in the "/cake/console/" directory) executable for this to work.

If you are using a single installation of CakePHP you may find it handy to use the following, modified, version of the above code which has the "-cli" and "-console" parameters hard-coded so you don't have to pass them in the piping command (discussed later).

  1. #!/bin/bash
  2. TERM=dumb
  3. export TERM
  4. cmd="cake"
  5. # adjust the paths below as necessary for your server
  6. PATH=$PATH:/usr/bin:/home/username/public_html/cake/console
  7. while [ $# -ne 0 ]; do
  8. cmd="${cmd} $1"
  9. shift
  10. done
  11. $cmd

As noted, be sure to adjust the "/usr/bin" and path to the CakePHP console directory as needed.  Either version of the above code will do exactly the same thing and the only difference is that the modified version allows you to use a shorter pipe command when setting up the email piping (discussed later).

Creating the Shell

The next step will be to create the CakePHP Shell to actually handle the incoming email and parse it into a format which is more readily usable in your app.  If you are not familiar with using and creating a CakePHP Shell, then you should start by reading the documentation on The CakePHP Console and Creating Shells & Tasks which will walk you through using the CakePHP console and creating your own CakePHP Shell.

The code for the shell is rather simple as most of the "heavy lifting" is done by the MIMEParser class for us.  The code for the shell is:

  1.  
  2. class MailShell extends Shell {
  3. /* Define the available tasks for this shell */
  4. public $tasks = array('EmailParser');
  5.  
  6. /**
  7.   * Holds the raw email received from stdin.
  8.   *
  9.   * @access private
  10.   */
  11. private $rawemail = null;
  12.  
  13. /**
  14.   * The email address that the message was sent from
  15.   *
  16.   * @access private
  17.   */
  18. private $sender = '';
  19.  
  20. /**
  21.   * The HTML content of the email
  22.   *
  23.   * @access private
  24.   */
  25. private $htmlBody = '';
  26.  
  27. /**
  28.   * The plain text content of the email
  29.   *
  30.   * @access private
  31.   */
  32. private $textBody = '';
  33.  
  34. public function __construct(&$dispatch) {
  35. parent::__construct($dispatch);
  36. }
  37.  
  38. /**
  39.   * Disable the Welcome message output so that no 'errors' are bounced back.
  40.   * Any form of output from the script (e.g. the shell 'Welcome' message)
  41.   * is interpreted by the MTA as an error and is bounced back to the sender.
  42.   *
  43.   * @access public
  44.   */
  45. public function _welcome() {}
  46.  
  47. /**
  48.   * The primary shell process
  49.   *
  50.   * @access public
  51.   * @return void
  52.   */
  53. public function main() {
  54. /* Read in the email content that was piped in */
  55. $mailstream =  fopen("php://stdin", "rb");
  56. $this->rawemail = stream_get_contents($mailstream);
  57. fclose($mailstream);
  58. if($this->rawemail !== false) {
  59. if($this->parseMail()) {
  60. // Do something with the mail here
  61. // The email content is available at
  62. // $this->sender - The senders email address
  63. // $this->textBody - The text content of the email
  64. // $this->htmlBody - The HTML content of the email
  65. }
  66. } else {
  67. $this->log('[' . __METHOD__ . '] No email received or failed to retrieve email contents', 'debug');
  68. }
  69. }
  70.  
  71. /**
  72.   * Parse the incoming email.
  73.   *
  74.   * @access private
  75.   * @return bool True on success, False on failure
  76.   */
  77. private function parseMail() {
  78. $parsed = $this->EmailParser->parse($this->rawemail);
  79. if($parsed !== false) {
  80. $this->sender = (isset($parsed['sender'])) ? $parsed['sender']:'';
  81. $this->textBody = (isset($parsed['text'])) ? $parsed['text']:'';
  82. $this->htmlBody = (isset($parsed['html'])) ? $parsed['html']:'';
  83. } else {
  84. $this->log('[' . __METHOD__ . '] Mail Parsing Failed: '.print_r($parsed, true), 'debug');
  85. }
  86. return $parsed !== false;
  87. }
  88. }
  89.  

Save this code in a file named "mail.php" (or whatever you wish ... just be sure to follow conventions and change the class name to match the filename you use) in the "/app/vendors/shells/" directory.  This will handle the following:

  • Receive the mail stream and check to make sure that something was received
  • Pass the email data to the "EmailParser" task (discussed later) to perform the actual parsing of the mail.
  • Allow you to take any necessary action with the email data
  • Log some notices to the CakePHP debug log if something goes wrong

What you actually do with the mail data once it's parsed is entirely dependent upon your intended usage and is beyond the scope of this article.  You could do something like breaking the email into individual lines and then loop through those to pull in data for processing or validate the sender address against a database.  You're only limited by your imagination really.

Creating the Shell Task

Now that we can receive the email and hand it over to the parser, we need to be able to get that data to the actual parser class.  This is accomplished by creating a Shell Task.  You could skip the shell task and embed the processing code into the shell itself (in the "parseMail" function) but creating a task has a few advantages:

  1. It follows the convention for shells.  Shells are for handling the logic (similar to a Controller) and the Tasks are for performing discreet, well, "tasks".
  2. Creating a task gives us functionality that can be easily re-used in another shell or multiple functions within the same shell

The code for the Task is also relatively simple, once again thanks to the MIMEParser class, and will handle interacting with the parser and pulling out the relevant sections of the email from the parsed data.  It will also log some information to the debug log in the event that something fails.  The code for the task is:

  1.  
  2. /** Import the vendor classes */
  3. App::import('Vendor', 'rfc822_addresses', array('file'=>'mimeparser'.DS.'rfc822_addresses.php'));
  4. App::import('Vendor', 'mime_parser', array('file'=>'mimeparser'.DS.'mime_parser.php'));
  5. class EmailParserTask extends Shell {
  6. /**
  7.   * Holds the decoded email data
  8.   *
  9.   * @access private
  10.   */
  11. private $decodedemail = null;
  12.  
  13. /**
  14.   * The email address that the message was sent from
  15.   *
  16.   * @access private
  17.   */
  18. private $sender = '';
  19.  
  20. /**
  21.   * The HTML content of the email
  22.   *
  23.   * @access private
  24.   */
  25. private $htmlBody = '';
  26.  
  27. /**
  28.   * The plain text content of the email
  29.   *
  30.   * @access private
  31.   */
  32. private $textBody = '';
  33.  
  34. /**
  35.   * Warnings generated during the email parsing
  36.   *
  37.   * @access private
  38.   */
  39. private $warnings = null;
  40.  
  41. /* Standard shell function */
  42. public function execute() {}
  43.  
  44. /**
  45.   * Parse the provided email message
  46.   *
  47.   * @param $mail Raw email message to parse
  48.   * @return mixed Array of decoded email parts (sender, text, html) or False on failure or if there is nothing to parse
  49.   */
  50. public function parse($mail = null) {
  51. $parsed = true; // return value
  52.  
  53. /* Quit if there is no raw email data to parse */
  54. if(is_null($mail) || $mail === false) { return false; }
  55.  
  56. /* Instantiate MIME parser */
  57. $mime = new mime_parser_class;
  58.  
  59. /* Set to 0 for parsing a single message file */
  60. $mime->mbox = 0;
  61.  
  62. /* Enable decoding of the message bodies */
  63. $mime->decode_bodies = 1;
  64.  
  65. /* Set to 0 to make syntax errors make the decoding fail */
  66. $mime->ignore_syntax_errors = 1;
  67.  
  68. /* Set to 0 to avoid keeping track of the lines of the message data */
  69. $mime->track_lines = 1;
  70.  
  71. /* Setup the MIMEParser parameters */
  72. $parameters=array(
  73. /* Read a message from a string instead of a file */
  74. 'Data'=>$mail,
  75. /* Get all the message body parts */
  76. 'SkipBody'=>0,
  77. );
  78.  
  79. /* Try to decode the email */
  80. if(!$mime->Decode($parameters, $decoded)) {
  81. $errmsg = '[' . __METHOD__ . '] MIME Decode Error: ' .$mime->error.' at Position: '.$mime->error_position;
  82. if($mime->track_lines && $mime->GetPositionLine($mime->error_position, $line, $column)) {
  83. $errmsg .= ' Line: '.$line.' Column: '.$column;
  84. }
  85. $this->log($errmsg, 'debug');
  86. $parsed = false;
  87. } else {
  88. /* Only accept single messages */
  89. if(count($decoded) == 1) {
  90. $this->decodedemail = $decoded[0];
  91. $analyzed = array();
  92. if(!$mime->Analyze($decoded[0], $analyzed)) {
  93. $parsed = false;
  94. $this->log('[' . __METHOD__ . '] Message analysis failed.', 'debug');
  95. } else {
  96. /* set the analyzed message data as the return value */
  97. $parsed = $analyzed;
  98. }
  99. } else {
  100. $parsed = false;
  101. $this->log('[' . __METHOD__ . '] Multiple messages detected in stream. This is not supported. Aborting parse.');
  102. }
  103.  
  104. /* Log any warnings that occurred during the decoding process */
  105. if(count($mime->warnings)) {
  106. $this->warnings = $mime->warnings;
  107. for($warning = 0, Reset($mime->warnings); $warning < count($mime->warnings); Next($mime->warnings), $warning++) {
  108. $w = Key($mime->warnings);
  109. $errmsg = '[' . __METHOD__ . '] Warning: '. $mime->warnings[$w]. ' at Position: '. $w;
  110. if($mime->track_lines &amp;&amp; $mime->GetPositionLine($w, $line, $column)) {
  111. $errmsg .= ' Line: '.$line.' Column: '.$column;
  112. }
  113. $this->log($errmsg, 'debug');
  114. }
  115. }
  116. return $parsed;
  117. }
  118. }
  119. }
  120.  

Save this in a file named "email_parser.php" in the "/app/vendors/shells/tasks/" directory.  Once again, you can change the name of the file / class if you wish but be sure to also change the corresponding names in the MailShell class.

Setting up the email piping command

We're almost home.  All that is left to do at this point is to setup the email piping command so that when an email is received it is handed over to CakePHP for processing by our new Mail shell.

How the email piping is setup will depend entirely on your particular server setup and is beyond the scope of this article.  If you're not sure how to set up email piping, contact your server administrator or hosting service support to request assistance with this process.

The first thing you will need for this part is an email address that will serve as the recipient for the emails you wish to pipe to CakePHP.  I recommend creating a new email address which will be used solely for this purpose.  You can use any (valid) email address you want but it is best not to re-use an email address for multiple functions as it may create some unexpected behavior, especially if you are performing multiple unrelated tasks (e.g. processing a hotel reservation and accepting support tickets by email).  Once this is setup, any email sent to the address will be handed over to CakePHP and will not be held in the mailbox.

The pipe command you use will depend on which version of the "cakeshell" code (discussed above in the "Preparing CakePHP" section) you chose to use.  If you used the default version as specified in the CakePHP book, then you will need to use the following pipe command (be sure to adjust the paths as necessary for your setup):

  1. | /home/example/public_html/app/vendors/cakeshell mail -cli /usr/bin -console /home/example/public_html/cake/console -app /home/example/public_html/app

If you chose to use the modified version you would, instead, use this pipe command:

  1. | /home/example/public_html/app/vendors/cakeshell mail -app /home/example/public_html/app

Both will accomplish the same thing. The second method is just less verbose.  I prefer the second method but that is just my preference.

If something fails during the piping process there will, usually, be a "delivery failure" email returned which will contain some information about the particular failure which may be helpful in troubleshooting any problems you may encounter.

Conclusion

You should now have everything in place to pipe emails into your CakePHP application.  I have used this in several CakePHP apps and have not, yet, run into any major issues.  Most of this should be readily portable to just about any server setup that supports CakePHP but it can be a slightly tricky the first time you try it.  A few pointers to keep in mind that may help with some of the initial hurdles are:

  • Ensure that your "cakeshell" (created above in the "/app/vendors/" directory) as well as the "cake" shell itself (in the "/cake/console/" directory) are executable
  • Ensure that the MIMEParser class files are located in the "/app/vendors/mimeparser/" directory (or whatever you chose to name that directory).
  • Ensure that the "MailShell" class file is located in the "/app/vendors/shells/" directory
  • Ensure that the "EmailParserTask" class file is located in the "/app/vendors/shells/tasks/" directory
  • Ensure that the email piping command is setup correctly for your server environment.