The Reclaim Team has recently brought back a professional development training session each week where larger questions and concepts are discussed. This gives us a chance to bring up any pain points that have been experienced over the last week in a place where everyone is devoted to learning for an hour. We have a #profdev slack channel where we make notes like Troubleshooting Slow Sites or Updating WHMCS, and then everyone agrees on an available time to meet. It’s been a great system and already proving to be very helpful.
The other week, our Professional Development session was focused on Troubleshooting Email. Configuring email can be tough as it is, so troubleshooting email can be a real bear. I’ve included my notes from the meeting below, as I’m planning on coming back to reference them. :) And if I can benefit from having these published, then maybe someone else can, too!
Goals for Troubleshooting:
-Look for/ Request IP address. Is it blocked?
-Request User’s email client settings
-Search for error messages
-Gathering as much information as possible using the tools below
Features & Tools in cPanel
Where you can create email addresses, view existing ones, and access webmail on the server. If you can view certain emails from within the webmail client but the user can’t receive email on their local client, this is usually an issue with their Email Configuration Settings.
This allows you to send a copy of any incoming email from one address to another so that you only have one inbox to check. You can also use this section to forward all email accounts from one domain to another. If forwarders aren’t set up properly, this may be a reason for why email isn’t being relayed to the expected location.
If you’re planning on using Google suite and another third-party mail client, you MUST set your email routing settings to Remote Mail Exchanger.
Set up automated emails to go out if your inbox receives mail. These are your ‘Out of Office’, etc. responses.
Default Address Maintenance
This feature will “catch” emails that are sent to an invalid email address for your domain. So for instance, if someone sends an email to firstname.lastname@example.org, I ordinarily wouldn’t receive it because I don’t have that email address set up. This feature allows you to say, “If anyone sends mail to an email address with my domain name in it, regardless if it’s been set up or not, forward it here.”
Reclaim Hosting’s default setting is to discard these emails since they’re generally loaded with spam. So if a user has this setting turned on and then complains of an excess of spam, this could likely be the smoking gun.
Track Delivery & Apache SpamAssassin
This is our go-to, one-stop shop for troubleshooting. This is where we as admins or the user can go to see what happened on the server. You can see successful email activity, failed sent emails, and deferred emails (meaning: emails are put on a queue to retry sending again). Time stamps, spam score ratings*, and event details (sender IP, user, router, etc.) are also available as well.
*Apache SpamAssassin is turned off by default, but can be enabled within the icon. It rates all incoming mail and filters what it defines as spam. SpamAssassin defines what spam is by rating it based on a set up specified rules. Everything over a 5+ rating is sent to the spam folder. You can modify the rating number and auto-delete spam under the Apache spam assassin icon.
Global Email Filters
We’ve received tickets in the past where folks had spam coming through so the users wanted to set a filter to automatically delete specific emails. cPanel has guides on this that can be found here.
Authentication & Calendar
Honestly, Reclaim Hosting isn’t all too familiar with this feature, but cPanel has written guides on setting it up here. There are so many better alternatives for managing contacts and calendars, so we rarely, if any at all, get requests for folks wanting to use their cPanel for this.
This is not that useful in our opinion. BoxTrapper “protects” you from spam by requiring all email senders not on your Whitelist confirm their identity before you can receive their mail. While that sounds great in theory, confirmation emails with a ‘noreply@’ email address would never make it to your inbox since there’s no one on the other end managing the address. This will also double your email incoming/outgoing quota.
Email disk usage
Allows you to view which email folders (inbox, spam, trash, etc.) are taking up the most space, and gives you the option to delete mail in bulk.
Email Tools in WHM
Mail Delivery Reports
This gives you the status of emails from all accounts on the server. It’s very similar to cPanel’s Track Delivery feature, but on a larger scale. This is a good starting point for admins who are troubleshooting mail delivery on the server.
Mail Queue Manager
This feature allows you to view and manage email messages queued for delivery. The only time I’ve seen the queue get backed up on the server is when there’s been a hacked account sending out a ton of email at once and hitting their ingoing/outgoing quota. So this feature can be helpful to track down what’s happening to a specific account.
ConfigServer Mail Queue
Same deal as above, but provides more options for refining your search. This is where Reclaim Hosting goes to delete queues that are backed up.
Email Processes in FTP
If you log in as Root on the server and navigate to var/log/, you can look through a couple different processes here:
- exim_mainlog– successful activity
- exim_paniclog– server level issues with the exim function
- exim_rejectlog– emails that were blocked due to a variety of things like spam blacklist
- var/log/maillog– shows failed login attempts; find user’s IP address
- messages– the catchall for mostly errors of some kind; firewall blocks
- lfd.log– login failure demon; failed attempts
- secure– SFTP/FTP logins
Searching through Email Processes in Terminal
grep "email@example.com" exim_mainlog
^searches firstname.lastname@example.org in the exim_mainlog
In order for an email to be sent/received, there are many processes that are happening behind the scenes. All of these processes are documented in the exim logs listed above, and are given the same Mail ID so we can track every process for one activity. I’ve highlighted an example of where the Mail IDs are located and what they look like above. If you’ve never looked at an exim file before, they can be a little tricky to read!
grep "mail ID" exim_mainlog
^searches mail ID in the exim_mainlog
In addition to time stamps, mail IDs, and error messages, the logs also document the user’s IP address. So in some cases, we can figure out what the user’s IP address is before he/she is able to get back to us with that information. From there, you can run the following command:
grep "IP Address" -R /var/log
^searches IP address in every file & folder in the /var/log directory