As Office 365\Exchange Online (EXO) admins, it’s important that we review messages that are submitted by our users so that we can use that information to make the solution more effective. Do you have a simple and effective method for your users to submit samples to you of false positives, false negatives, and phishing? The key word here is “simple”. I have found that most companies tend to set up a shared mailbox or a distribution list such as “email@example.com” and direct users to send the message in question there. While this can be sufficient for some, it’s not an ideal solution. For one, users have to know it exists. Secondly, even if they do, how do you know why the users submitted the message since they usually won’t provide a detailed explanation? Lastly, the users usually forward the message as-is rather than attach it so you can’t do any useful forensic analysis on the headers of the original message – it’s a pain in the rear for the user to attach the original message and for the admin to chase after the user for the headers.
Microsoft’s “Report Message” Add-In
My recommendation to all of my Exchange Online customers is to use the amazingly simple and versatile Report Message add-in. Not only is it super simple for your users to use, but it’s mostly platform agnostic. That means it will work for your Mac users just as well as it works for your Windows or Outlook on the web users (and, soon, Outlook mobile!). You can also deploy it from the Office 365 admin portal with ease to a group of users, to a single users, or to everyone in your organization.
Using this tool has a few advantages:
- It reports the message to Microsoft so that the analysts behind the Exchange Online Protection (EOP) magic can make the solution more effective for everyone.
- It automatically classifies a message as Junk (false negative), Not Junk (false positive), and Phish. As an added bonus, messages classified as Junk\Phish get moved to the Junk Mail folder automatically – cool!
- Since this tool sends a full copy of the email as part of the report, there is no need to chase after the users to get the headers to perform a forensic analysis.
Now, some of you are probably asking, “Wait, you said it sends the message to Microsoft, not to me – how is this useful to me as an EXO admin?”. Well, out of the box, it’s not. This is where the magic of a mail flow rule (and some inbox rules) comes in! You can also use Explorer in Security & Compliance Center, but more on that later.
First, Shared Mailbox
The first step in this solution is to create a Shared Mailbox. Now, if you are syncing your on-prem Active Directory via Azure AD Connect, I recommend creating the Shared Mailbox on-prem and then migrating it over (preferably using MRS via your hybrid relationship), or by creating a new RemoteMailbox using the “-Shared” parameter (as long as you are using Exchange 2013 CU21+, Exchange 2016 CU10+, or Exchange 2019). Whichever way you use to end up doing this, you should end up with a Shared Mailbox that exists in EXO that is dedicated to user submissions via the Report Message add-in.
Once you get this mailbox going and have tested mail flow to and from it, you should give permissions to users that will be accessing it. This could be your messaging security team, messaging admins, or whomever might benefit from being able to analyze these submissions.
One other thing I recommend doing in this mailbox is setting up Categories. This allows you to easily differentiate between what users have submitted as well as separate them out. You can create these rules via Outlook on the web. Once you log into a mailbox that was given permissions to access the new Shared Mailbox, you can create rules, like the following:
You should assign them useful colors as well:
So then it ends up looking like this:
Then you can click on a category to filter out the messages you aren’t interested in:
OK, so now that you’ve got your Shared Mailbox ready to go, let’s create the Mail Flow rule!
Mail Flow Rule Time
Head over to the EXO control panel at https://outlook.office.com/ecp and click on “mail flow”, then click “rules”. I’ve created the rule already, so it’s listed here, but we’ll drill in after:
Let’s break this down inside the rule itself. Now, there are three places this message could be sent to depending on how your user classified the message – firstname.lastname@example.org, email@example.com, and firstname.lastname@example.org. As such, that will the predicate\conditions you’ll use to get a rule match. Be aware that you might want to place this rule at a higher priority than a rule that may contain a “stop processing” action just in case the message gets caught there, otherwise, you won’t see the submission in your Shared Mailbox because it won’t hit our submission rule here.
Your actions will be two-fold. The first will be setting up a Bcc to your new Shared Mailbox. The second, which is optional, is to add an X-Header just to know that your rule took effect here. The second rule is just something I always do when I can as an Exchange Admin. I also recommend leaving the Audit selection in place so you can see how many times this rule has fired for your own edification.
Here is the rule itself:
Once the rule is created, give it about a half hour to replicate around your tenant mailbox servers, to be safe. Then you try testing it by using the tool to submit something. If you see the message in your Shared Mailbox with a category, you’re good to go!
Deploy the Add-In and Spread the Word!
Now that you’ve got all of the back end stuff taken care of and tested, you can deploy the add-in using the directions in the link at the beginning of this article to deploy the add-in to your users. I recommend starting with a large pilot group of people that are generally receptive to this sort of thing. They can then be your champions to help spread the word. Then, I recommend deploying it to the entire organization. At that point, you can start to go on a communication campaign to let your users know that this great tool exists that allows you, as a company, to analyze these messages and make the system better.
Having these message samples will be ESPECIALLY valuable for doing a forensic analysis on a message that was submitted as a phishing attempt. You can start to pick out users that seem to be getting targeted for phishing more than others and use that information to perhaps target them for more security training, or some deeper protections like adding them to your Anti-Impersonation policy (assuming you have Advanced Threat Protection (ATP) licenses – I hope you do!!).
I mentioned earlier that there is another way to get this information – by using Explorer (also known as Threat Explorer) in the Security & Compliance Center portal. I mention this option as an aside because it requires you to have a higher level of ATP license (P2) in order to use it. Plus, even if you have this license, I still recommend creating this Mail Flow rule since it’s difficult to retrieve the actual message for header analysis. Using Explorer for this is a good way to get overall reporting status (e.g. 5 users submitted false positives last week). It looks like this:
There are a few cool things you can do from here such as start an investigation and take actions on these messages, if you so choose. You can read up on these actions and more here.
It’s important to have a handle on how effective your message filtering solution is. Messaging security is of paramount importance to your overall cyber-security strategy as it is the most popular entry point for things like Advanced Persistent Threats (APTs). Empowering your users to be part of the solution rather than part of the problem is a great way to improve the experience you have all around, as well as improving your security posture!
<div class='sharedaddy sd-block sd-like jetpack-likes-widget-wrapper jetpack-likes-widget-unloaded' id='like-post-wrapper-127276047-109-604182aadfc6d' data-src='https://widgets.wp.com/likes/#blog_id=127276047&post_id=109&origin=pragmaticadmin.com&obj_id=127276047-109-604182aadfc6d' data-name='like-post-frame-127276047-109-604182aadfc6d'><h3 class="sd-title">Like this:</h3><div class='likes-widget-placeholder post-likes-widget-placeholder' style='height: 55px;'><span class='button'><span>Like</span></span> <span class="loading">Loading...</span></div><span class='sd-text-color'></span><a class='sd-link-color'></a></div>