Home > General > Archiving emails… the hard way

Archiving emails… the hard way

Email storage can be a problem. Many email providers limit the storage size for a given user. The wrong way to handle email storage is to limit how long an email can be kept.

Unfortunately, this is something I have to deal with. For whatever reason, the powers that be decided people don’t need emails longer then three months, so emails older then three months will be automatically deleted. I find the whole situation comical, but that’s an entirely different conversation.

Since I use Outlook/Exchange for these emails, a normal person would recommend archiving my emails to a PST file on my local machine. Unfortunately, a group policy was pushed out that added the registry key “PSTDisableGrow” for Outlook. This prevents Outlook from adding emails to PST files, even if it’s stored locally.

So now I’m stuck in a position where I can’t automatically archive my emails without paying for a third party product. I need a way to automatically save all my emails as either MSG or EML files to my hard drive, so at least I have a copy.

There are a couple options that I’m exploring. Be warned that the solutions I’m going to talk about are TERRIBLE. They are very much hacks and something that I would completely avoid if I had a chance. I’m open to any suggestions and/or free products.

The first thing I tried was to create a new rule in Outlook for all incoming emails. This rule would execute a custom VB script that saves a copy of the email to my machine. Unfortunately, I couldn’t get it to work. I fumbled around with it for about an hour before I gave up.

The second option was to utilize Exchange Web Services (EWS). Newer versions of Exchange expose a SOAP web service for anyone to use. Most of the time, the location of the web service can be discovered by going to the address http://webmail.example.com/ews/exchange.asmx, where “example.com” is your domain. Microsoft provides a managed interface called Exchange Web Services Managed API that simplifies access. I was quite surprised at how easy it was to develop a simple solution.

var service = new ExchangeService(ExchangeVersion.Exchange2010_SP1)
{
	Credentials = new WebCredentials("user", "password"),
	Url = new Uri("https://webmail.example.com/ews/exchange.asmx")
};

Folder folder = Folder.Bind(service, WellKnownFolderName.Inbox);
FindItemsResults<Item> emails = folder.FindItems(new ItemView(Int32.MaxValue));

service.LoadPropertiesForItems(emails, new PropertySet(ItemSchema.MimeContent));

string archiveDirectory = Path.Combine(@"D:\EmailArchive", DateTime.Now.ToString("yyyy-MM"));

if (!Directory.Exists(archiveDirectory))
	Directory.CreateDirectory(archiveDirectory);

foreach (Item email in emails)
{
	string path = Path.Combine(archiveDirectory, email.StoreEntryId + ".eml");

	if (!File.Exists(path))
		File.WriteAllBytes(path, email.MimeContent.Content);
}

This code snippet basically downloads my entire inbox and saves it locally. It doesn’t get any easier than that. EWS also supports streaming, push, and pull notifications. This allows me to monitor any incoming/outgoing emails and immediately archive them. I could fall back to iterating over the entire inbox every few days to catch any emails I missed.

As much as I like how simple this solution is, I can’t depend on it. Unfortunately, EWS can be disabled by an Exchange administrator. Knowing how people have reacted before, this feature of Exchange will probably be disabled once they realize someone is using it.

The final option I’m currently exploring is to create an Outlook addin using VSTO. Unfortunately, it utilizes COM objects for nearly everything. I have very little experience with COM, so I ran into several issues.

Folder inbox = (Folder) Application.Session.GetDefaultFolder(OlDefaultFolders.olFolderInbox);
string archiveDirectory = Path.Combine(@"D:\EmailArchive", DateTime.Now.ToString("yyyy-MM"));

if (!Directory.Exists(archiveDirectory))
	Directory.CreateDirectory(archiveDirectory);

foreach (object item in inbox.Items)
{
	var email = item as MailItem;

	if (email == null)
		continue;

	string path = Path.Combine(archiveDirectory, email.EntryID + ".msg");

	if (!File.Exists(path))
		email.SaveAs(path);
}

There are several things wrong with the code above. Since nearly everything is a COM object, I need to release them after I’m done. The first time I ran this, it worked for the first few hundred emails. At the 300 mark, I received the exception “Your server administrator has limited the number of items you can open simultaneously.” In the code above, each iteration of the loop would reference a new COM object. After a couple hundred iterations of the loop, it would fail because I never released any of them.

This article mentions a pretty good guideline.

1 dot good, 2 dots bad

This means I need to pay special attention to property chaining. For example:

Folder inbox = (Folder) Application.Session.GetDefaultFolder(OlDefaultFolders.olFolderInbox);

// Bad
inbox.Items.ItemAdd += OnItemAdd;

// Good
Items inboxItems = inbox.Items;
inboxItems.ItemAdd += OnItemAdd;

Since I need to release the COM objects in the opposite order of creation, I used a stack to keep track of all my references.

Stack<object> comObjects = new Stack<object>();

Folder inbox = (Folder) Application.Session.GetDefaultFolder(OlDefaultFolders.olFolderInbox);
comObjects.Push(inbox);

Items inboxItems = inbox.Items;
comObjects.Push(inboxItems);

Folder sent = (Folder) Application.Session.GetDefaultFolder(OlDefaultFolders.olFolderSentMail);
comObjects.Push(sent);

Items sentItems = sent.Items;
comObjects.Push(sentItems);

// 
// Do something
//

while (comObjects.Count != 0)
{
	object obj = comObjects.Pop();

	if (obj != null)
		Marshal.ReleaseComObject(obj);
}

While iterating through the Items collection using a for loop, I would immediately receive an exception saying that the array was out-of-bounds. Like any C# developer, I start the iterating arrays at index 0. However, the Items collection starts at index 1. MSDN documents it here.

The items collection contains an event that fires for each new email that is received. I can utilize this event for both the inbox and sent folders to immediately archive new emails. There is caveat to using this event that is mentioned in this article. Whenever 16 or more items are added at the same time, this event does not fire. I don’t need to worry about this limitation most of the time, but I would still need to fall back to iterating over the entire inbox every once in a while to make sure all the emails have been saved.

Creating an Outlook addin isn’t as simple as using the EWS Managed API. With the restrictions in place, it seems this is the only option I currently have. Even after I save all the emails to my local machine, I still need to create a separate service to parse and index all the emails.

A few people have started migrating all their old emails into OneNote by highlighting their entire inbox and pressing one button. If things get too complicated, I might have to fall back to this.

This is a lot of work for a simple problem. The wrong way to handle email storage is to limit how long an email can be kept.

  1. Patrick A Latimer Sr
    January 22, 2014 at 8:47 pm

    A very elegant solution. Nice Malvin.

  2. August 21, 2014 at 9:35 am

    Nice blog here! Also your website loads up fast! What host are
    you using? Can I get your affiliate link to your host? I wish
    my web site loaded up as quickly as yours lol

  3. August 22, 2014 at 9:48 pm

    Hi there to every body, it’s my first pay
    a visit of this website; this web site includes remarkable and really good material for readers.

  4. August 22, 2014 at 11:50 pm

    I very rarely comment on these articles, but
    I assumed this on deserved a thumb up

  5. August 25, 2014 at 3:00 pm

    It is strongly recommended that agents provide each client with a package prior to
    the actual listing appointment. You see, local business listings
    are the interactive yellow pages of the 21st century.
    B2b sites are filled with businesses trying to promote themselves and find the ones who
    meet your target b2b customer and get them added to your contacts and network lists and are becoming
    the best and easiest options of doing business for small
    and medium enterprises (SMEs).

  6. August 26, 2014 at 12:09 am

    How you build your mini site can have an effect on your sales.
    The privacy settings also helps the users to hide the
    users in their Circles as well as who has them in their Circle.
    Actually, these local searches will continue for assisting
    the people with getting business under that area.

  7. August 26, 2014 at 3:00 am

    Try to make it sound natural and less like a sales letter.
    You might produce your own citations at directories a lot like the Yellow Pages, Super Pages, City
    – Search, Insider Pages, Yahoo Nearby, Localize and a number of others.
    But at the same time, you have to differentiate yourself from all those other
    ads.

  8. August 26, 2014 at 8:31 pm

    The best links to use are contextual links, high PR links, indexable links and natural links.
    Therefore, facebook offers different options for creating a
    page, each one with its own set of advantages. (while you’re there, feel free
    to add us to your circles.

  9. August 27, 2014 at 5:39 am

    You can either have a website built by a professional or you can build one yourself if you’re on a tight budget.
    Google claims that about two-thirds of their visitors will click on one of the top three spots.

    People talk and that’s what you want especially on a social media site
    some of the strangest videos on You – Tube have gone viral your hope is that your Facebook fan page will do the same.

  10. September 3, 2014 at 1:34 am

    I have to thank you for the efforts you’ve put in penning this blog.
    I am hoping to view the same high-grade content by you later on as well.
    In truth, your creative writing abilities has encouraged
    me to get my own blog now 😉

  11. September 4, 2014 at 6:58 pm

    Heya! I know this is kind of off-topic however I needed to ask.
    Does building a well-established blog such as yours
    require a lot of work? I am completely new to writing a blog but I do write in my diary every day.
    I’d like to start a blog soo I will be able to share my experience and views
    online. Please let me know if you have any kiind of
    recommendations or tips foor new aspiring bloggers.
    Thankyou!

  12. September 5, 2014 at 2:11 pm

    If the customer is staying far away from the location of the company then company can e-mail the
    photograph of send it by courier (choice by the customer).
    To accomplish the particular qualifications assessment without any
    difficulty your prospect ought to undertake the practice test.
    Here are the three reasons that you should not
    sell on Amazon:.

  13. September 9, 2014 at 1:07 am

    After checking out a few of the articles on your site,
    I really appreciate your way of blogging. I added it to my bookmark website list and will be checking back in the near future.
    Please visit my website as well and tell me your opinion.

  14. September 12, 2014 at 3:51 pm

    Have you ever considered about adding a little bit more than just your articles?
    I mean, what you say is important and all. Nevertheless think
    of if you added some great images or videos to give your posts more, “pop”!
    Your content is excellent but with pics and video clips, this website could
    definitely be one of the greatest in its niche. Great blog!

  15. September 19, 2014 at 12:35 pm

    I think this is one of the most important information for me.
    And i am glad reading your article. But wanna remark on some general things, The
    site style is wonderful, the articles is really great : D.
    Good job, cheers

  16. September 19, 2014 at 10:07 pm

    Today, I went to the beach front with my kids.
    I found a sea shell and gave it to my 4 year old daughter and said “You can hear the ocean if you put this to your ear.” She placed
    the shell to her ear and screamed. There was a hermit crab inside and it pinched her ear.

    She never wants to go back! LoL I know this is totally off topic but I had to tell someone!

  17. September 21, 2014 at 7:59 am

    hey there and thank you for your information – I’ve certainly picked up anything new
    from right here. I did however expertise some technical issues using this web site, as
    I experienced to reload the web site many times previous to I
    could get it to load properly. I had been wondering if your
    web host is OK? Not that I am complaining, but sluggish loading instances times will often affect your placement
    in google and could damage your high quality score if ads and marketing with Adwords.
    Well I’m adding this RSS to my e-mail and
    can look out for a lot more of your respective
    intriguing content. Ensure that you update this again very soon.

  18. September 23, 2014 at 3:43 pm

    Good respond in return of this difficulty with genuine arguments and telling everything about that.

  19. September 23, 2014 at 11:51 pm

    Aw, this was a very nice post. Finding the time and actual effort to create a great article… but what can I
    say… I procrastinate a whole lot and don’t seem to get nearly
    anything done.

  20. knights and dragons hack
    September 24, 2014 at 7:57 pm

    I really like your blog.. very nice colors & theme. Did you make this website yourself or did
    you hire someone to do it for you? Plz reply as I’m looking to create my own blog and would like to know
    where u got this from. thanks a lot

  21. pou hack
    September 24, 2014 at 11:46 pm

    I’m gone to tell my little brother, that he should also go to see
    this web site on regular basis to obtain updated from hottest information.

  22. farm heroes saga cheats
    September 25, 2014 at 1:35 am

    I know this web site gives quality based content and extra
    data, is there any other website which gives these kinds of
    information in quality?

  23. September 25, 2014 at 4:34 am

    Hello! Would you mind if I share your blog with my zynga group?
    There’s a lot of folks that I think would really enjoy
    your content. Please let me know. Thank you

  24. October 3, 2014 at 4:14 pm

    When someone writes an post he/she maintains the plan of a user in his/her
    mind that how a user can be aware of it. Thus that’s why this article is outstdanding.
    Thanks!

  25. http://jungleheathackxz.wordpress.com/
    October 3, 2014 at 10:32 pm

    Thanks for ones marvelous posting! I actually enjoyed reading it, you may be a great author.I will
    make certain to bookmark your blog and will eventually come back in the foreseeable future.
    I want to encourage one to continue your great writing, have a nice holiday weekend!

  26. October 6, 2014 at 1:19 pm

    excellent issues altogether, you just won a brand new reader.
    What would you suggest in regards to your publish that you made a few
    days in the past? Any sure?

  27. October 7, 2014 at 5:02 am

    Yes! Finally something about japan cheap used cars for sale.

  28. October 8, 2014 at 7:56 pm

    Hi Dear, are you truly visiting this web page on a regular basis,
    if so then you will definitely get nice knowledge.

  29. October 16, 2014 at 8:48 am

    Spot on with this write-up, I seriously believe this amazing site needs far more attention. I’ll probably be
    back again to read through more, thanks for the info!

  1. January 23, 2014 at 12:30 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: