Thursday 13 June 2013

Outlook 2013 Bug when using IMAP - Beware the Bandwidth Blowout!

We recently had issues with our work internet connection that meant sizeable amounts of data were being chewed up without our users doing anything in particular or untoward.

Using the bandwidth monitor utility Networx (http://www.softperfect.com/products/networx/) (which breaks accumulated traffic down by process), we specifically found that Outlook 2013 was downloading large amounts of data (40-50MB) - many (but not all) times that a "Send and Receive" synch was performed. This was using IMAP running against the open source DoveCot (http://www.dovecot.org/) as a mail server.

Now by default, this is set to every 30 minutes. If a machine is on 24 hours day, this can add up to 2.4GB per user - which clogs up your network and really adds up on anything but an unlimited plan!

Cracking open Wireshark (http://www.wireshark.org/), I turned off SSL to see what was going on in terms of the IMAP commands going down the wire and the full extent of the download. Nothing stuck out apart from the fact that setting the "Mail to keep offline setting" to 24 months as below performs an IMAP "UID SEARCH SINCE 13-Jun-2011" to filter results. This is as expected.


To my surprise, with the default "All" Setting for "Mail to keep offline", you can also see that on regular but apparently random occasions every single email (including ALL content and ALL binaries) was being downloaded every time from the mail server - regardless of whether Outlook already had a locally cached version or not. Sometimes, the download will only be 5MB per click of "Send and Receive All". I tried to reproduce it and it tends to happen when you save Drafts or send/receive large items in your inbox - but I wasn't able to isolate the issue through a process of elimination.

This may well be part of the OST corruption issues mentioned here - the IMAP issues in Outlook 2013 are apparently high on the MS internal support agenda:
http://productforums.google.com/forum/#!topic/gmail/9alMlaOkMC4

In terms of the mail server, the installed version of Dovecot also seems to support LIST-EXTENDED for special folder support introduced in Outlook 2013 - so that doesn't appear to be the problem (unless there is a variation on the standard between Outlook/the IMAP server - which is not impossible).

So no clear conclusions I'm afraid. The one thing that is clear is that there are problems with Outlook 2013 and IMAP - based on the large number of problems I've seen floating around the web. The above problem is just one example of these. I've also seen that Outlook 2013 sometimes doesn't download new emails at all (even though other clients such as Android download from the same mail server correctly.). In addition, when working with drafts and saving the draft email repeatedly, Outlook will not be able to save it and say that it must be saved as a new copy within your Drafts folder.

Apparently, a fix to some of these problems should be released around August 2013 (as per http://productforums.google.com/forum/#!topic/gmail/9alMlaOkMC4)

In the meantime, to fix the heavy bandwidth usage issue, I can only suggest:
  1. Roll back to Outlook 2010 if you're using IMAP . This seems to be a popular option - although you cannot have Outlook 2013 and 2010 running simultaneously (unlike other MS Office Products).  [Recommended - this worked in my situation and reduced synch bandwidth usage from 50MB to 850KB]
  2. Perform a full initial synchronisation with the default "All" setting. Then reduce your "Mail to keep offline" time window to a smaller figure after this initial synchronization (e.g. 1 month) 
  3. Delete your local OST file and set up email again. This may give you temporary relief from the problem - but it wasn't a permanent fix for me.
  4. Use yet another client like Thunderbird - although this apparently also has issues with non-appearing folders with some IMAP providers as well.
  5. Reduce your synchronisation frequency (to something less than 30 minute full synchs). 
  6. Update your mail server if possible to ensure full compliance with standards. This isn't an option for me. You should also confirm this with a pilot/Proof of Concept to confirm that it fixes your specific issue.
  7. Clean out/auto-archive your emails!

DDK

88 comments:

  1. Thank you for your suggestions, I'm fighting against this bug since January when my 10 GB/month mobile internet plan was completely swallowed by Outlook 2013.

    ReplyDelete
  2. A few clients of ours are struggling with this exact problem, but don't have the option of rolling back to 2010.

    Have you discovered a permanent fix since writing this?

    ReplyDelete
  3. Thanks for posting. It has helped my sanity.

    I finally looked into my bandwidth issues because I was having to move from unlimited to limited bandwidth and identified Outlook 2013 as the culprit using free or trial versions of Networx and NetBalancer.

    I’m an Outlook 2013 home premium subscriber and I have 3 IMAP accounts set up whose data files total about 1.8 GB and a Gmail IMAP (700 MB) and Hotmail ActiveSync (120 MB) accounts. Nothing too unusual I would have though and I didn’t have any bandwidth issues on any of the previous 3 version of Outlook.

    I had Outlook set up to send and receive every 2 minutes and it managed to consume 210 to 240 MB per HOUR! Even when there were no emails being received.

    I also have no way of rolling back to a previous version so I called Microsoft who said that there is no issue and that I must contact my mail service provider or an issue with another device which checks my email. The guy said that this is the first time that he has ever heard of an issue like.

    It’s all very frustrating, particularly as there doesn't appear to be any end in sight.

    ReplyDelete
  4. This!

    This is EXACTLY the issue we're currently experiencing. We've been moving clients from pop to imap based accounts, and our bandwidth usage has skyrocketed. 1 client has about 2gb of mail in their account, about 60mb in their inbox. Watching the mail logs, they were doing 34mb of downloads every 5 minutes.

    Just want to clarify, did reducing the offline setting to something less than ALL cause outlook to do a proper sync and only download new messages, or does it just mean it now downloads a subset of messages based on the date, but still downloads every single message regardless of status?

    ReplyDelete
  5. This comment has been removed by the author.

    ReplyDelete
  6. Oops previous typo.....

    Me too.

    I'm chewing through about 5Gb a day. My 25Gb/month Telstra plan isn't holding up well.

    Is there a solution yet?

    JC

    ReplyDelete
  7. Since you are running Dovecot, I think suggestion #6 is a little silly. Dovecot is just about as standard as you can get when it comes to IMAP.

    ReplyDelete
  8. would be great if someone had an idea how to fix this?

    ReplyDelete
  9. Hah, IT guy here and me and my company have been scratching our heads how one of our clients went from - to over 50gb of data within a few days.

    This, we have determined, is the culprit. Now that I've seen these posts i'm sure of it.

    Sigh

    ReplyDelete
  10. Same issue here - I went back to POP3 a week ago, bandwith issues solved (I had peaks up 15Gb / day on IMAP !!!)

    ReplyDelete
  11. We operate mailservers and noticed that IMAP usage on one client account went from 20Mb or so per day to 6.5Gb and now 8Gb per day. We run Dovecot and client has Outlook 2013 so this looks like the issue. Not sure what changed on the day bandwidth shot up. Client was operating Outlook 2013 before and we have not updated Dovecot.

    ReplyDelete
  12. it looks like this issue has been fixed in the latest update - 15.0.4551.1005

    ReplyDelete
  13. Hi quartermileburner

    Sounds you got some good news for us there. Do you have a reference or Change History link for the latest release 15.0.4551.1005 so we can see the issue has been addressed? I can't find any info on the release.

    Thank you.

    ReplyDelete
  14. You wrote tha the preblem may be fixed with Outlook version 15.0.4551.1005. Is that really sure. Have you alaysed it?
    The huge data amound pisses me off. I would have no problem to change the client if there would be another Exchange active Sync Client for Windows 7.

    ReplyDelete
  15. Fought this problem for 3 days, I went in and increased the server timout period to 4min 20 secs in the Account Settings->More Settings->Advanced slider (nothing magic about that number I just grabbed the slider and moved it way right) and since then Outlook 2013 / IMAP on an Inbox with 7000 mails is flying - no repeat syncing - no huge bandwidth syncs.

    ReplyDelete
  16. Not sure if this will help, but I just went into Preferences/accounts/advanced/ and ticked 'download headers only' and 'except in inbox'.

    Seems to have done the trick. Went from 500+kb/sec to 3kb/sec/

    Hope it works for you too :)

    ReplyDelete
  17. I fixed my problem with Gmail/Outlook 2013 sync by going to Google and managing my account.

    There is a spot to turn on "Access for less secure apps." They don't recommend that you do it, but I'm now able to receive my Gmail after setting up a new account in Outlook 2013.

    I hope this helps.

    Sandman3

    ReplyDelete
  18. Reducing your "Mail to keep offline" time window to 1 month seems to work

    ReplyDelete
  19. I love this blog because it is really helpful and informative blog for us and I’m thankful to you for sharing this blog. For any kind of technical support your visit the website Outlook Phone Number UK or call at 0800-090-3220.

    ReplyDelete
  20. Its really helpful for the users of this site. I am also searching about these type of sites now a days. So your site really helps me for searching the new and great stuff.vmware training in bangalore

    ReplyDelete
  21. Very useful and information content has been shared out here, Thanks for sharing it.aws training in bangalore

    ReplyDelete
  22. These provided information was really so nice,thanks for giving that post and the more skills to develop after refer that post.salesforce developer training in bangalore

    ReplyDelete
  23. Your articles really impressed for me,because of all information so nice.salesforce admin training in bangalore

    ReplyDelete
  24. Linking is very useful thing.you have really helped lots of people who visit blog and provide them use full information.devops training in bangalore

    ReplyDelete
  25. Being new to the blogging world I feel like there is still so much to learn. Your tips helped to clarify a few things for me as well as giving.servicenow training in bangalore

    ReplyDelete
  26. Really it was an awesome article,very interesting to read.You have provided an nice article,Thanks for sharing.informatica training in bangalore

    ReplyDelete
  27. I know that it takes a lot of effort and hard work to write such an informative content like this.cloud computing training in bangalore

    ReplyDelete
  28. Your articles really impressed for me,because of all information so nice.sap tm training in bangalore

    ReplyDelete
  29. I am really happy to say it’s an interesting post to read. I learn new information from your article, you are doing a great job. Keep it up…

    Upgrade your career Learn AWS Training from industry experts get Complete hands-on Training, Interview preparation, and Job Assistance at Bangalore Training Academy Located in BTM Layout.

    ReplyDelete
  30. Very interesting, good job and thanks for sharing such a good blog.

    Upgrade your career Learn SharePoint Developer Training in Bangalore from industry experts get Complete hands-on Training, Interview preparation, and Job Assistance at Softgen Infotech.

    ReplyDelete
  31. I couldn’t resist commenting. Exceptionally well written! onsite mobile repair bangalore Saved as a favorite, I like your blog! asus display repair bangalore Great post. I am experiencing many of these issues as well.. huawei display repair bangalore

    ReplyDelete
  32. Hey there! I simply would like to give you a big thumbs up for your great info you've got right here on this post. I'll be coming back to your web site for more soon. online laptop repair center bangalore When I originally left a comment I appear to have clicked on the -Notify me when new comments are added- checkbox and from now on whenever a comment is added I receive four emails with the exact same comment. There has to be a way you can remove me from that service? Appreciate it! dell repair center bangalore

    ReplyDelete
  33. The next time I read a blog, Hopefully it won't fail me as much as this one. I mean, Yes, it was my choice to read, but I genuinely thought you would probably have something helpful to say. All I hear is a bunch of whining about something you could fix if you weren't too busy seeking attention. macbook repair center bangalore Spot on with this write-up, I actually believe this website needs far more attention. I’ll probably be returning to see more, thanks for the information! acer repair center bangalore


    ReplyDelete
  34. Really i appreciate the effort you made to share the knowledge. The topic here i found was really effective...

    Softgen Infotech have the best Python Training in Bangalore . Any professional who is looking out to switch their career can enroll with us.

    ReplyDelete
  35. Really i appreciate the effort you made to share the knowledge. The topic here i found was really effective...

    Looking for SAP FICO Training in Bangalore , learn from eTechno Soft Solutions SAP HR HCM Training on online training and classroom training. Join today!

    ReplyDelete
  36. This is too informative and interesting. Thanks for sharing an informative blog with us. keep doing.
    Regards,
    Digital Marketing Company in Chennai
    eCommerce Development Company
    Best Online Payment Gateway
    ERP Services

    ReplyDelete
  37. SAP ABAP Training in Bangalore with 100% placement. We are the Best SAP ABAP Training Institute in Bangalore. Our SAP ABAP courses are taught by working professionals who are experts in.

    sap abap training in bangalore

    sap abap courses in bangalore

    sap abap classes in bangalore

    sap abap training institute in bangalore

    sap abap course syllabus

    best sap abap training

    sap abap training centers

    ReplyDelete

  38. Well thats a nice article.The information You providied is good . Here is i want to share about dell boomi training and Mulesoft Training videos . Expecting more articles from you .

    ReplyDelete

  39. This is most informative and also this post most user friendly and super navigation to all posts. Thank you so much for giving this information to me.SAP ABAP training in Chennai.
    Java training in chennai | Java training in annanagar | Java training in omr | Java training in porur | Java training in tambaram | Java training in velachery

    ReplyDelete
  40. I have express a few of the articles on your website now, and I really like your style of blogging. I added it to my favorite’s blog site list and will be checking back soon…
    Business Analytics Training
    Business Analytics Course In Hyderabad

    ReplyDelete
  41. Wow it is really wonderful and awesome thus it is very much useful for me to understand many concepts and helped me a lot.

    SAP HCM Online Training

    SAP HCM Classes Online

    SAP HCM Training Online

    Online SAP HCM Course

    SAP HCM Course Online

    ReplyDelete
  42. I have recently visited your blog profile. I am totally impressed by your blogging skills and knowledge.

    Dell Boomi Training in Bangalore

    Best Dell Boomi Training Institutes in Bangalore

    ReplyDelete
  43. Awesome blog. I enjoyed reading your articles. This is truly a great read for me. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work!

    AWS training in Chennai

    AWS Online Training in Chennai

    AWS training in Bangalore

    AWS training in Hyderabad

    AWS training in Coimbatore

    AWS training


    ReplyDelete
  44. Great writing! You have a flair for informational writing. Your content has impressed me beyond words. I have a lot of admiration for your writing. Thank you for all your valuable input on this topic.
    Data Science training in Mumbai
    Data Science course in Mumbai
    SAP training in Mumbai

    ReplyDelete
  45. Mindblowing blog appreciating your endless efforts in developing a truly transparent content. Which probably the best one to come across disclosing the content which people might not aware of it. Thanks for bringing out the amazing content and keep sharing more further.

    360DigiTMG PMP Certification Course

    ReplyDelete
  46. Amazing article with unique content and top quality information thank you for sharing.

    typeerror nonetype object is not subscriptable

    ReplyDelete
  47. Awesome blog with interesting topic and information provided was very valuable waiting for next blog update thank you .
    Ethical Hacking Course in Bangalore

    ReplyDelete
  48. We have seen many blog but this the great one, Thanks for provide great informatic and looking beautiful blog, really nice required information & the things i never imagined and i would like to request, wright more blog and blog post like that for us. Thanks you once agian

    birth certificate in delhi
    birth certificate in noida
    birth certificate in ghaziabad
    birth certificate in gurgaon
    correction in birth certificate
    marriage registration in delhi
    marriage certificate delhi
    how to change name in 10th marksheet
    marriage registration in ghaziabad
    marriage registration in gurgaon

    ReplyDelete
  49. Oracle Corporation is an American multinational computer technology corporation headquartered in Redwood Shores, California.
    tally training in chennai

    hadoop training in chennai

    sap training in chennai

    oracle training in chennai

    angular js training in chennai

    ReplyDelete

  50. Thanks for the Valuable information.Really useful information. Thank you so much for sharing. It will help everyone.

    SASVBA is recognized as the best machine learning training in Delhi. Whether you are a project manager, college student, or IT student, Professionals are the best machine learning institute in Delhi, providing the best learning environment, experienced machine learning instructors, and flexible training programs for the entire module.
    FOR MORE INFO:

    ReplyDelete
  51. This is a really very nice post you shared, I like the post, thanks for sharing...

    business analytics course

    ReplyDelete
  52. I seriously love your site.. Very nice colors & theme. Did you create this site yourself? Please reply back as I’m trying to create my very own site and would like to learn where you got this from or exactly what the theme is named. Many thanks...

    DevOps Training in Hyderabad

    ReplyDelete
  53. You have completed certain reliable points there. I did some research on the subject and found that almost everyone will agree with your blog.
    Data Analytics Course in Bangalore

    ReplyDelete
  54. This is a great post. I like this topic.This site has lots of advantage.I found many interesting things from this site. It helps me in many ways.Thanks for posting this again. data science training in surat

    ReplyDelete
  55. I read your post and I found it amazing! thank! data science course in kanpur

    ReplyDelete
  56. Going to graduate school was a positive decision for me. I enjoyed the coursework, the presentations, the fellow students, and the professors. And since my company reimbursed 100% of the tuition, the only cost that I had to pay on my own was for books and supplies. Otherwise, I received a free master’s degree. All that I had to invest was my time. business analytics course in surat

    ReplyDelete
  57. A good blog always comes-up with new and exciting information and while reading I have feel that this blog is really have all those quality that qualify a blog to be a one. data science course in kanpur

    ReplyDelete
  58. 360DigiTMG, the top-rated organisation among the most prestigious industries around the world, is an educational destination for those looking to pursue their dreams around the globe. The company is changing careers of many people through constant improvement, 360DigiTMG provides an outstanding learning experience and distinguishes itself from the pack. 360DigiTMG is a prominent global presence by offering world-class training. Its main office is in India and subsidiaries across Malaysia, USA, East Asia, Australia, Uk, Netherlands, and the Middle East.

    ReplyDelete
  59. A good blog always comes-up with new and exciting information and while reading I have feel that this blog is really have all those quality that qualify a blog to be a one. data science course in mysore

    ReplyDelete
  60. The next time I read a blog, I hope that it doesnt disappoint me as much as this one. I mean, I know it was my choice to read, but I actually thought you have something interesting to say. All I hear is a bunch of whining about something that you could fix if you werent too busy looking for attention. data science course in mysore

    ReplyDelete
  61. SAS is a command-driven statistical software package widely used for statistical data analysis and visualisation. Statistical Analysis Software is referred to as SAS. The pronunciation of SAS is "SaaS." To know more about SAS join SAS Training in Chennai at FITA Academy.

    SAS Training in Chennai

    ReplyDelete
  62. Nice thanks for sharing an informative post like this keep posting.

    Java course in Hyderabad

    ReplyDelete
  63. Nice Article. It was Really Helpful to me Thanks for Sharing
    Best CA Colleges in Hyderabad

    ReplyDelete
  64. Your blog is fantastic! Thank you for sharing such wonderful content that is very useful to us. I have learned a lot of new information, and the way you explained it clearly is amazing. Please continue to post more useful information.

    B.Com Colleges in Hyderabad

    ReplyDelete

  65. I am grateful for your willingness to share your thoughts. Your efforts are truly appreciated and I look forward to reading more from you. Thank you again.

    Best CA Coaching in Hyderabad

    ReplyDelete
  66. Your blog is excellent and captivating. Please continue to share more informative posts like this. It was very helpful. Thank you for sharing. Check out my website.

    Best Junior Colleges in Hyderabad for CEC

    ReplyDelete
  67. Thanks for sharing the blog Keep posting
    Also read our latest blog:- CCNA Cloud Certification Training in India

    ReplyDelete