It is related to Outlook, where it is not duplicating in the Webmail access and Microsoft has had this issue in the past with other Outlook versions (ie. Express experienced this long ago). Easy fix, remove your email profile from Outlook, readd it and it'll download the messages as usual and only one copy will be there.
I don't have an answer if it will reoccur though, but this is how I've been resolving it so far.
I believe it is a corrupt local file of the user profile (I think .pst?) and when removing the account and readding, it creates a new one and the messages are not duplicated then.