Zammad: Enhance/rework message header of forwarded mails

@mantas @MrGeneration - IIRC we agreed on sending the name instead of the email address to avoid leaking internal email addresses, right? What exactly was the use case/user story we were trying to cover? I got it mixed up.

@thorsteneckel Yes, agent privacy was the main concern.

User story for forwarding header or for omitting email address?

How an agents address (or other internal mail) would end up in a forwarded email.

Taken from system user profile

To make my user story clearer with pictures and a real use case. So we sometimes get email in our self-hosted zammad e.g. from authorities or people who need support, but they simply wrote to the wrong email address (for ex. info@ instead of support@) and we usually just forward this email/ticket to our hosted zammad instance where our customer support works on. So simply click forward, enter the support@ address and click Update. We then expect that the original email sender email to be attached, so our customer support can directly answer them without going through our self-hosted zammad again. I really don't see an email leakage problem here, thats why I insist that this feature should be there/configurable.

For some pictured explanation -> before & after (mind the added email in the forwarding header):

While I agree that it's a potential privacy issue if it goes wrong, I also stated that it would be a good use case that you could enable manually:

Technically Zammad should use the FROM of the article you're forwarding.
Maybe this should be an optional setting that defaults to don't show mail address ?

I also pointed out that Zammad always has to ensure it's not sharing agent mail addresses:

While I agree on the e-mail thing, this is a potential issue.
You don't want, at any time, to tell the recipient what the agents mail address is.

If you tell someone the agents mail address, the agent might receive service requests directly
which technically is against the sense of Zammad as helpdesk software.

I do agree in the customer part, but we'll need to ensure it's just the customer address (without
ticket.agent permissions).

So in short: I personally do agree with Martin and also can remember being asked about this by users. (e.g. during demos) I believe this would at least affect 60% of our user base that would want to do so.

I'm 💯 on @martinseener use case as well

What about a following option for forwarding:

• include all email addresses
• include only customer addresses
• no email address included

I suppose the last one would be default

Another related issue is email including when replying. It could follow this setting as well to avoid further confusion.

I think it only should affect forwarding, as replying via UI does keep the recipients intact on replying to the article. I can also choose to remove a recipient if needed which then would also have to happen within the text field which might be troublesome and confusing for the user. I personally wouldn't like that.

For forwarding, "include all email addresses" should only affect the article you're forwarding.
Potentially you'll have a ton more people communicating within the same ticket which would cause Zammad to provide more mail addresses than it should in my opinion.

This will also ensure that Zammad orientates on the way mail clients work.
If Zammad behaves similar to a mail client, the user will feel like he's home.

But that's my opinion

I think it only should affect forwarding, as replying via UI does keep the recipients intact on replying to the article

It also keeps name and time. Yet some people do want full header included. If we're already adding a setting, I don't see why not extend it to this bit. It adds next-to-none hassle for us, no additional UX complexity, yet it may help someone.

For forwarding, "include all email addresses" should only affect the article you're forwarding.

Absolutely. It only affects the forwarding metadata line. No content of the article itself is touched.

So from my PoV, if we could have the following, you would cover all possible use cases:

1. no forwarding of emails or full headers (current default)
2. forwarding of original email address from sender (see my example above)
3. forwarding of original email sender + other recipients (mostly what thunderbird does. it includes original sender + all ppl in CC with name+email
4. forward full header (what thunderbird does by default)

no 4 would be the "thunderbird" style of forwarding, though this is maybe too much for a ticket-systems default but i would at least default to 1. which would be okayish for most people including my use case but you can configure 1-4 - maybe per queue/email integration. then you can tune your workflow as much as possible. (or leave it as a global config which would also be fine i guess).
Attached is Number 4. Thunderbird forwarding default.

Maybe as an example for No 3, i could imagine something like this.

So 1-3 would still "look" like zammad and No 4 would be in thunderbird style - maybe also quoted like shown in example above but with full header. That would be really nice to have and maybe not too hard to write (hopefully)

Good point on CCd emails.

Full header would be slightly more complicated. Right now we don't store raw email header in database. I how many people would actually use it vs being a nice option to have in settigns dropdown.

@MrGeneration what's your opinion on "only customers" option to prevent agent emails from being shared?

I think Option 1-3 would be sufficient then. Full header is rather "additional nice to have" but I also don't see a real benefit there.

I agree, I'd only see options 1-3 as usefull options.

No matter of the details of the option, Zammad should always not provide agents mail addresses.
What we however can do is to provide Zammads mail addresses. So if they appear within a TO or CC, it wouldn't hurt if we still provide those during forwarding if applicable.

@martinseener 1-3 from your list or my list? You listed an additional option with CC while my list has customer-only option :)

Personally I'd rather just include CC'ed emails whenever any email is included.

@MrGeneration do I read it correctly that we shouldn't have an option to forward agent emails at all? I could see that being an issue with agent-as-a-customer workflow.

Yeah basically your list @mantas . So no agents addresses but To/CC addresses from customers or other people.

@MrGeneration do I read it correctly that we shouldn't have an option to forward agent emails at
all? I could see that being an issue with agent-as-a-customer workflow.

No, what I meant:
You can forward every communication type article (just like already possible).
However, Zammad will filter out any agent email address that appears in the address list for the forward to ensure not to provide agents mail addresses to keep them private and safe.

That's what I tried to say - that article could be forwarded, but agent email wouldn't be forward and there would be no option to allow that :)

How would it work in agent-as-a-customer situation? Would it treat that agent as customer-by-sender-type (thus including email) or agent-by-role (thus overriding email)?

My bad. In my opinion Zammad wouldn't handle the agent being customer as customer.
You may want to check back to Thorsten or Martin though, because that's just my opinion.

I'm not sure if it's already merged, but I remember seeing something like that in our private work-in-progress branches recently... IIRC the use case was that in big companies one department may be a customer of another department.

Zammad by default knows all email addresses with the permission ticket.agent - I don't think you'd need more at this moment. Beside of extending tests later on if needed.

@MrGeneration #2815 is a predecessor/duplicate of this, right?

@thorsteneckel looks like yes

Can I please get a list of scenarios/use cases for each of the possible options and an estimation on how many percentage of our user base would use it?

So I've been talking with Mantas internally and this is basically what came out of that.
It consist of two parts: Part 1 describes the differences between replies and forwards to have the same point of view and Part 2 delivers possible use cases and an percantage that I think should be fitting.

I have no proove of those numbers below whatsoever - if we'd need real numbers, we'd need to start a survey.

I also agree on Issue 2815 - this is a duplicate of it. As this issue however has more input, I'd suggest to close issue 2815 or switch over as soon as we have pinned down how we proceed to keep the issue short and hopefully not cause confusion.

Okay so....

I hope the following does help better to understand the scopes.

Replies

When replying to a mail -no matter if in Zammad or not- where your mail client does quote the earlier text, it will add a short quote text e.g. like On 24th December 2020 John Doe wrote: followed by the quote.

This is useful if you quote several texts from maybe even several mails (which Zammad does support). It always does use the sender of the article, so basically the display name of the FROM.

A mail reply does always contain the original sender. In fact when hitting reply, your recipient will always be the sender of the mail you reply to.
This means that you always have all information in order to communicate with the original sender of that mail. A mail address within the qutoation intro is not needed at all.

Forwards

Forwards -at least usually- are supposed to send information to another third party. No matter if that's another Zammad system or an individual.
You usually forward a mail, because

• you either want to share information with a third party "for their information"
• the sender did choose the wrong mail address, you're not responsible but to help the sender forward the mail to the responsible person/system

During the process of forwarding (because TO and CC are not pre filled), you will loose the information who that email originated from.
This is the main reason, why the quote intro text provides the mail address like e.g.: On 24th December 2020 John Doe <[email protected]> wrote:

This will allow the recipient to get in direct touch with John if needed. If you simply reply to an forwarded message, you'll answer to the forwarding sender which in this case would be Zammad and the wrong recipient.

This causes agents either

• to think of that issue (we're not providing the mail address) and thus manually insert the mail address of the customer to allow the recipient of the forward to get in touch with the customer
• or having another question of the forwarding recipient "who shall I sent that to??" (causing unneeded ping pong)
• or the forwarding recipient to just reply to the mail and enforce the agents to forward this information to the customer

This is why Zammad should allow administrators to provide these information during forwards. In fact I think that this should become the default option for new instances. On existing instances I believe that this by default should be disabled.

Now to the use cases (now it gets complicated).

1. (current default) quote displayname only

Currently when forwarding a mail from Zammad to a third party, Zammad will use On 24th December 2020 John Doe wrote: as quote text.
While it tells you who wrote the message, this does also require the recipient of the Mail to know John Doe and thus the mail address to write to.

Personally I think there's a use case where you want Zammad to do exactly that:

• If you're supporting your customers but you don't handle the 2nd or 3rd level, you'll forward your clients mail to your 2nd/3rd level with maybe additional information you earlier provided.
  
  
  
  • in this use case you do not want your customer to notice that you're actually not doing all the hard word but just "proxying" the requests to someone else
  
  
  • To protect your customer, you'll only provide his name, but not mail address
  
  
  • To ensure quality of the reply to the customer (or because your 2nd level uses a different language than your customer), you're enforcing replies on the forwarded mail to go to Zammad
  
  
  • this gives you the full control of what information will be sent to your customer
  
  

Personally I think this use case is valid for ~20% of our user base.
This is why I think we only should have this activated as default for updating installations to allow backward compatibility. This also ensures users don't run into a new behavior they do not want and know.

2. quote displayname and FROM email address only

This will provide only the senders mail address to be provided.
This can be usefull if your customers work a lot with CC to inform they bosses, but you don't want the receiving part of your forward to see all those addresses.

I Personally think that this use case does affect less than 10% of our userbase. I honestly can't think of a use case (except the very thin one above) where you'd want half baked information

3. (default for new instances) quote displayname and all recipients of original mail

Especially if using a lot of CC's to inform more than one person, it'd be better to provide your forward recipient with all mail addresses because

• it reduces workload on the original sender to ensure all persons do receive the information
• the recipient of the forward can provide the information to all original recipients if needed
• the recipient of the forward might see scopes or priorities better (e.g. if the ceo is informed as well, he may want to priotize his answer :D )

With this step we will get close on the way other mail clients do behave.
This helps the agent/user because

• Zammad behaves like the user is used to from his mail client
• Changing from a normal email client to Zammad does no longer hurt as much if it at least behaves similar to a mail client (even though it's still different)
• the agent saves time for other stuff to do

In the end Zammad is supposed to help the agents, not to provide more work load ;x

I think that this will affect 70% and more of the user base.
I personally do prefer this behaviour.

I think behaviour 1 and 3 are the most important ways on how Zammad does work.

Mantas pointed out that it might be confusing to simply leave out the agents mail address and thus suggested to user the groups mail address.

I am however worried that this might lead to bug reports because Zammad might choose a mail address the user thinks is the wrong one.

The other possibility apart from removing agents or to use a mail address of Zammad would be to use John Doe <redacted> for agents.

I've read through and I would agree to all of it! Thanks for this great writing. I would love to see 3 coming into Zammad. 2. is maybe a "nice to have" if it doesn't add too much of a hassle as I also see it as a method with a very small user base.

For the very last part, in my opinion you should rather forward mails as Agents Name + Groups Mail like Martin S. <support@>, so you at least know the name of the Agent behind for a reply (if you need to answer you can say "Hi Martin, you forwarded me a mail i still have a question for" or something. What do you think?

For everything else you wrote -> :100: on your side :) Hope to see it soon, we can use it finally and save some work time.

Thanks a ton guys @martinseener @MrGeneration and @mantas ❤️

I'm convinced and totally on your side. To follow our philosophy we will skip option 1 and 2 completely and replace the current default behaviour (2) with 3 for all instances.

IIRC this was the original intend of the previous PR #3014 . Now that we specified the requirements and use cases/user stories in more detail I can see clearer. Thanks!

Depending of the size of the change I'll consider having it backported to 3.4 as well.

Looking forward to review the MR!

Any preferences on exact style?

I was clicking around several email clients and I'm warming up to @martinseener 's proposed Thunderbird default style

When more details are added, it's much easier to read than the human reply style. Even if we don't store full headers, we could imitate this style with the details we got.

@thorsteneckel which approach do you prefer for hiding agent emails? Printing name only or name <redacted>, name <[email protected]>?

@mantas I agree. When adding more informations, something that is readable and familiar (Thunderbird) is maybe a good option.
@thorsteneckel from my PoV it's maybe good to have only the Name without a partially redacted email since it's then easy to guess the full email. Then you could just put the full email in, so all or nothing I would say.

Apple mails uses (basically) the same format:

From: Marcel <[email protected]>
Subject: Re: [zammad/zammad] Make forwarding original FROM mail header configurable in UI/config (#3091)
Date: 19. June 2020 at 10:11:16 CEST
To: zammad/zammad <[email protected]>
Cc: Thorsten <[email protected]>, Mention <[email protected]>
Reply-To: zammad/zammad <reply+AA5RV25L5H5YCFNQT3KMG3547BKCJEVBNHHCMNFNPQ@reply.github.com>

👍

Can someone please add a short notice on how you proceed internally with this ticket now? Is there any roadmap or rough plan on when this will be available?

We're actively working on it. I'll make sure it doesn't get stuck in limbo :)

Hi @martinseener - it's done thanks to @mantas ! The release will be available in about 30 minutes from now. You can just update your Zammad via the system package manager and should be ready to go. Looking forward to your feedback 🚀

JFI: I'll accidentely hit "update" on your paid instance on the hosted setup later on (~19:00 - 20:00 ). So tomorrow will be a better day. 🙌

@thorsteneckel @mantas @MrGeneration thanks alot for that. I just upgraded to 3.4.0-1594825410.4cacfa4b via my system manager (also to ES 7.8). When now forwarding, the full header is visible there. Nice!One question regarding attachments though. I don't see an option to forward the mail including the attachment if there is one. Is there a way to also forward them (by default)? Maybe an additional ticket?

Zammad should always forward with attachments.
Your issue sounds like the following bug: https://github.com/zammad/zammad/issues/2942

@MrGeneration i'm sorry. I think I did a mistake.
I discarded the Forwarding-Draft and pressed Forward again. Now I see the attachment being in the Forward part and when i send this, the attachment is also really send too. So no issue here.