MORG BOF

1
MORG BOF

• IETF 72, DublinJuly 30th, 2008
• Chairs
• Alexey Melnikov ltalexey.melnikov_at_isode.comgt
• Randall Gellens ltrandy_at_qualcomm.comgt
• Mailing List ltmailtomorg_at_ietf.orggt
• Jabber morg_at_jabber.ietf.org

2
Note Well

• Any submission to the IETF intended by the
  Contributor for publication as all or part of an
  IETF Internet-Draft or RFC and any statement made
  within the context of an IETF activity is
  considered an "IETF Contribution". Such
  statements include oral statements in IETF
  sessions, as well as written and electronic
  communications made at any time or place, which
  are addressed to
• the IETF plenary session,
• any IETF working group or portion thereof,
• the IESG or any member thereof on behalf of the
  IESG,
• the IAB or any member thereof on behalf of the
  IAB,
• any IETF mailing list, including the IETF list
  itself, any working group or design team list, or
  any other list functioning under IETF auspices,
• the RFC Editor or the Internet-Drafts function
• All IETF Contributions are subject to the rules
  of BCP 78 and BCP 79.Statements made outside of
  an IETF session, mailing list or other function,
  that are clearly not intended to be input to an
  IETF activity, group or function, are not IETF
  Contributions in the context of this notice.
• Please consult BCP 78 for details.

3
Agenda
4
MORG BOF introduction

• MORG Message ORGanization
• Goals
• improve ability to find messages in an IMAP
  mailstore
• Don't make things worse while doing that (think
  about extension interaction)
• Should work for diverse clients
• Mobile devices, web clients, desktop clients

5
MORG BOF history

• Various proposals were suggested over the years
• New SEARCH/SORT/THREAD criteria
• Extensions to SEARCH
• Multimailbox searches, virtual views
• Message grouping and quick ways of returning
  information about number of messages in such
  groups
• Message contexts (e.g. voice/fax messages),
  STATUS-in-LIST, per-mailbox annotations

6
SEARCHINTHREADandTHREADREFS

• draft-gulbrandsen-imap-inthread-02.txt
• Presented by Timo Sirainen

7
SEARCHINTHREAD extension

• Adds 4 new SEARCH keys
• INTHREAD takes two arguments, a threading
  algorithm (as defined in SORT) and another
  search key.
• The INTHREAD search-key matches a message if its
  second parameter matches at least one message in
  the same thread as the message.
• Alexey Is this the same as return all messages
  in threads, which contain at least one message
  matching the specified search key?

8
SEARCHINTHREAD extension(continued)

• THREADROOT Search Key
• The THREADLEAF Search Key
• The MESSAGEID Search Key
• Similar to ltHEADER Message-Id valuegt, but
  normalizes the value

9
THREADREFS extension

• Like THREADREFERENCES, but
• No merging threads based on subject
• References header field is supported well enough
  nowadays
• False positives unrelated threads merged
• Sort thread roots based on the last received
  message within the thread, not by the roots
  Date header
• New messages are expected to be at the top/bottom
  of the mailbox

10
THREADREFS Open Issues

• Should THREADREFS use the In-Reply-To header
  field value if there is no References header
  field?

11
New address sort extension to SORT

• draft-karp-morg-sortdisplay-00.txt
• Presented by Alexey Melnikov

12
IMAP4 Multimailbox SEARCH Extension

• draft-melnikov-imapext-multimailbox-search-03.txt
• Barry Leiba Alexey Melnikov

13
Recent changes

• Clarified that a multimailbox SEARCH/UID SEARCH
  always returns UIDs in the ESEARCH response
• Updated references

14
Open Issues

• Replace DEPTH option with something used by IMAP
  NOTIFY? I.e.
• Change tag1 SEARCH IN (("folder1" "folder2/")
  (depth 1)) unseen
• To tag1 SEARCH IN (Subtree ("folder1"
  "folder2")) unseen
• Need to define interaction with IMAP CONTEXT
  (draft-cridland-imap-context)
• Might also have to prohibit UPDATE option from
  draft-ietf-lemonade-imap-notify

15
Multimailbox SEARCH suggested changes

• Timo ESEARCH replies should include mailbox
  UIDVALIDITY
• Timo alternative proposal create virtual view
  from multiple mailboxes

16
IMAP Extension for per-mailbox/per-server
attributes

• draft-daboo-imap-annotatemore-13.txt
• Cyrus Daboo

17
STATUS-in-LIST

• draft-melnikov-imapext-status-in-list-00.txt
• Alexey Melnikov Timo Sirainen

18
STATUS-IN-LIST

• Most clients display mailbox list with number of
  (unseen) messages
• Requires LIST and STATUS for each listed mailbox
• Do it all in one command LIST "" RETURN
  (STATUS (MESSAGES UNSEEN))
• Requires LIST-EXTENDED
• Advantages
• Avoid one round-trip (or more)
• save some bandwidth
• allow server to optimize the lookup more easily

19
STATUS-IN-LIST Protocol

• Standard LIST and STATUS replies are returned to
  make it easy for clients to implement support for
  the extension
• 1 LIST "" RETURN (STATUS (MESSAGES UNSEEN))
• LIST () "." "INBOX"
• STATUS "INBOX" (MESSAGES 17 UNSEEN 16)
• LIST () "." "foo"
• STATUS "foo" (MESSAGES 30 UNSEEN 29)
• 1 OK

20
STATUS-IN-LIST Issues

• STATUS replies add more potential failure cases,
  such as
• ACLs prevent STATUS reply
• Internal server failures prevent a successful
  STATUS lookup (but LIST succeeds)
• These are normally handled by a NO reply to
  STATUS
• Possible ways to let clients know about them
• Return \NoSelect and no STATUS (good for ACLs)
• STATUS replies with empty parameters
• NO reply to LIST if any STATUS replies are missing

21
New collations (comparators) for IMAP and Sieve

• Alexey Melnikov

22
Suggested comparators

• Ned Freed space ignoring comparator (e.g. for
  Japanese texts)
• Signed numeric comparator (iascii-numeric
  doesn't deal with leading '-', '' or SP)
• Telephone number mapping ignore punctuation
  such as SP, '-', '(' and ')'
• Case sensitive version of iunicode-casemap?

23
Sorting by Relevancy

• All modern search engines return results sorted
  by relevancy to the search query. IMAP should
  allow the same.
• How the relevancy is calculated or presented is
  implementation specific.
• The scores may be calculated differently in
  different installations or even different
  connections (client location, users language).

24
Sorting by Relevancy

• Relevancy score is search query-specific, so its
  value cant really be FETCHed.
• New SCORE SORT key
• SORT (SCORE DESC) UTF-8 TEXT hello world
• Highest score highest relevancy, so should DESC
  be used to get highest scores first?

25
Sorting by Relevancy

• New SCORE SEARCH key?
• SEARCH TEXT hello world SCORE 0.5
• Is this useful? It would require some kind of a
  standardization of score values, which may make
  it difficult to implement with some search
  engines.
• Would clients still want to know the score and
  display it to the user in some way (color,
  percentage, ..)? How would this be possible?
• FETCH SCORE would be updated after each SEARCH
• SORT/SEARCH returns scores in untagged reply

26
Flag/Keyword Management

• Allow explicitly adding and removing keywords
• This allows client UI to display a list of
  existing keywords that can be assigned to
  messages
• Currently keyword with typos or keywords that are
  no longer relevant cant be removed
• Allow specifying whether flag or keyword should
  be private or shared in shared mailboxes
• Different use cases require different
  configurations
• Notify client about these

27
Flag/Keyword Management

• Send shared flag state whenever FLAGS and
  PERMANENTFLAGS are sent
• OK SHAREDFLAGS (\Seen \Answered hello)
• Add keywords or change private/shared state of an
  existing flag/keyword
• FLAGS ADD PRIVATE (\Answered pvt1) SHARED
  (\Flagged world)
• Remove keywords
• FLAGS REMOVE (pvt1 world)

28
Flag/Keyword Management

• Backwards compatibility with clients not
  supporting the new extension
• Allow server to drop unused keywords created
  with STORE
• But dont allow dropping explicitly created
  keywords?

29
Flag/Keyword Management

• Interaction with ACL?
• w right to create new keywords?
• a right to modify private/shared state of
  system flags? What about keywords created by you?
  Keywords created by someone else?
• Allow user to create a private keyword to
  override an existing shared keyword, effectively
  hiding the shared keyword from the one user?

30
Proposed WG Charter Discussion
31
Basic questions

• Does the list of drafts/ideas presented include
  documents you will be willing to work on?
• Does this look like a WG?

32
Proposed WG description

• The IETF Message Organization extensions Working
  Group is tasked to work on IMAP extensions that
  improve clients ability to find messages or group
  of messages in an IMAP mailstore. This includes
  an extension for multimailbox searches,
  extensions defining new search and sort criteria,
  an extension defining client controlled per
  mailbox attributes. As the secondary goal the WG
  will try to ensure that designed extensions
  minimize number of round-trips and and bandwidth
  overhead.

33
Proposed workitems (1 of 2)

• note that not all of them will become WG items
• (a) The IMAP SEARCHINTHREAD and THREADREFS
  Extensions (draft-gulbrandsen-imap-inthread-02.txt
  )
• (b) New search method using prefix match and a
  comparator (TBD) ?
• (c) New address sort criteria extending the SORT
  extension (draft-karp-morg-sortdisplay-00.txt)
• (d) IMAP4 Multimailbox SEARCH Extension
  (draft-melnikov-imapext-multimailbox-search-03.txt
  )
• (e) IMAP Extension for per mailbox and per server
  attributes (draft-daboo-imap-annotatemore-13.txt)

34
Proposed workitems (2 of 2)

• (f) STATUS-in-LIST (draft-melnikov-imapext-status-
  in-list-00.txt)
• (g) Formalize a way to return counters by message
  types (e.g. voice, fax) using STATUS command and
  ESEARCH extension
• (h) New comparators (TBD) ?
• (i) Sorting by relevancy (TBD) ?
• (j) IMAP keyword management (TBD) ?
• (k) Mailbox type management extension (TBD) ?