WebAPI/WebActivities/LessonsLearned

Lessons Learned

Based on internal experience, as well as experience talking with other partners like Google and Pocket, we have learned a lot about how to design the next iteration of WebActivities.

To simplify the discussion, here's two standard flows when using WebActivities

Using an activity that returns a value

1. User launches facebook
2. User clicks "attach a picture"
3. Facebook launches the "pick" WebActivity
4. The activity picker comes up.
5. User chooses to use the camera app as activity handler
6. Camera app is launched
7. User takes a picture
8. Camera app returns the picture as result of the activity
9. Camera app is closed and user is back in facebook app.

Using an activity that does not return a value

1. User goes to news website and finds an interesting article
2. User clicks "share" button
3. News website lauches the "share" WebActivity
4. The activity picker comes up.
5. User chooses to use the twitter app as activity handler
6. Twitter app is launched
7. Twitter app prefills URL of news article as tweet content
8. User writes short message to go along with the URL and submits the tweet

UX issues with activities that return a value

In FirefoxOS, if the user in step 7 of the returns a value flow temporarily switches app, for example to look something up, or to answer an incoming phone call, then the activity is aborted and facebook receives an error message. We were forced to do this in order to deal with the situation when the user might switch back to the facebook app, in which case we didn't want to force facebook to deal with having to implement a sensible UI while the activity was still in progress. However aborting the activity just because the user looked something up is obviously bad.

In Chrome's Web Intents, step 6 of returns a value flow launches the camera app in a separate tab in the browser. That creates an awkward relationship between the tab with the facebook app and the tab with camera app. In particular, if the user closes the facebook tab, then when the picture is taken, it's dropped on the floor since there is no facebook app to receive it.

Recommended solutions For activities that return a value, the activity handler should be overlayed on top of the initiating app. I.e. in the example above the camera app should be rendered on top of the facebook app.

In a small-screen device this means that when the application switcher shouldn't display facebook and the camera as two separate apps currently running. Instead only the facebook app should be listed, though possibly the UI could somehow indicate that the facebook app is currently using the camera app or some such. Switching to the facebook app should render the camera app on top of it.

On a large-screen device with a traditional tabbed browser UI, the camera app should be rendered in the same tab as the facebook app. I.e. the camera app would be on top of the facebook app.

UX issues with opening in activities in a new app/tab

In FirefoxOS a webactivity always launches a new fullpage app. This makes activities always have a fairly heavy flow since it involves two app switches, one to the activity handler, and one to switch back to the original app.

Activity handlers that want to implement things like "save this for later" or "share on my photo stream" only needs to display minimal UI that doesn't take the user out of the current app.

Recommended solutions We should implement a "disposition: 'inline'" like what Web Intents has. This is already specified for WebActivities but was never implemented. An inline handler would be rendered like a "dialog" on top of the current app.

On large-screen devices this likely means that it sizes to content. On small-screen devices this might simply mean that it doesn't take up the full screen.

Pocket also has dome some really great research here.

This research proposes even allowing overlays that render directly on top of the initiating app. This provides for some pretty awesome UI, but also exposes issues like clickjacking. Ideally disposition:inline can bring most of the benefits of this proposal, while still not exposing clickjacking risks.

UX issues if the handler app is already running when the activity is launched

In FirefoxOS's implementation of WebActivities, if the twitter app was already running in the background when the does not return a value flow happens, we switch to the twitter app and send a message to it and ask it to handle the activity. However this means that the app has to leave it's current state in order to do so. So if the user was in the middle of some other task within the twitter app, that is now lost.

Recommended solutions

If we follow the recommendation above for UX issues with activities that return a value that actually solves the problem for activities that return a value. A new page will always be opened and rendered on top of the app that initiated the activity.

Likewise disposition:inline activities will also not suffer this problem since they open a new page inside the inline UI on top of the page that initiated the activity.

For other activity handlers a good solution might be to fire an event in the handlers ServiceWorker and let the worker decide if a new page should be opened, or if an existing page could be used. But a reasonable default seems like always opening activity handlers in a new context. I.e. only allow reusing an existing context by registering a ServiceWorker as the handler.

Lack of ability to save intermediate results

Consider a "Google Drive" page that uses an "edit" activity to launch a "photoshop" page in order to edit a picture file.

In the current WebActivities and Web Intents implementations, the only way to accomplish this would be to have the photoshop "edit" activity handler return the edited image once the user was done editing, and then have the Google Drive page save the resulting file.

There are a couple of issues with this flow though. First off given that the activity would be one returning a value, the photoshop app would have to be opened on top of Google Drive. This isn't always desirable.

A bigger problem is that photoshop would not be able to save intermediate drafts to the Google Drive page. It would either not save them at all, which means risking more dataloss in case of a crash or accidentally closing the page, or it would have to save them somewhere in photoshop's storage area. In case of crash it would be awkward to get the edited data back into Google drive. The user would have to relaunch the edit activity and select photoshop again, and then photoshop would have to detect that the same file was being edited and offer to reuse the previously saved draft.

A desired flow here is instead to enable Google Drive to launch the "edit" activity such that Photoshop could be opened in a new window. But also enable Photoshop to have an open communication channel back to Google Drive. However the user should be able to close the Google Drive tab while still enabling Photoshop to send data to Google Drive in order to save intermediate drafts.

Recommended solutions When launching an activity, it should be possible to also provide "back channel" information. If provided, the activity handler would be able to postMessage arbitrary information back to the app that initiated the activity.

These messages would likely need to be sent not to the execution context that initiated the activity, but rather to its Service Worker. This way messages can be sent even if the execution context that initiated the activity has been closed.

The app initiating the activity would also need to provide some arbitrary data which is passed back any time messages are sent from the activity handler and which can't be altered by the activity handler. In the example above the Google Drive app could provide the filename of the file being edited.

Ability to switch from inline to full-page handler

A disposition:inline handler might need to defer to a more complex UI depending on user actions. For example a facebook "share" handler might start as an inline handler, but need to switch to a more complex UI if the user wants to configure security settings or add complex data to the post.

This gets especially tricky for activity handlers that return a value. In this case the full-page handler would need to be rendered on top of the current page, I.e. it couldn't be handled like a normal window.open with target=_blank. Additionally the new page needs to take over the responsibility of returning a value.

Recommended solution

Sadly I don't have any recommendations here. Possibly simply not supporting this scenario is the right solution for now. Instead we can allow a display:inline handler to resize itself to handle the more complex UI.

And activity handlers that don't return a value can always open new tabs using target=_blank links or window.open calls.

Should activity launcher have a say in disposition of the handler?

This is mostly based on notes from a WebActivities/Web Intents discussion. I sadly don't remember all the details here.

For activities that does not return a value, the initiating page might want to treat launching the activity handler as either a "navigation" or as a "open in new tab".

In the Google Drive/Photoshop example above, it seems like it should be the decision of the Google Drive app if Photoshop should replace the Google Drive page, or if opening Photoshop should be treated like opening a <a target="_blank"> link.

Recommended solution Potentially we could support a target attribute when initiating an activity. However I don't know of use cases which would involve targeting named windows.

Another question is if it should be possible to target _self if the current page is open in an iframe. I.e. should activity handlers need to worry about possibly being opened in a subframe?