Firefox OS/Cloud Storage
Overview
This project aims at to support a framework for web apps to interact with cloud storage.
Cloud Storage framework wants to support following features
- Universal Storage API
Universal Storage API is a general API for web apps to access all kinds of cloud storage.
- Virtual Storage Interface
Virtual Storage Interface is plugin system for cloud storage framework to extend cloud storage option.
Cloud Storage Framework
Universal Storage API
Universal Storage API is a general API for web apps to access all kinds of storage.
According to different purposes, Universal Storage API can be separated into following two categories
- Document based API
Document based API is used to help web apps save/operate their application data, such as indexedDB, on the cloud storage.
- Filesystem based API
Filesystem based API supports web apps to access cloud storage with filesystem operations, such as file create, delete, open, read, write, close, etc.
Reference for Filesystem based API: W3C FileAPI
filer
DeviceStorageAPI
Virtual Storage Interface
Virtual Storage Interface is plugin system for cloud storage framework to cooperate with different cloud storage.
Cache Architecture
Background
- Improve performance
- The response time of each file operation should be as short as possible to provide better user experience when users want to view a photo, listen an audio, or watch a video.
- Offline Support
- Cache store is a subset of cloud storage, so Cache system still can provide cached file content from cache store even no network connection available.
Architecture
- Cache Manager
- Dispatch the request from Cloud Storage Interface to Data/Meta Cache first
- Get data from Web Storage Manager if there is no data existed in Cache.
- Cloud Storage Interface
- The interface for using Cloud Storage cache is a general design. This can be used in JS Library or Gecko implementation.
- Meta Cache
- Each file has its meta data like mdate, cdate, size, etc. These information should be cached.
- Data Cache
- Data Cache should store a file content requested by user for each file in order to avoid extra network transaction.
- Web Storage Manager
- Provide an interface for each web storage provider (Dropbox, Box.net, etc) to communicate with Cache Manager.
- 3rd party plugin could be in the form of add-on for developer to support new web storage easily.
- Cache Policy
- Cache store entry can be flexible to adjust for different requirement, i.e. storage size. Any new request can replace one existed entry if the maximum entry is exceeded. ‘existed entry’ has not been well-defined yet, and it depends on user story.
- Cache policy is a set of method to prefetch file content or meta aggressively, so the response time for each request can be improved.
Data Cache Implement Proposal
- File-based Cache
- minimum data chunk - File
- Block-based Cache
- minimum data chunk - Block (with fixed-size)
- File System level operation can be mapped to cloud storage.
| File-based | Block-based | |
|---|---|---|
| Cross Platform/Browser | Yes | TBD... |
| Response Time for large-size file (ex: video) |
poor | good |
| Response Time for small-size file (ex: photo, audio) |
good | good (depends on block-size) |
| Implementation Idea | Gaia: JS Library (Filer) | Gaia: TBD... Gecko: FUSE |
| Related Web API | Blob, MSE(?), ... | Blob, Device Storage, ... |
Reference
Proposed Solution(s)
FUSE based solution
Mounting Cloud Storage to device filesystem through the FUSE mechanism.
Benefits
- DeviceStorageAPI provides well defined filesystem based API to access files/directories on internal storage and SD card. Once FUSE based solution supported, Web app can used DeviceStorageAPI directly to access cloud storage without any modification.
- FUSE interface is a good reference for Virtual Storage Interface. By propagating FUSE interface to FirefoxOS, FirefoxOS can easy connect to many kinds of storage, such as cloud storage, NAS, etc. Here is an commercial example.
Drawbacks
- DeviceStorageAPI is not a standard API for all browser.
- FUSE is not supported in all platforms.
Prototype
- MozCloudStorage github repo
- CloudStorageServiceAPI (this is only for prototype)
CloudStorageServiceAPI is a WebAPI for enabling/disabling cloud storage.
enum CloudStorageType {
"Dummy", "Dropbox",
};
Promise<boolean> enable(DOMString cloudName, CloudStorageType cloudType, DOMString accessToken)
Promise<void> disable(DOMString cloudName)
JS library solution
Benefits
Drawbacks
Framework Issues
- Cloud Storage Account management and Authentication
- Security issues
Issue Study
- Video Support with MSE
- Media Source Extensions
- The challenge with HTML5 audio and video is still the fragmented support for audio and video formats.
- In Firefox browser, the maximum size of each fragment is 75MB. Getting QuotaExceededError if the fragment size is over 75MB.
- Fragmented Video Format - the video MUST be in fragmented format, and this video can be played well.
- Non-fragmented Video - this more general format can be played in MSE way. Browser treats a non-fragment video as a one big fragment file because the maximum size of a non-fragment video is 75MB.
Related Use cases
Play media files saved on cloud
John saves his media files on dropbox or any other cloud storage and wants to play these media online on his mobile device.
In traditional, John needs to do with following steps
- Open the cloud storage app, such as dropbox and google drive, and download the media files to the mobile device.
- Open the player app, such as Gallery and Music, and play the media files locally.
On FirefoxOS device, the steps can be reduced to
- Open the player app and play the media files on cloud storage directly.
Bob wants to take a picture and share it with friend on flickr.
In traditional, Bob needs to do with following steps
- Open the camera app to take a picture and save the file on the device.
- (Optional) Open a third party app to edit the picture on the device.
- Open the flickr app to upload the file and set it as shared.
On FirefoxOS device, the steps can be reduced to
- Open the camera app to take a picture and save and share the file on cloud storage directly.
- (Optional) Open a third party app to edit the picture on cloud storage directly.
File management between clouds
Amy wants to copy a finished work document form her personal dropbox space to the company google drive space
In traditional, Amy needs to do with following steps
- Open the dropbox app and download the document file to device.
- Open the google drive app and upload the document file.
On FirefoxOS device, the steps can be reduced to
- Open a file browser app and copy the document from dropbox to google drive directly.