Firefox OS/Cloud Storage

Overview

FirefoxOS cloud storage support is a project that supports a framework to integrate user's cloud storage into their FirefoxOS device.

Current Cloud Storage User Scenario

In android or iOS, viewing a media file on cloud, it has following possible ways.
1. Using one viewing app and one cloud app. It needs following steps.

1. Using the cloud app to download the media file to device.
2. Using the viewing app to open the downloaded media file.

In this way, user must download the whole media file first. Download brings following drawbacks.

• File duplication.

File occupies the local storage. If device local storage is almost full, user might need to cleanup the local storage first.

• Need to download the whole file before viewing.

Some files might be not suitable to be downloaded, for example 4k video files.

2. Using a viewing app which support the specified cloud accessing. It needs following steps.

1. Using the viewing app to open the media file on cloud.

In this way, user can view the media in stream way. However, the viewing app must to support the user preferred cloud. Otherwise, user might select another app or cloud they don't want.

In this project, we want to support user

• Choose their preferred apps and cloud in free.
• No need to save the whole file in local.

Cloud Storage Support Framework

Framework

The basic idea of this framework is mounting cloud storage as a FirefoxOS fake volume in FirefoxOS Volume System. By mounting cloud as a fake volume, FirefoxOS becomes a proxy for apps to access user's cloud, such that apps needn't know how to access cloud data in special way and maintain cloud by themselves. On the other hand, FirefoxOS can support streaming access to avoid downloading the whole file into local storage.
Instead of accessing cloud directly in Gecko, we design FileSystemProvider API to provide apps/addons can create a virtual file system in Gaia, and accessing cloud in these apps/addons. According to this design, we can gain following benefits

• No need to maintain third party cloud APIs in Gecko.

It means FirefoxOS no need to FOTA to support the third party cloud APIs upgrade. It only needs to update the corresponding Cloud Storage Addons.

• Easy to reuse the framework on other kinds of storage.

Not only cloud storage, but also NFS, personal NAS can reuse this framework to plug into FirefoxOS Volume System, such that apps can also interact with these storage directly.

FirefoxOS Volume System

FirefoxOS Volume System is a middle layer between FirefoxOS and low-level OS kernel. It supports basic volume operations, such as mounting, unmounting, and formatting, to manage volumes in the device. In addition to the basic volume operations, the Volume System also provides fake volume creation and deletion mechanism.
In this framework, we creates fake volumes for each connected cloud, therefore these connected cloud can be recognized, identified and accessed by apps through DeviceStorage API and File API.

DeviceStorage API and File API

DeviceStorage API and File API are interface used by apps to access FirefoxOS file systems. Apps can use these APIs to perform file read, write, creation, and deletion. Here are more detail about DeviceStorage API and File API
In this framework, we let apps still use these APIs to access cloud data. It means apps can access user's cloud without any modification which just like they access the device local storage.

FileSystemProvider API

FileSystemProvider API provides apps/addons capability to create virtual file systems in Gaia. It provides

• Methods to request Volume System to create/delete a fake volume.
• Events and callbacks to request data from virtual file systems created by apps/addons.

Types

OpenFileMode

Mode for open a file.

Enum

"READ" or "WRITE"

MountOptions

Options for the mount method.

Properties
DOMString	fileSystemId	The string identifier of the file system. It must be unique for each file system.
DOMString	displayName	The fake volume name.
boolean	writable	Writable file system.
unsigned long	openedFilesLimit (optional)	The maximum number of opened files.

UnmountOptions

Options for the unmount method.

Properties
DOMString	fileSystemId	The string identifier of the file system.

OpenedFile

Represents an opened file.

Properties
unsigned long	openRequestId	The open request ID.
DOMString	filePath	The opened file path.
OpenFileMode	mode	The mode of open file.

FileSystemInfo

Represents a mounted file system.

Properties
DOMString	fileSystemId	The string identifier of the file system.
DOMString	displayName	The fake volume name.
boolean	writable	Writable file system.
unsigned long	openedFilesLimit	The maximum number of opened files.
sequence <OpenedFile>	openedFiles	The sequence of opened files.

FileSystemProviderError

FileSystemProvider error types.

Enum

"Failed", "InUse", "Exists", "NotFound", "AccessDenied", "TooManyOpened", "NoMemory", "NoSpace", "NotADirectory", "InvalidOperation", "Security", "Abort", "NotAFile", "NotEmpty", or "InvalidURL"

EntryMetadata

Represents metadata of a file or a directory.

Properties
boolean	isDirectory	True if it is a directory.
DOMString	name	Name of this entry (not full path name). Must not contain '/'. For root it must be empty.
unsigned long long	size	File size in bytes.
DOMTimeStamp	modificationTime	The last modified time of this entry.
DOMString	mimeType (Optional)	Mime type for the entry.

UnmountRequestedOptions

Options for unmountrequested event.

Properties
DOMString	fileSystemId	The string identifier of the file system.
unsigned long	requestId	The unsigned integer identifier of this request.

AbortRequestedOptions

Options for abortrequested event.

Properties
DOMString	fileSystemId	The string identifier of the file system.
unsigned long	requestId	The unsigned integer identifier of this request.
unsigned long	operationRequestId	The unsigned integer identifier of the aborting request.

GetMetadataRequestedOptions

Options for getmetadatarequested event.

Properties
DOMString	fileSystemId	The string identifier of the file system.
unsigned long	requestId	The unsigned integer identifier of this request.
DOMString	entryPath	The path of the entry to fetch metadata about.

ReadDirectoryRequestedOptions

Options for readdirectoryrequested event.

Properties
DOMString	fileSystemId	The string identifier of the file system.
unsigned long	requestId	The unsigned integer identifier of this request.
DOMString	directoryPath	The path of the directory which contents are requested.

OpenFileRequestedOptions

Options for openfilerequested event.

Properties
DOMString	fileSystemId	The string identifier of the file system.
unsigned long	requestId	The unsigned integer identifier of this request.
DOMString	filePath	The path of the file to be opened.
OpenFileMode	mode	Whether the file will be used for reading or writing.

CloseFileRequestedOptions

Options for closefilerequested event.

Properties
DOMString	fileSystemId	The string identifier of the file system.
unsigned long	requestId	The unsigned integer identifier of this request.
unsigned long	openRequestId	The unsigned integer identifier of the corresponding open request.

ReadFileRequestedOptions

Options for readfilerequested event.

Properties
DOMString	fileSystemId	The string identifier of the file system.
unsigned long	requestId	The unsigned integer identifier of this request.
unsigned long	openRequestId	The unsigned integer identifier of the corresponding open file request.
unsigned long long	offset	Position in the file (in bytes) to start reading from.
unsigned long long	size	Number of bytes to be returned.

CreateDirectoryRequestedOptions

Options for createdirectoryrequested event.

Properties
DOMString	fileSystemId	The string identifier of the file system.
unsigned long	requestId	The unsigned integer identifier of this request.
DOMString	directoryPath	The path of the directory to be created.
boolean	recursive	Whether the operation is recursive (for directories only).

DeleteEntryRequestedOptions

Options for deleteentryrequested event.

Properties
DOMString	fileSystemId	The string identifier of the file system.
unsigned long	requestId	The unsigned integer identifier of this request.
DOMString	entryPath	The path of the entry to be deleted.
boolean	recursive	Whether the operation is recursive (for directories only).

CopyEntryRequestedOptions

Options for copyentryrequested event.

Properties
DOMString	fileSystemId	The string identifier of the file system.
unsigned long	requestId	The unsigned integer identifier of this request.
DOMString	sourcePath	The source path of the entry to be copied.
DOMString	targetPath	The destination path for the copy operation.

MoveEntryRequestedOptions

Options for moveentryrequested event.

Properties
DOMString	fileSystemId	The string identifier of the file system.
unsigned long	requestId	The unsigned integer identifier of this request.
DOMString	sourcePath	The source path of the entry to be copied.
DOMString	targetPath	The destination path for the copy operation.

CreateFileRequestedOptions

Options for createfilerequested event.

Properties
DOMString	fileSystemId	The string identifier of the file system.
unsigned long	requestId	The unsigned integer identifier of this request.
DOMString	filePath	The path of the file to be created.

WriteFileRequestedOptions

Options for writefilerequested event.

Properties
DOMString	fileSystemId	The string identifier of the file system.
unsigned long	requestId	The unsigned integer identifier of this request.
unsigned long	openRequestId	The unsigned integer identifier of the corresponding open file request.
unsigned long long	offset	Position in the file (in bytes) to start writing the bytes from.
ArrayBuffer	data	Buffer of bytes to be written to the file.

Methods

mount

navigator.fileSystemProvider.mount(MountOptions options)

Mount a file system with the given MountOptions.

Parameters
MountOptions	The mount options for mount a file system.
Return value
Promise<void>

unmount

navigator.fileSystemProvider.unmount(UnmountOptions options)

Unmount a file system by the given UnmountOptions.

Parameters
UnmountOptions	The unmount options for unmount a file system.
Return value
Promise<void>

get

navigator.fileSystemProvider.get(DOMString fileSystemId)

Returns information about a file system with the passed fileSystemId.

Parameters
DOMStrig	The file system ID.
Return
FileSystemInfo

getAll

navigator.fileSystemProvider.getAll()

Returns all file system information mounted by FileSystemProvider API

Return value

sequence<FileSystemInfo>

Events

unmountrequested

Raised when unmounting for the file system with the fileSystemId identifier is requested.

addListener

navigator.fileSystemProvider.addListener("unmountrequested", function handler)

Properties
UnmountRequestedOptions	options		The requested unmount options.
function	successCallback		Success callback for unmountrequested event.
	Return value
	void
function	errorCallback		Error callback for unmountrequested event.
	Parameter
	FileSystemProviderError	errorCode
	Return value
	void

abortrequsted

Raised when aborting an operation with operationRequestId is requested. The operation executed with operationRequestId must be immediately stopped and successCallback of this abort request executed. If aborting fails, then errorCallback must be called. Note, that callbacks of the aborted operation must not be called, as they will be ignored. Despite calling errorCallback, the request may be forcibly aborted.

addListener

navigator.fileSystemProvider.addListener("abortrequested", function handler)

Properties
AbortRequestedOptions	options		The requested abort options.
function	successCallback		Success callback for abortrequested event.
	Return value
	void
function	errorCallback		Error callback for abortrequested event.
	Parameter
	FileSystemProviderError	errorCode
	Return value
	void

getmetadatarequested

Raised when metadata of a file or a directory at entryPath is requested. The metadata must be returned with the successCallback call. In case of an error, errorCallback must be called.

addListener

navigator.fileSystemProvider.addListener("getmetadatarequested", function handler)

Properties
GetMetadataRequestedOptions	options		The requested getmetadata options.
function	successCallback		Success callback for getmetadatarequested event.
	Parameter
	EntryMetadata	metadata
	Return value
	void
function	errorCallback		Error callback for getmetadatarequested event.
	Parameter
	FileSystemProviderError	errorCode
	Return value
	void

readdirectoryrequested

Raised when contents of a directory at directoryPath are requested. The results must be returned in chunks by calling the successCallback several times. In case of an error, errorCallback must be called.

addListener

navigator.fileSystemProvider.addListener("readdirectorytrequested", function handler)

Properties
ReadDirectoryRequestedOptions	options		The requested readdirectory options.
function	successCallback		Success callback for readdirectoryrequested event. If more entries will be returned, then hasMore must be true, and it has to be called again with additional entries. If no more entries are available, then hasMore must be set to false.
	Parameter
	Sequence<EntryMetadata>	entries
	boolean	hasMore
	Return value
	void
function	errorCallback		Error callback for readdirectoryrequested event.
	Parameter
	FileSystemProviderError	errorCode
	Return value
	void

openfilerequested

Raised when opening a file at filePath is requested. If the file does not exist, then the operation must fail.

addListener

navigator.fileSystemProvider.addListener("openfilerequested", function handler)

Properties
OpenFileRequestedOptions	options		The requested openfile options.
function	successCallback		Success callback for openfilerequested.
	Return value
	void
function	errorCallback		Error callback for openfilerequested.
	Parameter
	FileSystemProviderError	errorCode
	Return value
	void

closefilerequested

Raised when opening a file previously opened with openRequestId is requested to be closed.

addListener

navigator.fileSystemProvider.addListener("closefilerequested", function handler)

Properties
CloseFileRequestedOptions	options		The requested closefile options.
function	successCallback		Success callback for closefilerequested event.
	Return value
	void
function	errorCallback		Error callback for closefilerequested event.
	Parameter
	FileSystemProviderError	errorCode
	Return value
	void

readfilerequested

Raised when reading contents of a file opened previously with openRequestId is requested. The results must be returned in chunks by calling successCallback several times. In case of an error, errorCallback must be called.

addListener

navigator.fileSystemProvider.addListener("readfilerequested", function handler)

Properties
ReadFileRequestedOptions	options		The requested readfile options.
function	successCallback		Success callback for the readfilerequested event. If more data will be returned, then hasMore must be true, and it has to be called again with additional entries. If no more data is available, then hasMore must be set to false.
	Parameter
	ArrayBuffer	data
	boolean	hasMore
	Return value
	void
function	errorCallback		Error callback for the readfilerequested event.
	Parameter
	FileSystemProviderError	errorCode
	Return value
	void

createdirectoryrequested

Raised when creating a directory is requested. The operation must fail with the EXISTS error if the target directory already exists. If recursive is true, then all of the missing directories on the directory path must be created.

addListener

navigator.fileSystemProvider.addListener("createdirectoryrequested", function handler)

Properties
CreateDirectoryRequestedOptions	options		The requested createdirectory options.
function	successCallback		Success callback for createdirectoryrequested event.
	Return value
	void
function	errorCallback		Error callback for createdirectoryrequested event.
	Parameter
	FileSystemProviderError	errorCode
	Return value
	void

deleteentryrequested

Raised when deleting an entry is requested. If recursive is true, and the entry is a directory, then all of the entries inside must be recursively deleted as well.

addListener

navigator.fileSystemProvider.addListener("deleteentryrequested", function handler)

Properties
DeleteEntryRequestedOptions	options		The requested deleteentry options.
function	successCallback		Success callback for deleteentryrequested event.
	Return value
	void
function	errorCallback		Error callback for deleteentryrequested event.
	Parameter
	FileSystemProviderError	errorCode
	Return value
	void

copyentryrequested

Raised when copying an entry (recursively if a directory) is requested. If an error occurs, then errorCallback must be called.

addListener

navigator.fileSystemProvider.addListener("copyentryrequested", function handler)

Properties
CopyEntryRequestedOptions	options		The requested copyentry options.
function	successCallback		Success callback for copyentryrequested event.
	Return value
	void
function	errorCallback		Error callback for copyentryrequested event.
	Parameter
	FileSystemProviderError	errorCode
	Return value
	void

moveentryrequested

Raised when moving an entry (recursively if a directory) is requested. If an error occurs, then errorCallback must be called.

addListener

navigator.fileSystemProvider.addListener("moveentryrequested", function handler)

Properties
MoveEntryRequestedOptions	options		The requested moveentry options.
function	successCallback		Success callback for moveentryrequested event.
	Return value
	void
function	errorCallback		Error callback for moveentryrequested event.
	Parameter
	FileSystemProviderError	errorCode
	Return value
	void

createfilerequested

Raised when creating a file is requested. If the file already exists, then errorCallback must be called with the "EXISTS" error code.

addListener

navigator.fileSystemProvider.addListener("createfilerequested", function handler)

Properties
CreateFileRequestedOptions	options		The requested createfile options.
function	successCallback		Success callback for createfilerequested event.
	Return value
	void
function	errorCallback		Error callback for createfilerequested event.
	Parameter
	FileSystemProviderError	errorCode
	Return value
	void

writefilerequested

Raised when writing contents to a file opened previously with openRequestId is requested.

addListener

navigator.fileSystemProvider.addListener("writefilerequested", function handler)

Properties
WriteFileRequestedOptions	options		The requested writefile options.
function	successCallback		Success callback for writefilerequested event.
	Return value
	void
function	errorCallback		Error callback for writefilerequested event.
	Parameter
	FileSystemProviderError	errorCode
	Return value
	void

Cloud Storage Addons

Cloud Storage Addons are in charge of the communication with the target cloud. It should provides at least

• User authentication and authorization mechanism
• Cloud access operations through the public APIs from cloud providers

Moreover, Cloud Storage Addons might provide more features, such as

• Data catch
• Data Synchronization
• Account management

Usage Flow

Mounting a Cloud on FirefoxOS

Mount Flow

To mount a cloud on FirefoxOS, FileSystemProvider API providers methods and events for Cloud Storage Addons.
The picture shows the flow that Cloud Storage Addon mounts a cloud "MyCloud" on FirefoxOS through FileSystemProvider API.

1. Cloud Storage addon registers the event listener on FileSystemProviderEvents.
2. Call FileSystemProvider.mount() method with parameters to request Volume System to create a fake volume.

Following is an example code to mount a cloud "MyCloud"

 // Code in Cloud Storage Addons
 var fileSystemProvider = navigator.fileSystemProvider;
 // Event handler definition
 function GetMetadataHandler(event) {
   var path = event.options.entryPath;
   // get metadata through cloud APIs with entry path
   if (metadata) {
      event.successCallback(metadata);
   } else {
      event.errorCallback('NotFound');
   }
 }
   ...
 // Register event listener on FileSystemProviderEvents
 fileSystemProvider.addEventListener("getmetadatarequested", GetMetadataHandler);
 fileSystemProvider.addEventListener("openfilerequested", OpenFileHandler);
 fileSystemProvider.addEventListener("closefilerequested", CloseFileHandler);
 fileSystemProvider.addEventListener("readdirectoryrequested", ReadDirectoryHandler);
 fileSystemProvider.addEventListener("readfilerequested", ReadFileHandler);
   ...
 // Call mount method to create fake volume "MyCloud" in FirefoxOS
 fileSystemProvider.mount({fileSystemId: 'MyCloud',
                           displayName: 'MyCloud',
                           writable: true,
                           openedFilesLimit: 256})
                           .then(
                           function() { dump('mount successfully\n');}
                           function(aError) { dump('mount failed ['+aError+']\n');}
                           );

Accessing Cloud through the Framework

Read File Flow

To access a mounted cloud, app can reuse DeviceStorage and File APIs, just like they access local storage.
The picture shows the flow how Gallery app read the "test.jpg" on cloud "MyCloud".

1. Gallery app asks "MyCloud/test.jpg" data through the File API.
2. Volume System get the request and send FileSystemProvderReadFileEvent to ask corresponding Cloud Storage Addons to get the data from cloud.
3. Cloud Storage Addon receives the FileSystemProviderReadFileEvent and call ReadFileEventListener to get "test.jpg" data from cloud through the cloud access API.
4. Once read success or fail, call the event.successCallback with data or event.errorCallback with error code.
5. Volume System receives the callback data and send it back to app through the File API.

Following is an example to access cloud 'MyCloud' through DeviceStorage and File APIs.

 // Get file through DeviceStorage API
 var myStorage = navigator.getDeviceStorage('MyCloud');
 if (myStorage) {
   // Request to send GetMetadataEvent to get the information of "myFile.txt"
   var request = myStorage.get('myFile.txt');
   request.onsuccess = function() {
     var file = this.result;
     dump(file.name+'\n');
     var fileReader = new FileReader();
     fileRead.onloadend = function() {dump('load finished\n');}
     // Request to send OpenFileEvent, ReadFileEvents and CloseFileEvent
     fileReader.readAsArrayBuffer(file);
   }
 }

Mozlando

Presentation slide: https://goo.gl/Yd5lqT

Team

• Frontend Engineer: Sean Lee
• Backend Engineer: Kershaw Chang Eden Chuang
• EPM: Aaron Wu