User:Clouserw/APKFactory

C&C APK Factory

We're building an APK Factory! In short, this is a black box which will accept a manifest URL as input and return a .apk file. This project includes some UI changes on the Marketplace but the vast majority of the project is backend and APIs.

The primary use case is:

• Developer uploads new app to the Marketplace
• Marketplace validates the app
• Marketplace puts app in queue for reviewers
• Marketplace dispatches information to the APK Factory and requests a reviewer copy of the APK (that means signed with a debug key)
• APK Factory returns the apk, this could take several seconds (7 - 10 seconds)
• Marketplace can continue to retrieve the App APK, they will be cached in the APK Factory (hundreds of milliseconds for 2nd request)

 * It's safe to do a "fire and forget" which would prime the cache

• When app is approved, Marketplace dispatches a second request to APK factory asking for a production copy of the APK (signed with a real per-app key)
• APK Factory returns the APK
• Marketplace continues to fetch the APK from this URL, as the APKs will be cached locally in the APK Factory service.
• Marketplace passes through the APK to the end users when requested

Secondary use case (public factory):

• Developer sends URL directly to the APK Factory
• APK Factory returns a reviewer copy of the APK (signed with a debug key) and a production copy of the APK (signed with a real per-app key)
  • (this is a secondary function. Some product decisions need to happen around whether this is a requirement)

Changes needed

      +------------------+       +---------------------+         +------------------+
      |                  |       |                     |         |                  |
      |   Marketplace    |+----->|     APK Factory     |+------> |    APK Signer    |
      |    (modify)      |<-----+|      (create)       |<------+ |     (create)     |
      |                  |       |                     |         |                  |
      |                  |       |                     |         |                  |
      +------------------+       +---------------------+         +------------------+

Marketplace

• [uiwanted] Adjust developer flow to opt-out of automatic APK generation
• Adjust developer flow to send manifest (or mini-manifest) URL to APK Factory
• Adjust developer tools to add APK download link to versions page of the app page
• [uiwanted] Adjust developer tools to accept a user specified signing key (associated with app)
• Adjust reviewer tools to add APK download link to review detail page
• Adjust mini-manifests to include APK download URL (should automatically make all install buttons work)

Download links are the same as APK generation links.

APK Factory

• Requires Java Development Kit (JDK) not just an environment (JRE). Requires new hardware?
• API to accept manifest as input
  • Differentiate between hosted & packaged apps
  • (...) the actual packaging here. Need to duplicate/enhance prototype for functionality and scale
• (v2) API to accept signing key if specified
• Ability to sign using an android debug key for all non-reviewed manifests.
• Ability to sign using production app keys if signing for final packages
• (v2) API to send and receive keys from the APK Signer
• GC to clean up old packages on disk
• Track apk-factory-library version as a dependency for calculating if a "newer version" of an app is available
• Provide a "updates available" service which accepts a POST with a list of manifests and returns the list of Apps which have a newer version available

APK Signer

• Requires new hardware
• Based heavily on solitude
• A secure storage area for production signing keys
• No outside network traffic (see above diagram)

User Stories

Use case: install

As a user, I can tap install on a marketplace page and an APK is downloaded to my device so that my phone's native package installer can install the app.

We will be modifying the implementation of the mozApps.install() and mozApps.installPackage() APIs on Android. On webapp install, Fennec will download an APK which is generated from the manifest URL which is passed to the apk-factory.

In this manner, we are able to avoid changing any marketplace code, and support third-party webapp marketplaces.[at the very least, with this method we avoid the marketplace trying to detect whether the phone is android or not -BillW]

Implementation of the apk-factory

It is intended that apk-factory use Android's standard toolchain.

The apk-factory accepts a URL to a JSON document, that could be a webapp manifest or mini-manifest. It should detect the difference between a packaged app and hosted app and act accordingly.

The apk-factory constructs an Android project from a template and data in the webapp manifest, including icons, string translations, permissions, and web activities.

Both webapps and Android have very similar concepts. There is some impedance mismatch, though this is not particularly relevant for the scope of this project.

The exception to this is around APK signing. Android apps are expected to be signed by the developer, regardless of whether they seek any special permissions. The same signature is used to sign successive versions of the same APK, otherwise the update will not be allowed to continue.

For packaged webapps, the package.zip and mini-manifest files will be included in the APK.

Once the apk-factory has created an Android project, it can build it with ant, javac and the rest of the Android toolchain.

By design, the Android project's template contains no Java code. All code is contained in a library-project. Both, especially the library project, are being actively developed in parallel to the apk-factory.

It is desirable that the project template and library code be taken live from a code repository so that changes are easy to incorporate in to the workflow.

Use case: update

As a user, I can have my phone check for updates and have my phone download them so that I may keep my webapps safely up-to-date.

The synthesized APK is relatively lightweight. For hosted apps, Gecko can manage updating appcache and resources generated at runtime. Updating anything used to build the APK should trigger a rebuild of the APK itself. In practice, this means the APK should be rebuilt anytime the webapp manifest changes. From a computation point of view, the detection of change in the manifest and the rebuilding of the APK would be sensibly done on the server.

Fennec will periodically check with the APK Factory service to see if any apps can be updated.

For packaged apps, any change of the app should trigger a rebuild, however, we are working with the assumption that a release of the app is needed, and a change in the mini-manifest will be triggered.

App freshness is a function of:

• the manifest
• any packaged component
• changes to the Android Library code

Android apps have a hard requirement that the versionCode, a non user facing integer representation of the version string, be positively increasing.

The same android signing key should be used to sign the updated APK as was used to sign the previous version of the APK.

Use case: developer's own keys

As a developer, I can use my own key to sign my Android APK, so I can use the key elsewhere.

Changing keys for an already published APK is impossible.

For a large proportion of developers, a randomly generated APK signing key will be sufficient.

However, a few will want to be able to use their own. During app submission, if the app is to be on Android, they should have the opportunity to upload their own key.

Use case: developer opt-out

As a Developer, I can chose not to have my app auto-generated as an Android APK

The developer may already have an Android app or may not wish to have the app deployed on Android of various reasons.

Use case: pre-submission development

As a developer I would like to be able to test my APK without submitting it to the marketplace.

It is assumed that the Android toolchain is rather cumbersome and not interesting enough for the webapp developer to actually set it up successfully.

The apk-factory should accept a POST of the webapp manifest and resources needed for the webapp, and return an APK. A command line or option in the app manager will be used to send the POST.

The APK should not be publishable on an APK app-store, e.g. signed with a debug key, icon marked with "IN TEST".

This use of the apk-factory should be invoked by the developer from the command line or App Manager.

In coming releases, we would like the developer to be able to construct an APK without calling out to a server, either from the command line or App Manager.

Use case: post-submission review

• As a reviewer, I would like to follow the same review process as I do for the normal app.
• As a reviewer I can run the APK on my Android phone.
• As a developer I can run the same APK as the reviewer.

The APK should not be publishable on an APK app-store, e.g. signed with a debug key, icon marked with "IN REVIEW".

Use case: post-review acceptance

As a developer, I can download the APK that will be installed on user's devices, so I can distribute it myself.

Once the webapp is ready for publishing, the developer should receive (as part of an existing email?), a link to the APK.