On short notice I received an assignment to put together a quick, functional prototype for an application. The prototype only needed to demonstrate that some bit of functionality was possible. I wanted to be able to update some of the assets that were used by the application without doing a redeploy. Part of the reason for this is that the application was going to be demonstrated by someone in another city, and I wouldn’t be able to do any last minute updates myself through a redeploy. I managed to put together a system that allowed me to make content updates on a website that the demonstration device could download when the application was run. I’m sharing that solution here.

A few things to keep in mind though. Since this was for a prototype that had to be put together rapidly, there are some implementation details that I would probably not do in a real application; such as performing the downloads using a thread instead of a coroutine.

To make this work, the application by design loads assets from the file system. The assets that it uses are packaged with the application. On first run, the app will pull those assets from its package and write them to the file system. The application that I am demonstrating here loads a list of images and captions for those images and displays them on the screen. For the asset collection that is baked into the application, I only have one image and one caption.

To demonstrate, I’ve created a new sample application (as I can’t share the prototype that I made) that list the images that it has on the screen. For the initial, this is a list of a single image. If you would like to see the complete code, you can clone it from https://github.com/j2inet/AndroidContentDownloadSample.git. When the application is run, it downloads an alternative content set. The images I used were taken in the High Museum of Art in Atlanta.

There are a few folder locations that I’ll use for managing files. A complete content set will be present at the root of the application’s file. There will be a subfolder that will hold partially downloaded files when they are sourced from the internet. Once a file is completely downloaded, it will be moved to a different temporary folder. If the application is disrupted while downloading, anything that is in the partial download folder is considered incomplete and will be deleted. A file that is present in the completed folder is assumed to have all of it’s data and will not be downloaded again the next time the application starts. Once all files within a content set are downloaded, they are moved to the root of the application files system. This is the function that is used to ensure that the necessary folders are present.

companion object {
    public val TAG = "ContentUpdater"
    val STAGING_FOLDER = "staging"
    val COMPLETE_FOLDER = "completed"
}

fun ensureFoldersExists() {
   val applicationFilesFolder = context.filesDir.absoluteFile;
    val stagingFolderPath = Paths.get(applicationFilesFolder.absolutePath, STAGING_FOLDER)
    val stagingFolder:File =  stagingFolderPath.toFile()
    if(!stagingFolder.exists()) {
        stagingFolder.mkdir()
    }
    val downloadSetPath = Paths.get(applicationFilesFolder.absolutePath, COMPLETE_FOLDER)
    val completedFolder:File = downloadSetPath.toFile()
    if(!completedFolder.exists()) {
        completedFolder.mkdir()
    }
}

To package the assets, I added an “assets” folder to my Android project. By default the Android Studio project does not have an assets folder. To add one, within Android Studio, select File -> New -> Folder -> Assets Folder. Android Studio will place the Assets folder in the right place. Place the files that you want to be able to update within this folder in your project. Most of the files that I placed in this folder are specific to the application that I was working on and can largely be viewed as arbitrary. The one file that absolutely must be present for this system to work is an additional file I made named updates.json. The file 3 vital categories of data.

  "version": 0,
  "updateURL": "https://j2i.net/apps/downloader/updates.json",
  "assets": [
    {
      "url": "",
      "name": "assetsManifest.json"
    },
    {
      "url": "",
      "name": "image0.png"
    },
    {
      "url": "",
      "name": "caption0.txt"
    }
  ]
}

The most important category of content are the names of the files that make up the content. The code is going to use these names to know what assets to pull out of the application package. The other two important items are the asset version number and the update URL for grabbing updates. We will look at those items in a moment.

We want the code to check the file system to see if updates.json has already been extracted and written. If it is not present, then the code will copy it out of the package and place it in the file system. If it is already present, then it will not be overwritten. The file is never overwritten during this check because the files that is there on the filesystem could be a more recent version than what was packaged with the application. After the application has ensured that this file is present, it reads through the properties for each asset. Each asset is composed of a url (that indicates where the resource can be found) and a name (which will be used for the file name when the file is extracted). In the above, all of the files have an empty string for the URL. If the URL is not blank, then the file is assumed to be part of the application package. The routine for pulling out an asset and writing it to the file is based on something that is fairly routine. It accepts the name of the file and a flag indicating whether it should be overwritten if the file is already present. You might recall seeing a form of this function in the previous entry that I made on this blog.

private fun assetFilePath(context: Context, assetName: String, overwrite:Boolean = false): String? {
    val file = File(context.filesDir, assetName)
    if (!overwrite && file.exists() && file.length() > 0) {
        return file.absolutePath
    }
    try {
        context.assets.open(assetName).use { inputStream ->
            FileOutputStream(file).use { os ->
                val buffer = ByteArray(4 * 1024)
                var read: Int
                while (inputStream.read(buffer).also { read = it } != -1) {
                    os.write(buffer, 0, read)
                }
                os.flush()
            }
            return file.absolutePath
        }
    } catch (e: IOException) {
        Log.e(TAG, "Error process asset $assetName to file path")
    }
    return null
}

To ensure that the assetFilePath function is called on each file that must be pulled from the application, I’ve written the function extractAssetsFromApplication. This function is generously commented. I’ll let the comments explain what the function does.

fun extractAssetsFromApplication(minVersion:Int, overwrite:Boolean = false) {
    //ensure that updates.json exists in the file system
    val updateFileName = "updates.json"
    val file = File(context.filesDir, updateFileName)
    val updatesFilePath = assetFilePath(this.context,updateFileName, overwrite);
    //Load the contents of updates.json
    val updateFile = File(updatesFilePath).inputStream();
    val contents = updateFile.bufferedReader().readText()
    //Use a JSONObject to parse out the file's data
    val updateObject = JSONObject(contents)
    //IF the version in the file is below some version, assume that it is
    //an old version left over from a previous version of the application.
    //restart the extraction process with the overwrite flag set
    val assetVersion = updateObject.getInt("version")
    if(assetVersion < minVersion) {
        extractAssetsFromApplication(minVersion,true)
        return
    }
    //Let's start processing the individual asset items.
    val assetList = updateObject.get("assets") as JSONArray
    for(i in 0 until assetList.length()) {
        val currentObject = assetList.get(i) as JSONObject
        val currentFileName = currentObject.getString("name")
        val uri:String? =  currentObject.getString("url")

        if(uri.isNullOrEmpty() || uri == "null") {
            //There is no URL associated with the file. It must be within
            // the application package. Copy it from the application package
            //and write it to the file system
            assetFilePath(this.context, currentFileName, overwrite)
        } else {
            //If there is a URL associated with the asset, then add it to the download
            //queue. It will be downloaded later.
            val downloadRequest = ResourceDownloadRequest(currentFileName, URL(uri))
            downloadQueue.add(downloadRequest)
        }
    }
}

When the application first starts, we may need to address files that are lingering in the staging or completed folder. The completed folder contains files that have successfully been downloaded. But there may be other files for the file set that have yet to be downloaded. If the file set is complete there will be a file named “isComplete” in the folder. If that file is found, then the contents of the folder are copied to the root of the application’s file system and are deleted from the completed folder. Any files that are in the staging folder when the application starts are assumed to be incomplete. They are found and deleted.

fun applyCompleteDownloadSet() {
    val isCompleteFile = File(context.filesDir, COMPLETE_FOLDER + "/isComplete")
    if(!isCompleteFile.exists()) {
        return;
    }
    var downloadFolder = File(context.filesDir, COMPLETE_FOLDER)
    val fileListToMove = downloadFolder.listFiles()
    for(f:File in fileListToMove) {
        val destination = File(context.filesDir, f.name)
        f.copyTo(destination, true)
        f.delete()
    }
}


fun clearPartialDownload() {
    val stagingFolder = File(context.filesDir, STAGING_FOLDER)
    //If we have a staging folder, we need to check it's contents and delete them
    if(stagingFolder.exists())
    {
        val fileList = stagingFolder.listFiles()
        for(f in fileList) {
            f.delete()
        }
    }
}

To check for updates online, the application loads updates.json and reads the version number and the updateURL. The file at the updateURL is another instance of updates.json. Though if it is an update then it will contain a different set of content. The version in the online version of this file is compared to the local version of the file. If the online version has a greater number then it is downloaded. Otherwise no further work is done on the file. Any version of updates.json must have the url properties populated for the assets. If this value is missing, then the file is not valid. The download URLs and intended file names are collected (as the source URL might not contain the file name in it at all).

fun checkForUpdates() {
    thread {
        val updateFile = File(context.filesDir, "updates.json")
        val sourceUpdateText = updateFile.bufferedReader().readText()
        val updateStructure = JSONObject(sourceUpdateText)
        val currentVersion = updateStructure.getInt("version")
        val updateURL = URL(updateStructure.getString("updateURL"))
        val newUpdateText =
            updateURL.openConnection().getInputStream().bufferedReader().readText()
        val newUpdateStructure = JSONObject(newUpdateText)
        val newVersion = newUpdateStructure.getInt("version")
        if (newVersion > currentVersion) {
            val assetsList = newUpdateStructure.getJSONArray("assets")
            for (i: Int in 0 until assetsList.length()) {
                val current = assetsList.get(i) as JSONObject
                val dlRequest = ResourceDownloadRequest(
                    current.getString("name"),
                    URL(current.getString("url"))
                )
                downloadQueue.add(dlRequest)
            }
            downloadFiles();
        }
    }
}

The downloadFiles function starts to get into the real work of what the component does. For any file, this function will make up to three attempts to download the file before it gives up on the file. The file contents are downloaded through the URL object. The URL object provides an outputStream to the resource identified through the URL. I’m arbitrarily downloading the file in 8 kilobyte chunks (8192 bytes). As mentioned before, the chunks are written to a temporary folder. Once a file is completed, it gets moved.

    @WorkerThread
    fun downloadFiles() {
        val MAX_RETRY_COUNT = 3
        val failedQueue = LinkedList<ResourceDownloadRequest>()
        var retryCount = 0;
        while(retryCount<MAX_RETRY_COUNT && downloadQueue.count()>0) {

            while (downloadQueue.count()>0) {
                val current = downloadQueue.pop()
                try {
                    downloadFile(current)
                } catch (exc: IOException) {
                    failedQueue.add(current)
                }
            }
            downloadQueue.clear()
            downloadQueue.addAll(failedQueue)
            ++retryCount;
        }
        if(downloadQueue.count()>0) {
            //we've failed to download a complete set.
        } else {
            //A complete set was downloaded
            //I'll mark a set as complete by creating a file. The presence of this file
            //markets a complete set. An absence would indicate a failure.
            val isCompleteFile = File(context.filesDir, COMPLETE_FOLDER + "/isComplete")
            isCompleteFile.createNewFile()
        }
    }

    fun downloadFile(d:ResourceDownloadRequest) {
        downloadFile(d.name, d.source)
    }

    fun downloadFile(name:String, source: URL) {
        val DOWNLOAD_BUFFER_SIZE = 8192
        val urlConnection:URLConnection = source.openConnection()
        urlConnection.connect();
        val length:Int = urlConnection.contentLength

        val inputStream:InputStream = BufferedInputStream(source.openStream(), DOWNLOAD_BUFFER_SIZE)
        val targetFile = File(context.filesDir, STAGING_FOLDER + "/"+ name)
        targetFile.createNewFile();
        val outputStream = targetFile.outputStream()
        val buffer = ByteArray(DOWNLOAD_BUFFER_SIZE)
        var bytesRead = 0
        var totalBytesRead = 0;
        var percentageComplete = 0.0f
        do {
            bytesRead = inputStream.read(buffer,0,DOWNLOAD_BUFFER_SIZE)
            if(bytesRead>-1) {
                totalBytesRead += bytesRead
                percentageComplete = 100F * totalBytesRead.toFloat() / length.toFloat()
                outputStream.write(buffer, 0, bytesRead)
            }
        } while(bytesRead > -1)
        outputStream.close()
        inputStream.close()
        val destinationFile = File(context.filesDir, COMPLETE_FOLDER + "/"+ name)
        targetFile.copyTo(destinationFile, true, DEFAULT_BUFFER_SIZE)
        targetFile.delete()
    }

That covers all of the more complex functionality in the code. How is it used? Usage starts with the constructor. When the ContentUpdater is extantiated, it will create the folders (if they do not already exists), extract the content from the application (if there is no content present) and clear the partial download folder. It does not automatically apply the new downloaded content to the application.

class ContentUpdater {

    companion object {
        public val TAG = "ContentUpdater"
        val STAGING_FOLDER = "staging"
        val COMPLETE_FOLDER = "completed"
    }
    val context:Context
    val downloadQueue = LinkedList<ResourceDownloadRequest>()

    constructor(context: Context, minVersion:Int) {
        this.context = context

        ensureFoldersExists()
        extractAssetsFromApplication(minVersion);
        this.clearPartialDownload()
    }
}

In theory, I could have the routine do this as soon as a complete download set is preset. But changing the content in the middle of a session within an application could cause problems. The application using the component could ask the component to apply downloaded content at any time through by calling applyCompleteDownloadSet(). I have the application doing this in the onCreate event of the main activity. That way the most recent content is applied before the reset of the application begins to get initialized.

There are a lot of scenarios that I might consider if I ever use something like this in a production application. This includes possibly notifying the user of the progress of the download, giving the user the option to load the new content once it is complete, and some other scenarios on handling having multiple versions of the application in user’s hands at once. I would also move the download code to either a coroutine (instead of a thread) or possibly a service (for larger downloads) and consider limiting the downloads to WiFi. I wouldn’t suggest the code that I’ve presented here to be copied directly into a production application, But it can be a good starting point if you are trying to figure out your own solution.

If you needed to include additional information with your Android application that isn’t already supported by Android Studio and the various functionality natively, one solution is to place the content in the project’s Assets folder. By default, a new projects do not have an Assets folder. You can easily add one through the menu sequence File -> New -> Folder -> Assets Folder. Within this folder. Assets that you add to this folder will now be packaged with you app. They will also be compressed.

You have the option of not compressing the files. You may want to do this if the files are already in a compressed format and thus are not significantly reduced in size by additional compression. If you want a file type exempted for compression, you can direct the compiler to not compress it by making an addition to the build.gradle for the module. If I wanted txt files to be exempted from compression, I would make the following addition.

android {
    aaptOptions {
        noCompress 'txt'
    }
}

Uncompressed files are easy to read. If I placed a files named “readMe.txt” in my assets folder, I can get an InputStream for the file with the following code line.

val myInputStream = context.assets.open("readMe.txt")

You may want to write the files out to the files system for faster access. The following function, when given the name of an asset, will return the absolute path to the location of the file derived from the asset. It first checks to see if the asset has already be extracted to a file. If it has not, then it will take care of extracting it. Accessing the assets this way has an advantage. After an application has been deployed, your application could at runtime check a web location for updated versions of the assets and write them to the file system. Without any further changes in logic, the application could just attempt to read the asset as normal and it will receive the updated version.

    fun assetFilePath(context: Context, assetName: String): String? {
        val file = File(context.filesDir, assetName)
        if (file.exists() && file.length() > 0) {
            return file.absolutePath
        }
        try {
            context.assets.open(assetName).use { inputStream ->
                FileOutputStream(file).use { os ->
                    val buffer = ByteArray(4 * 1024)
                    var read: Int
                    while (inputStream.read(buffer).also { read = it } != -1) {
                        os.write(buffer, 0, read)
                    }
                    os.flush()
                }
                return file.absolutePath
            }
        } catch (e: IOException) {
            Log.e(TAG, "Error process asset $assetName to file path")
        }
        return null
    }

In my next entry, I’ll be using this function to create an application that can also update its content from online content.