Recompiling the V8 JavaScript Engine on Windows

I decided to compile the Google V8 JavaScript engine. Why? So that I could include it in another program. Google doesn’t distribute the binaries for V8, but they do make the source code available. Compiling it is, in my opinion, a bit complex. This isn’t a criticism. There are a lot of options for how V8 can be built. Rather than making available the permutations of these options for each version of V8, one could just set options themselves and build it for their platform of interest.

But Isn’t There Already Documentation on How to Do This?

There does exists documentation from Google on compiling Chrome. But there are variations from those instructions and what must actually be done. I found myself searching the Internet for a number of other issues that I encountered and made notes on what I had to do to get around compilation problems. The documentation comes close to what’s needed, but isn’t without error and deviation.

Setting Up Your Environment

Before touching the v8 source code, ensure that you have installed Microsoft Visual Studio. I am using Microsoft Visual Studio 2022 Community Edition. There are some additional components that must be installed. In an attempt to make this setup process as scriptable as possible, I’ve have a batch file that will have the Visual Studio Installer add the necessary components. If a component is already installed, no action is taken. Though the Google V8 instructions also offer a command to type to accomplish the same thing, this is where I encountered my first variation from their instructions. Their instructions assume that the name of the Visual Studio Installer command to be setup.exe (it probably was on a previous version of Visual Studio) where my installer is named vs_installer.exe. There were also additional parameters that I had to pass, possibly because I have more than one version of Visual Studio installed (Community Edition 2022, Preview Community Edition 2022, and a 2019 version).

pushd C:\Program Files (x86)\Microsoft Visual Studio\Installer\

vs_installer.exe --productid Microsoft.VisualStudio.Product.Community --ChannelId VisualStudio.17.Release --add Microsoft.VisualStudio.Workload.NativeDesktop  --add Microsoft.VisualStudio.Component.VC.ATLMFC  --add Microsoft.VisualStudio.Component.VC.Tools.ARM64 --add Microsoft.VisualStudio.Component.VC.MFC.ARM64 --add Microsoft.VisualStudio.Component.Windows10SDK.20348 --includeRecommended

popd

You may need to make adjustments if your installer is located in a different path.

While those components are installing, let’s get the code downloaded and put int place. I did the download and unpacking from powershell. All of the commands that follow were stored in a power shell script. Scripting the process makes it more repeatable and is easier to document (since the scripts are also a record of what was done). You do not have to use the same file paths that I do. But if you change them, you will need to make adjustments to the instructions when one of these paths is used.

I generally avoid placing folders directly in the root. The one exception to that being a folder I make called c:\shares. There’s a structure that I conform to when placing this folder on Windows machines. For this structure, Google’s code will be placed in subdirectories of c:\shares\projects\google. In the following script you’ll see that path used.

$depot_tools_source = "https://storage.googleapis.com/chrome-infra/depot_tools.zip"
$depot_tools_download_folder= "C:\shares\projects\google\temp\"
$depot_tools_download_path = $depot_tools_download_folder + "depot_tools.zip"
$depot_tools_path = "c:\shares\projects\google\depot_tools\"
$chromium_checkout_path = "c:\shares\projects\google\chromium"
$v8_checkout_path = "c:\shares\projects\google\"

mkdir $depot_tools_download_folder
mkdir $depot_tools_path
mkdir $chromium_checkout_path
mkdir $v8_checkout_path

pushd "C:\Program Files (x86)\Microsoft Visual Studio\Installer\"
.\vs_installer.exe install --productID Microsoft.VisualStudio.Product.Community --ChannelId VisualStudio.17.Release --add Microsoft.VisualStudio.Workload.NativeDesktop  --add Microsoft.VisualStudio.Component.VC.ATLMFC  --add Microsoft.VisualStudio.Component.VC.Tools.ARM64 --add Microsoft.VisualStudio.Component.VC.MFC.ARM64 --add Microsoft.VisualStudio.Component.Windows10SDK.20348 --includeRecommended
popd

Invoke-WebRequest -Uri $depot_tools_source -OutFile $depot_tools_download_path
Expand-Archive -LiteralPath $depot_tools_download_path -DestinationPath $depot_tools_path

After this script completes running, Visual Studio should have the necessary components and the V8/Chrome development tools are downloaded and in place.

There are some environment variables on which the build process is dependent. These variables could be set within batch files, could be set to be part of the environment for an instance of the command terminal, or set at the system level. I chose to set them at the system level. This was not my first approach. I set them at more local levels initially. But several times when I needed to open a new command terminal, I forgot to apply them, and just found it easier to set them globally.

ENVIRONMENT VARIABLEVALUE
DEPOT_TOOLS_WIN_TOOLCHAIN0
vs2022_installC:\Program Files\Microsoft Visual Studio\2022\Community
PATHc:\shares\projects\google\depot_tools\;%PATH%
Environment Variables that must be set

From here on, we will be using the command prompt, and not PowerShell. This is because some of the commands that are part of Google’s tools are batch files that only run properly in the command prompt.

From the command terminal, run the command gclient. This will initialize the Google Tools. Next, navigate to the folder in which you want the v8 code to download. For me, this will be c:\shares\projects\google. The download process will automatically make a subfolder named v8. Run the following command.

fetch v8 --no-history

This command can take a while to complete. After it completes you will have a new directory namd v8 that contains the source code. Navigate to that directory.

cd v8

The online documentation that I see from Google for v8 is for verion 9. I wanted to compiled version 12.0.174.

git checkout 12.0.174

Today I am trying to only rebuild v8 for Windows. Eventually I’ll rebuild it for ARM64 also. Run the following commands. It will make the build directories and configurations for different targets.

python3 .\tools\dev\v8gen.py x64.release
python3 .\tools\dev\v8gen.py x64.debug
python3 .\tools\dev\v8gen.py arm64.release
python3 .\tools\dev\v8gen.py arm64.debug

The build arguments for each environment are in a file named args.gn. Let’s update the configuration for the x64 debug build. To open the build configuration, type the following.

notepad out.gn\x64.debug\args.gn

This will open the configuration in notepad. Replace the contents with the following.

is_debug = true
target_cpu = "x64"
v8_enable_backtrace = true
v8_enable_slow_dchecks = true
v8_optimized_debug = false
v8_monolithic = true
v8_use_external_startup_data = false
is_component_build = false
is_clang = false

Chances are the only difference between the above and the initial version of the file are from the line v8_monolithic onwards. Save the file. You are ready to start your build. To kick off the build, use the following command.

ninja -C out.gn\x64.debug v8_monolith

This will also take a while to run, but this will fail. There is a third party component that will fail concerning a line in a file named fmtable.cpp. You’ll have to alter a function to fix the problem. Open the file in the path .\v8\third_party\icu\source\i18n\fmtable.cpp. Around line 59, you will find the following code.

static inline UBool objectEquals(const UObject* a, const UObject* b) {
     // LATER: return *a == *b
     return *((const Measure*)a) == ((const Measure*)b);
}

You’ll need to change it so that it contains the following.

static inline UBool objectEquals(const UObject* a, const UObject* b) {
     // LATER: return *a == *b
     return *((const Measure*)a) == *b;
}

Save the file, and run the build command again. While that’s running, go find something else to do. Have a meal, fly a kit, read a book. You’ve got time. When you return, the buld should have been successful.

Hello World

Now, let’s make a hellow world program. Google already has a v8 hellow would example that we can use to see that our build was successful. We will use it for now, as I’ve not discussed anything about the v8 object library yet. Open Microsoft Visual Studio and create a new C++ Console application. Replace te code in the cpp file that it provides with Google’s code.


// Copyright 2015 the V8 project authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include "libplatform/libplatform.h"
#include "v8-context.h"
#include "v8-initialization.h"
#include "v8-isolate.h"
#include "v8-local-handle.h"
#include "v8-primitive.h"
#include "v8-script.h"

int main(int argc, char* argv[]) {
    // Initialize V8.
    v8::V8::InitializeICUDefaultLocation(argv[0]);
    v8::V8::InitializeExternalStartupData(argv[0]);
    std::unique_ptr<v8::Platform> platform = v8::platform::NewDefaultPlatform();
    v8::V8::InitializePlatform(platform.get());
    v8::V8::Initialize();

    // Create a new Isolate and make it the current one.
    v8::Isolate::CreateParams create_params;
    create_params.array_buffer_allocator =
        v8::ArrayBuffer::Allocator::NewDefaultAllocator();
    v8::Isolate* isolate = v8::Isolate::New(create_params);
    {
        v8::Isolate::Scope isolate_scope(isolate);

        // Create a stack-allocated handle scope.
        v8::HandleScope handle_scope(isolate);

        // Create a new context.
        v8::Local<v8::Context> context = v8::Context::New(isolate);

        // Enter the context for compiling and running the hello world script.
        v8::Context::Scope context_scope(context);

        {
            // Create a string containing the JavaScript source code.
            v8::Local<v8::String> source =
                v8::String::NewFromUtf8Literal(isolate, "'Hello' + ', World!'");

            // Compile the source code.
            v8::Local<v8::Script> script =
                v8::Script::Compile(context, source).ToLocalChecked();

            // Run the script to get the result.
            v8::Local<v8::Value> result = script->Run(context).ToLocalChecked();

            // Convert the result to an UTF8 string and print it.
            v8::String::Utf8Value utf8(isolate, result);
            printf("%s\n", *utf8);
        }

        {
            // Use the JavaScript API to generate a WebAssembly module.
            //
            // |bytes| contains the binary format for the following module:
            //
            //     (func (export "add") (param i32 i32) (result i32)
            //       get_local 0
            //       get_local 1
            //       i32.add)
            //
            const char csource[] = R"(
        let bytes = new Uint8Array([
          0x00, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00, 0x01, 0x07, 0x01,
          0x60, 0x02, 0x7f, 0x7f, 0x01, 0x7f, 0x03, 0x02, 0x01, 0x00, 0x07,
          0x07, 0x01, 0x03, 0x61, 0x64, 0x64, 0x00, 0x00, 0x0a, 0x09, 0x01,
          0x07, 0x00, 0x20, 0x00, 0x20, 0x01, 0x6a, 0x0b
        ]);
        let module = new WebAssembly.Module(bytes);
        let instance = new WebAssembly.Instance(module);
        instance.exports.add(3, 4);
      )";

            // Create a string containing the JavaScript source code.
            v8::Local<v8::String> source =
                v8::String::NewFromUtf8Literal(isolate, csource);

            // Compile the source code.
            v8::Local<v8::Script> script =
                v8::Script::Compile(context, source).ToLocalChecked();

            // Run the script to get the result.
            v8::Local<v8::Value> result = script->Run(context).ToLocalChecked();

            // Convert the result to a uint32 and print it.
            uint32_t number = result->Uint32Value(context).ToChecked();
            printf("3 + 4 = %u\n", number);
        }
    }

    // Dispose the isolate and tear down V8.
    isolate->Dispose();
    v8::V8::Dispose();
    v8::V8::DisposePlatform();
    delete create_params.array_buffer_allocator;
    return 0;
}

If you try to build this now, it will fail. You need to do some configuration. Here is a quick list of the configuration changes. If you don’t understand what to do with these, that’s find. I’ll will walk you through applying them.

VC++ Directories : 
	Include : v8\include
	Library Directories<Debug>: v8\out.gn\x64.debug\obj
	Library Directories<Release>: v8\out.gn\x64.release\obj

C/C++
	Code Generation
		Runtime Library <Debug>: /MTd
		Runtime Library <Release> /Mt
	Preprocessors
		VV8_ENABLE_SANDBOX;V8_COMPRESS_POINTERS;_ITERATOR_DEBUG_LEVEL=0;
		
Linker
	Input
		Additional Dependencies: v8_monolith.lib;dbghelp.lib;Winmm.lib;

Right-click on the project file and select “Properties.” From the pane on the left, select VC++ Directories. In the drop-down on the top, select All Configurations. On the right there is a field named Include. Select it, and add the full path to your v8\include directory. For me, this will be c:\shares\projects\google\v8\include. If you build in a different path, it will be different for you. After adding the value, select Apply. You will generally want to press Apply after each field that you’ve changed.

Change the Configuration drop-down at the top to Debug. In the Library Directories entry, add the full path to your v8\out.gn\x64.debug\obj folder and click Apple. Change the Configuration dropdown to Release and in Library Directories add the full path to your v8\out\gn\x64.release\obj folder.

From the pane on the left, expand C/C++ and select Code Generation. On the right, set the Debug value for Runtime Library to /MTd and set the Release value for the field to /Mt.

Change the Configurations option to All and set add the following values to Preprocessors

V8_ENABLE_SANDBOX;V8_COMPRESS_POINTERS;_ITERATOR_DEBUG_LEVEL=0;

Keep the Configurations option on ALL. Expand Linker and select Input. For Additional Dependencies enter v8_monolith.lib;dbghelp.lib;Winmm.lib;

With that entered, press Okay. You should now be able to run the program. It will pass some values to the JavaScript engine to execute and print out the values.

What’s Next

My next set of objectives is to demonstrate how to project a C++ object into JavaScript. I also want to start thinning out the size of these files. On a machine that is using the v8 binaries, the entire build tools are not needed. At the end of the above process the b8 folder has 12 gigs of files. If you copy out only the build files and headers needed for other projects, the file size is reduced to 3 gigs. Further reductions could occur through changing some of the compilation options.


Mastodon: @j2inet@masto.ai
Instagram: @j2inet
Facebook: @j2inet
YouTube: @j2inet
Telegram: j2inet
Twitter: @j2inet

Image Maps Made for Creatives

Many of the people with which I work are classified as being technical or creative (though there is a spectrum between these classifications). On many projects, the creative workers design UIs while the technical people transform those designs into something that is working. I’m a proponent of empowering those that are creating a design with the ability to implement it. This is especially preferred on projects where a design will go through several iterations.

I was recently working on a project for which there would be a menu with a map of a building. Clicking on a room in the map would take the user to web page that had information on the room. I had expected the rooms on the map to generally be rectangular. When I received the map, I found that many of the rooms had irregular shapes. HTML does provide a solution for defining shapes within the image that are clickable through Image Maps. I’ve never been a fan of those, and for this specific project I would not be able to ask the creatives to update the image map. I decided on a different solution. I can’t show the picture of the map that was the image being displayed. As an example, I’ll use a picture of some lenses that are sitting in the corner of my room.

Collection of Lenses

Let’s say I wanted someone to be able to click on a lens and get information about them. In this picture, these lenses overlap. Defining rectangular regions isn’t sufficient. I opened the picture in a paint program and applied color in a layer over the objects of interest. Each color is associated with a different object classification. Image editing isn’t my skill though. The result looks rough, but sufficient. This second image will be used in an HTML page to figure out which object that someone has clicked on. I’ll have a mapping of these color codes to objects.

When a user clicks on the real image, the pixel color data is extracted from the associated image map and converted to a hex string. To extract the pixel data, the image map is rendered to a canvas off-screen. The canvas’s context exposes methods for accessing the pixel data. The following code renders the image map to a canvas and sets a variable containing the canvas 2D context.

function prepareMap(width, height) {
    var imageMap = document.getElementById('target-map');
    var canvas = document.createElement('canvas');
    canvas.width = width;
    canvas.height = height;
    var canvasContext = canvas.getContext('2d');
    canvasContext.drawImage(imageMap, 0, 0, imageMap.width, imageMap.height);
    areaMapContext = canvasContext;
}

I need to know the position of the image relative to the browser’s client area. To retrieve that information, I have a method that recurses through the positioning containers for the image and accumulates the positioning settings to a usable set of coordinates.

function FindPosition(oElementArg) {
    if (oElementArg == undefined)
        return [0, 0];
    var oElement = oElementArg;
    if (typeof (oElement.offsetParent) != "undefined") {
        for (var posX = 0, posY = 0; oElement; oElement = (oElement.offsetParent)) {
            posX += oElement.offsetLeft;
            posY += oElement.offsetTop;
        }
        return [posX, posY];
    }
    return [0.0];
}

The overall flow of what happens during a click is defined within mapClick in the example code. To convert the coordinates on which someone clicked (relative to the body of the document) to coordinates relative to the image, I only need to subtract the offsets that are returned by the FindPosition function. The retrieved colorcode for the area on which the user clicked can be used as an indexer on the color code to product identifier mapping. The product identifier is used as a indexer on the product identifier to product data mapping.

function mapClick(e) {
    var PosX = e.pageX;
    var PosY = e.pageY;
    var position = FindPosition(targetImage);
    var readX = PosX - position[0];
    var readY = PosY - position[1];

    if (!areaMapContext) {
        prepareMap(targetImage.width, targetImage.height);
    }
    var pixelData = areaMapContext.getImageData(readX, readY, 1, 1).data;
    var newState = getStateForColor(pixelData[0], pixelData[1], pixelData[2]);
    var selectedProduct = productData[newState];
    showProduct(selectedProduct);
}

Once could simplify the mappings by having the color data map directly to product information. I chose to keep the two separated though. If the color scheme were ever changed (which I think is very possible for a number of reasons) I thought it better that these two items of data be decoupled from each other.

You can find the full source code for this post on GitHub at this url. Because of security restrictions in the browser, you must run this code within a local HTTP server. Attempting to run it from the file system will fail due to limitations in how an application can use the data it loads when loaded from a local file. I also have brief videos on my social media account to walk through the code.


Posts may contain products with affiliate links. When you make purchases using these links, we receive a small commission at no extra cost to you. Thank you for your support.

Twitter: @j2inet
Instagram: @j2inet
Facebook: j2inet
YouTube: j2inet
Telegram: j2inet

Uploading Large Files in Express

I’m back from taking a break. I’ve been working on a number of projects that culminated a crunch-time scenario where there was lots to get done before I went on a vacation for which I would have no computer and an Internet connection that would range from flaky-to-none. When going on trips I often bring some movies with me to watch when I’m taking a break and retreating to my lodging. For this trip, I did something a bit different than usual. Instead of copying my media directly to my mobile device, I decided I would copy it to a Raspberry Pi and set it up as a movie streaming server. (More on elements of my particular scenario at the end of this post) Early last year I had posted something on using Express and NodeJS for streaming. For this trip, I copied that to a Pi that had a lot of storage and did some other things to prepare it for my environment. These included:

  • Setting up the Pi as a Wireless Access Point
  • Setting my Application to Run on Boot
  • Adding the Ability to Upload files

While I didn’t perform these changes during my trip, I realized there was some other functionality I wanted this solution to have, including

  • Converting videos of other formats to MP4
  • Ensuring the MP4’s metadata was arranged for streaming
  • Adding some service to convert uploaded files as needed

Many of these changes could be topics of their own. Let’s talk about uploading files.

Uploading a File with express-fileupload

My first solution for uploading a file was to make a simple web form allowing a file to be selected and add a router to my web application to handle file uploads. I’ll start off telling you that this didn’t work. Skip ahead to the next section if you don’t want to read about this initial failed attempt. The webform itself is simple. A minimalist version of it follows.

<form action='/upload' enctype='multipart/form-data'  method='post'>
	<input type='file' id='videoUploadElement' name='video' />
	<input type='submit' />
</form>

On the server-side, I added the a router to accept my file uploads and write them to a target folder. Upon receiving the bits of the file, I’m writing it to a designated folder preserving the original file name.

//Do not use. For demonstration purposes only. While this
//appears to work. But the entire file is loaded to memory
//before being written to disk. I've found with large files
//this can easily slow down or halt the Pi. Use the router

const express = require('express');
const router = express.Router();

router.post('/', async(req,res) => {
	try {
		if(!req.files) {
			console.log(req.body);
			res.send({
				status: false,
				message: 'No File'
			});
		} else {
			let video = req.files.video;
			video.mv('./uploads/' + video.name);
			res.send({
				status: true,
				message: ' File Uploaded',
				data: {
					name: video.name,
					mimetype: video.mimetype,
					size: video.size
				}
			});
		}
	} catch(err) {
		res.status(500).send(err);
	}
});
module.exports = router;

Testing it out on a small file, it works just fine. Testing it out on a large file locks up the Pi. The issue here is that **all** of the file must be loaded to memory before it is written to storage. The files I was uploaded were feature-length high-definition videos. I tried uploading a 4.3 gig video and saw the Pi progress from less responsive to all responsive. This wasn’t going to work for my needs.

Uploading and Streaming to Storage with Busboy

Instead of using express-fileupload I used connect-busboy. Using Busboy I’m able to stream the bits of the file to storage as they are being uploaded. The complete file does **not** need to be loaded to memory. When the upload form sends a file, the busboy middleware makes the file’s data and meta-data available through the request object. To save the file, create a file stream to which the data will be written and pipe the upload (busboy) to the file storage stream.

//Uses Busboy. See: https://github.com/mscdex/busboy

const express = require('express');
const router = express.Router();
const path = require('node:path');
const Busboy = require('connect-busboy');
const fs = require("fs");

const UPLOAD_FOLDER = 'uploads';

if(!fs.existsSync(UPLOAD_FOLDER)) {
    fs.mkdirSync(UPLOAD_FOLDER, { recursive: true });
}

router.post('/', async(req,res, next) => {
	try {        
        
        console.log('starting upload');        
        console.log(req.busboy);
        req.busboy.on('field', (name, val, info)=> {
            console.log(name);
        });
        
        req.busboy.on('file', (fieldname, uploadingFile, fileInfo) => {
            console.log(`Saving ${fileInfo.filename}`);
            var targetPath = path.join(UPLOAD_FOLDER, fileInfo.filename);
            const fileStream = fs.createWriteStream(targetPath);
            uploadingFile.pipe(fileStream);
            fileStream.on('close', ()=> {
                console.log(`Completed upload ${fileInfo.filename}`);
                res.redirect('back');
            });
        });
        req.pipe(req.busboy);
    } catch (err) {
        res.status(500).send(err);
    }
});

module.exports = router;

A fairly complete version of my app.js follows, including the reference to the router that has the streaming upload method.

const express = require('express');
const createError = require('http-errors');
const path = require('path');
const { v4: uuidv4 } = require('uuid');
const Busboy = require('connect-busboy');
require('dotenv').config();

app = express();
app.use(Busboy({
   immediate: true,
   limits: 10* 1073741824,
   highWaterMark: 4 * 1048576, // Set 4 megabyte buffer
}));

app.use(express.static('public'));


const libraryRouter = require('./routers/libraryRouter');
const videoRouter = require('./routers/videoRouter');
const uploadRouter = require('./routers/uploadRouter');
const streamRouter = require('./routers/streamRouter');

app.use('/library', libraryRouter);
app.use('/video', videoRouter);
app.use('/upload', uploadRouter);
app.use('/streamUpload', streamRouter);


app.use(function (req, res, next) {
   console.log(req.originalUrl);
   next(createError(404));
});
app.set('views', path.join(__dirname, 'views'));
app.engine('html', require('ejs').renderFile);
app.set('view engine', 'html');

With this in place, I have a working upload feature. I don’t want to stop there though. FFMPEG runs on the Pi and I may take advantage of this to do additional transformations on media to prepare it for streaming. I’m going to add code for handling this processing next.

GEeting the Code

If you want to try the code out, it is on GitHub. You can find it here: https://github.com/j2inet/VideoStreamNode.

About my Scenario

I only brought the Pi for movies, but I ended up using it for a lot more. I had an iPhone and an Android phone with me on the trip. I managed to fill up the memory on my iPhone and needed a place to move some video. The Android phone had plenty of space, but there was no direct method to transfer data between the two devices. At one point I also wanted to edit video using Adobe Rush, only to find that Adobe doesn’t allow Rush to be installed on Samsung’s latest generation of phones at this time. My iPhone had Rush, thus I had to get information transferred from my Android Phone to my iPhone to do this.

Had these use cases come to mine I probably would have acquired a USB adapter for my Phone (the Android phone will connect to my USB-C drives with no special adapters needed). I’m sure there are apps that could be used to bridge this gap, but my iPhone has no coverage when I’m out of the country. I didn’t have access to try out such apps. Some part of me feels that it is a bit silly that data can’t move between these two systems without some intermediary, such as another computer, or dongle and drives. But given that we are still not at a point where media in text messages is reliably transferred without quality loss it isn’t surprising.

Given more time, I might make an app specifically for transferring data between devices in the absence of a common network. (This is not something I am promising to do though).


Posts may contain products with affiliate links. When you make purchases using these links, we receive a small commission at no extra cost to you. Thank you for your support.

Twitter: @j2inet
Instagram: @j2inet
Facebook: j2inet
YouTube: j2inet
Telegram: j2inet

Saving your Automotive data from Automatic.com with NodeJS

One of the less glamorous experiences that has come with Consumer IoT products is a parade of items that cease to work when the company that made them shuts down. The economic down turn of 2020 has seen it’s share of products that have experienced this. In a few days the automotive adapters from Automatic.com will be on the list of causalities.

Automatic.com provided a device that connect to a car through the ODB-II port and would relay information about a vehicle back to the owner that they could view through an application or through the web portal.  Through these views someone could see where their car is, read any engine trouble code, view the paths the car has traveled, and view information about hard breaks, hard accelerations, and other car data.

I have three of these adapters and have data from tracking the vehicles for the past 5 years. I would rather keep my information. Looking on Automatic.com’s page about the shutdown there is a statement about exporting one’s data.

To download and export your driving data, you’ll need to log in to the Automatic Web Dashboard on a desktop or laptop computer at dashboard.automatic.com. Click on the “Export” button in the lower right-hand corner of the page.
[…]
Although the number of trips varies between each user, the web app may freeze when selecting “Export all trips” if you are exporting a large amount of driving data. We recommend requesting your trip history in quarterly increments, but if you drive in excess of 1,500 trips per year, please request a monthly export.”

I tried this out myself and found it to be problematic. Indeed, after several years of driving across multiple vehicles the interface would freeze on me. I could only actually export a month of data at a time.  Rather than download my data one month at a time across 60 months it was easier to just write code to download my data. Looking through the API documentation there were three items of data that I wanted to download. I’ll be using NodeJS to access and save my data.

To access the data it’s necessary to have an API key. Normally there would be the process of setting up OAUTH authentication to acquire this key. But this code is essentially throw away code; after Automatic completes it’s shutdown it won’t be good for much. So instead I’m going to get a key directly from the developer panel on https://developer.automatic.com. I’ve got more than one automatic account. It was necessary to do this for each on of the accounts to retrieve the keys.

On https://developer.automatic.com/my-apps#/ select “Create new App.”  Fill out some description for the app. After the entry is saved select “Test Token for your Account.”

Automatic.com_TestToken

You’ll be presented with a key. Hold onto this. I placed my keys in a comma delimited string and saved it to an environment variable named “AutomaticTokens.”  It was an easy location from which to retrieve them where I won’t have to worry about accidentally sharing them while sharing my code.  In the code I will retrieve these keys, break them up, and process them one at a time. 

var AutomaticTokensString = process.env.AutomaticTokens;
const AutomaticTokenList = AutomaticTokensString.split(',');
For calling Automatic.com’s REST based API most of the calls look the same differing only in the URL. I’ve made a method to make calls, accumulate the responses, and pass them back.
function AutomaticAPI(path) {
    return new Promise(function(resolve,reject) {
        var options = {
            host: 'api.automatic.com',
            path: path,
            port:443,
            method: 'GET',
            headers: {Authorization:`Bearer ${AuthorizationToken}`}
        };
    
        var req = https.request(options,function(res) {
            let data = ''
            res.setEncoding('utf8');
            res.on('data', function (chunk) {
                data += chunk;
            });
            res.on('end',function() {
                resolve(JSON.parse(data));
            });
        });
    
        req.on('error', function(e) {
            console.error('error',e);
            console.log('problem with request: ' + e.message);
            reject(e);
          });    
          req.end();
    });
}
This greatly simplifies the implementations of the rest of the calls.
Now that I have the keys and something in place to simply the calls the first piece of information to retrieve is a list of vehicles in the account. This information is the root of of the other information that I wanted to save.
function listVehicles() {
    return new Promise((resolve,reject)=>{
        AutomaticAPI('/vehicle/')
        .then(function(d) {
            resolve(d);
        })    
    });
}

Let’s take a look at one of the responses from this call.

{
     _metadata: { count: 1, next: null, previous: null },
     results: [
          {
               active_dtcs: [],
               battery_voltage: 12.511,
                created_at: '2017-01-28T21:49:24.269000Z',
               display_name: null,
               fuel_grade: 'regular',
               fuel_level_percent: -0.39215687,
               id: 'C_xxxxxxxxxxxxxxxxx',
               make: 'Honda',
               model: 'Accord Sdn',
               submodel: 'EX w/Leather',
               updated_at: '2018-07-24T19:57:54.127000Z',
               url: 'https://api.automatic.com/vehicle/C_xxxxxxxxxxxxxxxxx/',
               year: 2001
          }
     ]
}

From the response I need to id field to retrieve the other information. While this response doesn’t change any ground breaking information I’m persisting it to disc so that I can map the other data that I’m saving to a real car.

The next thing I grab is the MIL. This contains the last set of engine trouble codes with date stamps.

function getMil(vehicleID, limit,) {
    return new Promise((resolve,reject)=>{
        var url = `/vehicle/${vehicleID}/mil/`;
        console.debug('url',url);
        AutomaticAPI(url)
        .then((data)=>resolve(data));
    });
}

Here is a sample response.

{
   "_metadata": {
      "count": 3,
      "next": null,
      "previous": null
   },
   "results": [
      {
         "code": "P0780",
         "on": false,
         "created_at": "2019-07-09T20:19:04Z",
         "description": "Shift Error"
      },
      {
         "code": "P0300",
         "on": false,
         "created_at": "2018-02-24T16:05:02Z",
         "description": "Random/Multiple Cylinder Misfire Detected"
       },
      {
         "code": "P0306",
         "on": false,
         "created_at": "2018-02-24T16:05:02Z",
         "description": "Cylinder 6 Misfire Detected"
      }
   ]
}

The last, and most important piece of information that I want is the trip data. The trip data contains a start address, end address, and the path traveled.  Information about hard stopping and hard acceleration and many other items of data is stored within trips. For the REST API a start time and end time are arguments to the request for trip information. The API is supposed to support paging when there are a lot of trips to return. Some number of trips are returned from a request along with a URL that contains the next page of data. When I’ve requested the second page I get an error response back. Given the short amount of time until the service shuts down it doesn’t feel like the time to report that dependency to the staff at Automatic.com. Instead I’m requesting the travel information for 7 to 9 days at a time. The results come back in an array. I’m writing each trip to it’s own file.

To more easily navigate to a trip I’ve separated them out in the file system by date. The folder structure follows this pattern.

VehicleID/year/month/day

The information within these files is the JSON portion of the response for that one trip without any modification.  The meaning of the information in most of the fields of a response are easy to intuitively understand without further documentation. The field names and the data values are descriptive. The one exception is the field “path.” While the purpose of this field is known (to express the path driven) the data value is not intuitive. The data value is an encoded poly line. But documentation on how this is encoded can be found in the Google Maps documentation ( https://developers.google.com/maps/documentation/utilities/polylinealgorithm ).

Now that I’ve got my data saved I may implement my own solution for continuing to have access to this functionality. At first glance I see some products that appear to offer similar services. But the lack of an API for accessing the data makes them a no-go to me. I’m instead am learning towards making a solution with an ELM327 ODB-II adapter, something I’ve used before.

Download Code: https://github.com/j2inet/AutomaticDownload

twitterLogofacebookLogoyoutubeLogoInstagram Logo

Linked In

 

 

 



ODB II Bluetooth Adapter



OSB II Scanner

TypeScript in Tizen

I was writing a program to run on my television and encountered a scenario that I’ve encountered many times before; an HTML enabled device supports a JavaScript standard that is older than the one that I would like to use. The easiest workaround for this is to use a tool that will compile from a more recent version of JavaScript (or something similar) back to the version that is supported by the hardware. This is something I’ve done when developing for BrightSign and other devices.

For targeting the Tizen based Television I decided that I would use TypeScript to accomplish this; in addition to getting access to some more recent features that can be found in JavaScript there’s we also get type checking.

A bit of work was required to get this working though. On my first attempt I tried includint the TypeScript files in the same folder as the project. This doesn’t work;when the project is being compiled the compiler will try to take these files and package them in the solution. This isn’t something that we want to happen. It’s necessary to have these files in a folder that is outside of the project folder to prevent this from happening. I moved the files and made a TypeScript configuration file that specified the destination to which I wanted the resulting JavaScript files moved.

{
  "compilerOptions": {
    /* Basic Options */
    "target": "es2015",
    "module": "commonjs",  
    "sourceMap": false,   
    "outDir": "../tizenWorkspace/projectName/js"
    "strict": true,                           
    "noImplicitAny": true,                 
  }
}

This almost works. The next problem encountered is that when there is a reference to anything on the tizen object the compiler will complain about it net having been declared. The tizen object, not being a web standard object, is not something that is recognized by the compiler. There are two ways to handle this project. A work around would be to declare the tizen object as being of type any. With this declaration the compiler will just ignore what ever we do with the object and not complain.

I made a TypeScript definition file named tizen.d.ts in which to place my definitions. TypeScript already has an understanding of the interface provided by the Window object. To augment this I declare another interface that will be merged with the understanding that TypeScript has and added a definition for the tizen member there.

declare	interface Window {  tizen:any }

That works, but that’s also eliminating some of the type safety features that that TypeScript has to offer. Instead of working around the problem I wanted to address it. I wanted to provide the type definitions for the Tizen object.

There’s a project called Definitely Typed in which contributors make TypeScript definitions that can be downloaded and shared to other developers that are targeting the same environment. At first glance there appears to be existing entries for targeting Tizen within the collection. But upon further inspection it turns out that the definitions that are there (at the time of the writing of this post) are for targeting a cross development tool that also supports Tizen. that’s not what I needed. Instead of relying on community provided definitions I’ll have to make my own. When I’m done though I may have a definition file that could be shared through Definitely Typed. Since that repository is constantly being updated I would encourage seeing what it has to offer before using the code that I provide here.

declare	interface Window {  tizen:ApplicationManager}

This is when I start my descent down the rabbit hole. To define the ApplicationManager interface that is implemented by the tizen object there are a number of other interfaces that must be defined. Those interfaces have dependencies on other interfaces.

The interfaces for the various objects are documented and can be found on a Tizen.org page. Browsing through it there are some types mentioned that ultimately are strings of some type of another. Within TypeScript we can make a declaration that is similar to a typedef for equating some custom type to another.

type ApplicationId = string;
type ApplicationContextId = string;
type PackageId = string;

There is also a frequently used callback type for successes and errors of callbacks. The links to the documentation for the functions’ call signatures are broken taking me to a 404 page. I was more generic with defining these in my type definitions until I can get the specifics of the actual accepted call signatures.

type SuccessCallback = (...args: any[]) => void;
type ErrorCallback = (...args: any[]) => void;

The rest of the definitions are interfaces and follow the same patterns. I’m showing a few of the interfaces closer to the root of the definitions.

declare	interface Window {  tizen:ApplicationManager}
declare var tizen:tizenInterface;

interface tizenInterface {
    application:ApplicationManager;
}

interface ApplicationManager { 
    getCurrentApplication():Application;

    kill(contextId:string,
              successCallback:SuccessCallback,
              errorCallback:ErrorCallback):void ;

    launch( id:string, //ApplicationId
                successCallback:SuccessCallback,
                errorCallback:ErrorCallback):void;
    launchAppControl(appControl:ApplicationControl,
                        id?:ApplicationId, //ApplicationId
                          successCallback?:SuccessCallback,
                          errorCallback?:ErrorCallback,
                          replyCallback?:ApplicationControlDataArrayReplyCallback):void ;
     findAppControl(appControl:ApplicationControl,
                        successCallback:FindAppControlSuccessCallback,
                        errorCallback:ErrorCallback):void;

    getAppsContext(successCallback:ApplicationContextArraySuccessCallback,
                        errorCallback:ErrorCallback):void ;
    getAppContext(contextId:string):ApplicationContext;
    getAppsInfo(successCallback:ApplicationInformationArraySuccessCallback,
                     errorCallback?:ErrorCallback):void;
    getAppInfo(id?:ApplicationId ):ApplicationInformation;
    getAppCerts(id?:ApplicationId ):Array;
    getAppSharedURI(id?:ApplicationId ):string;
    getAppMetaData(id?:ApplicationId ):Array;
    addAppInfoEventListener(eventCallback:ApplicationInformationEventCallback):number;
    removeAppInfoEventListener( watchId:number):void ;    
}

There are a lot more objects that could be defined for Tizen. If you’ve come along this article checkout the DefinitelyType archives first. If you don’t find Tizen devinitions there you can download the version of the video that I have from here.

Progressive Web Apps in Chrome

Progressive Web Apps (PWA) are HTML based applications that run as though they are desktop applications.  Google Chrome received support for PWAs on Chrome OS in May with the release of Chrome 67.  Linux and Windows received support in August with the release of Chrome 70.  Support for Mac OS X is yet to come.

Download code (415 KB)

siderealLarge

One of the first differences that stands out for PWAs is that they can run in their own application window and are indistinguishable from other applications running on a machine. That difference is largely visual. But the differences extend well beyond what is visible. Resources that are not usually available to an HTML page are available to a PWA such as access to Bluetooth, serial ports, UDP networking, and more.  Chrome PWAs can be installed and have their own icon in your programs menu and function offline.

There are requirements that must be satisfied before an HTML page can be installed as a PWA.  These are the conditions that must be met.

  • The page must be served over SSL/HTTPS.
  • The page must have a service worker with a fetch handler.
  • User engagement requirements must be met (interaction with the domain for at least 30 seconds).
  • A manifest must be present.
    • 192px and 512px icons must be included.
    • Application must have a short name and long name.
    • The display mode must be specified.
    • start_url must be specified.

 

If all of these requirements are met Chrome will trigger a beforeinstallprompt event for the web page. Once this event is triggered your application can present the user with an install prompt.  Depending on the Chrome version your application may be able to suppress this prompt and display it to the user later (allowing you to decide where in the interaction flow that the prompt shows up) or your app might not be allowed to suppress it.

I’ll make a minimilastic application that satisfies the requirements for being a PWA.  The application that I’ll make will calculate sidereal time. Sidereal time is a time tracking system used by astronomers and is always expressed in 24 hour format. The usual system of tracking time was formed around trying to map the time of the day to the position of the sun (solar time, though it is far less than perfect). Sidereal time is based on the position of the stars relative to the observer. I will not talk much about the algorithm behind this calculation much here. I talked about calculating sidereal time in an application I had made for the now defunct Windows Phone 7; while that OS is no more the description I gave on how sidereal time works is still applicable.

Using SVG I’ve made a simple 24 hour clock face. The clock face is really there for aesthetics. Chances are if you try to read the hands of the clock the hour hand will cause confusion since it’s position on a 24 hour clock will not meet expectations that have been formed from being able to read a 12 hour clock.  The digital readout is the part that will actually give the information of interest. Every second the time is updated and the hands animate to their new position. There’s also a gear icon for opening the settings interface.

sampleApp

Satisfying the SSL/HTTP Requirement

A lot of the necessary features are only available if your application is being served over SSL. If you don’t see HTTPS in the address bar then these features simply will not work. To satisfy this requirement for now I’m using Google Firebase and the temporary URL that it has assigned to me. I don’t plan on keeping this URL forever, but at the time of this post you can play with the application over at https://siderealtimepiece.firebaseapp.com.

Satisfying Manifest Resources Requirements

The manifest for my application is in the root directory of the application. It is a JSON formatted file with information on where the program icons can be found, the starting URL, and the name of the application as it should appear on the user’s machine.

{
    "short_name": "Sidereal",
    "name": "Sidereal Time Piece",
    "icons": [
      {
        "src": "./images/sidereal192.png",
        "type": "image/png",
        "sizes": "192x192"
      },
      {
        "src": "./images/sidereal512.png",
        "type": "image/png",
        "sizes": "512x512"
      }
    ],
    "start_url": "index.html?pwa=true",
    "background_color": "#000080",
    "display": "standalone",
    "scope": "./",
    "theme_color": "#FFFFFF"
  }
  

The Service Worker

To satisfy the service worker requirement there’s a JavaScript file in the root of this application’s files named sw.js. The service worker works in the background behind the page. For this application we only want the service worker to do two things; respond to an install event by caching the required files locally and serve up those files when needed. The list of the files that are to be cached are in an array named urlsToCache. When the service worker response to the install event it will pass this list of URLs to a call of the addAll method on the cache object. The cache object will then download the resources at these URLs and save them locally where we can use them offline.

var CACHE_NAME = 'siderealclock-cache';
var urlsToCache = [
  './',
  './styles/main.css',
  './scripts/app.js',
  './scripts/jquery-3.3.1.min.js',
  './images/sidereal192.png',
  './images/sidereal512.png',
  './images/siderealLarge.png',
  './404.html'
];

self.addEventListener('install', function(event) {
  // Perform install steps
  event.waitUntil(
    caches.open(CACHE_NAME)
      .then(function(cache) {
        console.log('Opened cache');
        return cache.addAll(urlsToCache);
      })
  );
});

For the fetch event I’m using code from a Google recommendation. This handler will serve the contents from the cache when there is a cache hit and also add new files to the cache when a request is made for a file that isn’t already there.

self.addEventListener('fetch', function(event) {
    event.respondWith(
      caches.match(event.request)
        .then(function(response) {
          // Cache hit - return response
          if (response) {
            return response;
          }
  
          // IMPORTANT: Clone the request. A request is a stream and
          // can only be consumed once. Since we are consuming this
          // once by cache and once by the browser for fetch, we need
          // to clone the response.
          var fetchRequest = event.request.clone();
  
          return fetch(fetchRequest).then(
            function(response) {
              // Check if we received a valid response
              if(!response || response.status !== 200 || response.type !== 'basic') {
                return response;
              }
  
              // IMPORTANT: Clone the response. A response is a stream
              // and because we want the browser to consume the response
              // as well as the cache consuming the response, we need
              // to clone it so we have two streams.
              var responseToCache = response.clone();
  
              caches.open(CACHE_NAME)
                .then(function(cache) {
                  cache.put(event.request, responseToCache);
                });
  
              return response;
            }
          );
        })
      );
  });
  

This file must be registered as the service worker for it to be able to do anything. In one of the JavaScript files loaded by the page I check the navigator object to ensure there is a serviceWorker member (if there isn’t then the browser in which the code is running currently doesn’t support service workers). If it is there then the service worker can be registered with navigator.serviceWorker.register(path_to_service_worker).

if('serviceWorker' in navigator) {
    navigator.serviceWorker
             .register('./sw.js')
             .then(function() { console.log("Service Worker Registered"); });
  }

Handling the Install Prompt

If your code is running on a Chrome implementation that supports it you can defer the presentation of the installation prompt. In my case I’ve decided to defer it and make a button available in the settings UI. The variable installPrompt will hold the reference to the event object that when activated will present the user with the Chrome install UI. When the event is raised the variable is populated with the event object and the install button within my settings UI is made visible.

var installPrompt;


function beforeInstall(e) { 
    console.log('beforeInstallPrompt()')
    e.preventDefault();
    installPrompt = e;
    $('.installUI').show();
}


window.addEventListener('beforeinstallprompt', beforeInstall);
    $('.installButton').on('click', function(){
        installPrompt.prompt();
        installPrompt.userChoice
          .then((choiceResult) => {
            $('.installUI').hide();
            installPrompt = null;
          });
      
    });

Testing the application on Chrome on Ubuntu Linux when I select my install button the Chrome install prompt shows.

Chrome Desktop Install Prompt
The Install Prompt that shows on Google Chrome on a desktop

Program Launchers on the Desktop

On the desktop once installed the icon for the PWA shows up in the computer’s program launcher.  It also shows up in the Chrome app list. When launched since this application was made to run in standalone mode the application runs in it’s own window with the OS appropriate buttons for going full screen, minimizing, and closing the window. My test application uses location services to acquire the longitude at which the sidereal time is being calculated. When run in a regular browser window I’m prompted each time I visit the page to give permission for location information. This gets a little annoying after a while. When the application is running in stand alone mode the application’s border shows an icon indicating that the location is being detected. Clicking on the icon gives the user the ability to change the location permissions for the application.

Samsung Internet Compatibility

Samsung Internet, the default browser for a long period on many Samsung phones, also supports PWAs. (Samsung Internet can also be installed on non-Samsung phones). Samsung Internet is a Chromium based browser and Samsung is one of the contributors to the Chromium project.  It may come as no surprise that no code changes are necessary for this application to work on  The UI it presents for installing PWAs is different than what Chrome presents. When Samsung Internet detects that a page can be installed as a PWA an icon is shown in the address bar that resembles a house with a plus in the center. Selecting it will add the icon to the home screen. The icon shows with a smaller image of the Samsung Internet icon indicating that it is a PWA.  The beforeinstallprompt event will never be triggered. Since the presentation of the custom install button was driven by this event it simply will not show.

SamsungPWACentered

Adding iOS Compatibility

If you saw the original iPhone announcement back in 2007 Steve Jobs had announced that making apps for the iPhone could be done with HTML; at the time there was no SDK available to developers and if they wanted to target the iPhone they were making a web app that had an icon on the home screen. From 2007 to 2018 Apple didn’t do much to advance the platform. It wasn’t until March 2018 that Apple made significant updates to their support to HTML based applications. Apple added support for web manifest, and services workers, web assembly, and other features.

There’s not 100% parity between iOS and Android for available features in PWA. On iOS storage is limited to 50MB per app. On Android the application can request more storage. Android BWAs also have access to Bluetooth features, speech recognition, background sync, and other features. For my sample application none of these mentioned differences matter. While the Android implementations have UI notifications that let the user know that the app can be installed on iOS there’s no visual notification. To install the application the user must select the share option and add the page to their home screen.

Safari ignores most of the attributes of the manifest. It also doesn’t save state if the user leaves the application. So the developers must make their own implementation to save state as the user jumps in and out of the application. If you want a custom icon to show in Safari for your application Apple has a document on specifying the icon using the link tag. An icon can be specified like the following.

    <link rel="apple-touch-icon"  href="./images/icons/apple-icon-57x57.png">

If you want to specify multiple icons and allow the phone to select the most appropriate one for the user’s resolution add a sizes attribute to the tag.

   <link rel="apple-touch-icon" sizes="57x57" href="./images/icons/apple-icon-57x57.png">
    <link rel="apple-touch-icon" sizes="60x60" href="./images/icons/apple-icon-60x60.png">
    <link rel="apple-touch-icon" sizes="72x72" href="./images/icons/apple-icon-72x72.png">

My clock icon for the program shows up in the iPhone favourites list as the following.

FavouriteIcon

Offline Functionality

This application doesn’t need the internet for any functionality. It’s only inputs are the current local time and the user’s longitude. With the lack of need for any network resources and the service worker caching the required files for the application it will work just fine offline after it has been installed. If you make an application that requires network access you will want to give some thought to what to do when there is no data connection. Even if the application can’t do anything without a connection it would be better to show a friendly message than to just let the application not work.

An Alternative to the App Store

PWAs longtime might turn out to be a good alternative to app stores for some types of applications. Whether or not it is a good fit for the needs that you have will depend on the functionality that your applications require and what is available on the devices that you need to target. Apple appears to be behind on this front at the moment. But I hope that the attention that they’ve put on the platform this year to be indicative of future efforts. I’m personally am interested in what could be done when PWAs and WebAssembly are combined together. These are topics to which I hope to give a good bit of attention over the following months.

Simplified Sidereal Time

While preparing for a full moon / blue moon, I was looking at an algorithm for calculating sidereal time and had a mini epiphany. The algorithm is basically an elaborate modulo operation. Modulo is generally applied to integer values, but it can be used with decimal numbers and even fractions.

For the algorithm that I have generally used, a lot of the calculations are only for converting the date to some linear expression of time. The calendar that is usually used does not express time linearly.

The amount of time from the beginning of one month to the beginning of another month could be 28 to 31 days. With linear representations of dates, a subtraction operation is all that is needed to know the amount of time between two moments in time.

In JavaScript, this linear representation of time is shown by calling getTime() on a date object. The time value for 2019 January 10 @16:40:20 UTC  is 1547138420000. This value is the number of milliseconds since another date and time. This time and date is also 00:00:00 Sidereal time. The number of milliseconds in a sidereal day (23 hours 56 minutes 4.1 seconds) is 86164100. For any date after 2019-01-10T16:40:20 we could get the Sidereal time by doing the following:

  • Acquire the getTime() value for the date in question.
  • Subtract 1547138420000 from that value.
  • Get the modulo 86164100 for the resulting value.
  • Multiply the result by 24/86164100.

The result of these operations is the sidereal time in decimal. If you want to convert it to hour:minute:second format do the following:

var hour = Math.floor(result);
var minute = (result % 1) * 60;
var second = (minute % 1) * 60;
minute = Math.floor(minute)

solstice

-30-

NodeJS on BrightSign

When I left off I was trying to achieve data persistence on a BrightSign  (model XT1144) using the typical APIs that one would expect to be available in an HTML application. To summarize the results, I found that using typical methods of checking localStorage and indexedDB show as being available; but indexedDB isn’t actually available; and localStorage appears to work, but doesn’t survive a device reset.

The next method to try is NodeJS.  The BrightSign devices support NodeJS, but the entry point is different than a standard entry point of a NodeJS project. A typical NodeJS project will have its entry point defined in a JavaScript file. For BrightSign, the entry point is an HTML file. NodeJS is disabled on the BrightSign by default. There is nothing in BrightAuthor that will enable it. There is a file written to the memory card (that one might otherwise ignore when using BrightAuthor) that must be manually modified. For your future deployments using BrightAuthor, take note that you will want to have the file modification described in this article saved to a back-up device so that it can be restored if a mistake is made.

The file, AUTORUN.BRS, is the first point of execution on the memory card. You can look at the usual function of this file as being like a boot loader; it will get your BrightSign project loaded and transfer execution to it. For BrightSign projects that use an HTML window the HTML window is actually created by the execution of this file. I am not going to cover the BrightScript language. For those that were ever familiar with the language, it looks very much like a variant of the B.A.S.I.C. language. When an HTML window is being created it is done with a call to the CreateObject method with “roHtmlWidget” as the first parameter to the function. The second parameter to this call is a “rectangle” object that indicates the coordinates at which the HTML window will be created. The third (optional) parameter is the one that is of interest. The third parameter is an object that defines options that can be applied to the HTML window.  The options that we want to specify are those that enable NodeJS, set a storage quota, and define the root of the file system that we will be accessing.

The exact layout of your Autorun.js may differ, but in the one that I am currently working with, I have modified the “config” object by adding the necessary parameters. It is possible that in your AutoRun.brs that the third parameter is not being passed at all. If this is the case, you can create your own “config” object to be passed as a third parameter. The additions I have made are in bold in the following.

is = {
    port: 3999
}    
security = {
        websecurity: false,
        camera_enabled: true
}
    
config = {
    nodejs_enabled: true,
    inspector_server: is,
    brightsign_js_objects_enabled: true,
    javascript_enabled: true,
    mouse_enabled: true,
    port: m.msgPort,
    storage_path: "SD:"
    storage_quota: 1073741824            
    security_params: {
        websecurity: false,
        camera_enabled: true
    },
    url: nodeUrl$
}
    
htmlWidget = CreateObject("roHtmlWidget", rect, config)

Once node is enabled the JavaScript for your page will run with the capabilities that you would generally expect to have in a NodeJS project. For my scenario, this means that I now have acces to the FS object for reading and writing to the file system.

fs = require('fs');
var writer = fs.createWriteStream('/storage/sd/myFile.mp4',{defaultEncoding:'utf16le'});
writer.write("Hello World!\r\n");
writer.end()

I put this code in an HTML page and ran it on a BrightSign. After inspecting the SD card after the device booted up and was on for a few moments I saw that my file was still there (Success!).  Now I have a direction in which to move for file persistence.

One of the nice things about using the ServiceWorker object for caching files is that you can treat a file as either successfully cached or failed. When using a file system writer there are other states that I will have to consider. A file could have partially downloaded, but not finished (due to a power outage; network outage; timeout; or someone pressing the reset button; etc.). I’m inclined to be pessimistic when it comes to guaging the reliability of external factors to a system. I find it necessary to plan with the anticipation of them failing.

With that pessimism in mind, there are a couple of approaches that I can immediately think to apply to downloading and caching files.  One is to download files with a temporary name and change the name of the file from its temporary to permanent name only after the download is successful. The other (which is a variation of that solution) is to download the file structure to a temporary location. Once all of the files are downloaded, I could move the folder to its final place (or simply change the path at which the HTML project looks to load its files). Both methods could work.

I am going to try some variations of the solutions I have in mind and will write back with the results of one of the solutions.

-30-

BigInt in JavaScript

As a developer, there are some problems for which I get enjoyment out of solving.  There are some problems for which JavaScript had not been my tool of choice because of its limits on precision of the Number type.  That is no longer the case with the JavaScript type BigInt.  The number of bytes used to store a BigInt scales with the magnitude of the number.  On some browsers the following JavaScript code will show a difference between Number and BigInt.  The value in the BigInt variable increases as one would naturally expect it to.  The value in the Number variable will stay the same.

var myBigInt = BigInt(Number.MAX_SAFE_INTEGER);
var myBigResult;
console.log('BigInt value ', myBigInt);
myBigResult = myBigInt * 4n;
console.log('BigInt value * 4 = ', myBigResult);

var myNumber = Number.MAX_SAFE_INTEGER-0.9;
var myResult;
console.log('Number value ', myNumber);
myResult = myNumber *4 ;
console.log('Number value * 4 = ', myNumber);

The output for the above was as follows:

BigInt value  9007199254740991n
BigInt value * 4 =  36028797018963964n
Number value  9007199254740990
Number value * 4 =  9007199254740990

For any operation that involves values that are beyond the maximum safe integer value, the resulting value could be wrong. It is also possible to have values that appear identical when printed as a sting, but are unequal to each other when compared.  BigInt literals are expressed as an integer number suffixed with a lowercase ‘n’.  If you use the typeof operator on a BigInt the string 'bigint‘ is returned.

While there are no additional floating number types that offer high precision, BigInt can be used for some types of calculations.  For example, if you needed a big decimal value for money calculations  you could use BigInt and have your presentation of the results take into account that the number type is not storing a decimal position.  For example, if the result of a calculation were 1234 when printing the number it could be converted to a string and a period could be inserted into the right position producing the string 12.34 to the user.

The BigInt type is supported in Chrome 67.  Apple added support for Safari version 12.  Mozilla is currently working on support.  Microsoft is also working on an implementation.

 

Basic Hue Lighting Control: Part 2

This is the second part of a two-part post.  The first part can be found here.

At the end of the first part, I had gotten discovery of the bridge implemented and had performed the pairing of the bridge.  In this part, I will show you how to create a query for the state of the light groups and control them.

Querying Group State

I’m only allowing the modification of the state of groups of lights on the Hue.  First I need to query the bridge for what states exist.  The list of groups and the state of the group are available at `http://${this.ipAddress}/${this.userName}/groups`. Here the data in this.userName is the user name that was returned from the Hue bridge in the pairing process.  With this information I am able to create a new UI element for each group found.  I only show groups of type “room” from this response.  It is also possible that the user has grouped an arbitrary set of lights together in a group.  I don’t show these.

var hueDB = (function () {
    var db = {};
    var datastore = null;
    var version = 1;
    db.open = function (callback) {
        var request = indexedDB.open('hueDB', version);
        request.onupgradeneeded = function (e) {
            var db = e.target.result;
            e.target.transaction.onerror = db.onerror;
            var store = db.createObjectStore('bridge', { keyPath: 'bridgeID' });
        };
        request.onsuccess = function (e) {
            datastore = e.target.result;
            callback();
        };
    };

    db.getBridgeList = function () {
        return new Promise((resolve, reject) => {
            var transaction = datastore.transaction(['bridge'], 'readonly');
            transaction.onerror = function (e) {
                reject(e.error);
            };
            transaction.oncomplete = function (e) {
                console.log('transaction complete');
            };

            var objStore = transaction.objectStore('bridge');
            objStore.getAll().onsuccess = function (e) {
                console.log('bridge retrieval complete');
                resolve(e.target.result);
            };

            var bridgeList = [];


        });
    };

    db.addBridge = function (bridge) {
        console.log('adding bridge ', bridge);
        return new Promise((resolve, reject) => {
            var transaction = datastore.transaction(['bridge'], 'readwrite');
            transaction.onerror = function (e) {
                reject(e.error);
            };
            transaction.onsuccess = function (e) {
                console.log('item added');
            };
            var objStore = transaction.objectStore('bridge');
            var objectStoreRequest = objStore.add(bridge);
            objectStoreRequest.onsuccess = function (e) {
                resolve();
            };
        });
    };

    return db;
})();

Changing the State of a Light Group Attributes

There are several elements of a light group’s state that can be modified.  I’m only considering two: the brightness of the light group and whether or not the group of lights is turned on.  Both can be set with a PUT request to the bridge at the the URL http://${this.ipAddress}/${this.userName}/groups/${id}/action`.  This endpoint accepts a JSON payload.  Turning a group of lights on or off; changing the brightness; activating a scene to change the color; and many other options can be changed through this end point.  It is not necessary to specify all of the possible attributes when calling this endpoint.  If an attribute is not specified it will remain at its current state.  I have made a method named setGroupState that will be used by all other methods that make use of this endpoint.  The methods will differ in the payloads that they build and pass to this method.

    setGroupState(groupName, state) {
        var id = this.groupToGroupID(groupName);
        var reqBodyString = JSON.stringify(state);
        return new Promise((resolve, reject) => {
            fetch(`http://${this.ipAddress}/api/${this.userName}/groups/${id}/action`, {
                method: "PUT",
                headers: { "Content-Type": "application/json" },
                body: reqBodyString
            })
                .then(resp => resp.json())
                .then(jsonResp => {
                    resolve(jsonResp);
                })
                .catch(err => reject(err));
        });
    }

Of the many attributes that could be packaged in the payload are bri and on.  The on state sets whether or not the lights are turned on.  The bri attribute accepts a value in the range of 0 to 254.  Note that a value of 0 doesn’t mean off.  Zero is the value level assigned to the lowest level of illumination above off that the light will provide.

Activating Scenes

Scenes, or a collection of settings that applies to lights, can by associated with a predefined light group or with some arbitrary group of lights.  The Hue API labels scenes as either LightScene or GroupScene accordingly.  I am only working with groups scenes.  A list of all of the scenes defined on the bridge is retrievable through the the endpoint http://${this.ipAddress}/api/${this.userName}/scenes.

The object returned is a dictionary of the scene IDs and the attributes.  The scene ID is a string of what appears to be random characters.  It’s not user friendly and should only be used internally by the code and never presented to the user.   Here is a response showing only two scenes.

{
    "8AuCtLbIiEJJRNB": {
        "name": "Nightlight",
        "type": "GroupScene",
        "group": "1",
        "lights": [
            "2"
        ],
        "owner": "rF0JJPywETzJue2G8hJCn2tQ1PaUVeXvgB0Gq62h",
        "recycle": false,
        "locked": true,
        "appdata": {
            "version": 1,
            "data": "5b09D_r01_d07"
        },
        "picture": "",
        "lastupdated": "2017-01-16T23:35:24",
        "version": 2
    },
    "7y-J6Qyzpez8c2R": {
        "name": "Dimmed",
        "type": "GroupScene",
        "group": "1",
        "lights": [
            "2"
        ],
        "owner": "rF0JJPywETzJue2G8hJCn2tQ1PaUVeXvgB0Gq62h",
        "recycle": false,
        "locked": false,
        "appdata": {
            "version": 1,
            "data": "Nmgno_r01_d06"
        },
        "picture": "",
        "lastupdated": "2017-01-16T23:35:24",
        "version": 2
    }
}

To activate a scene on a group I use the same endpoint that is used for turning light groups on and off or setting their brightness level.  The JSON payload will have a single element named scene whose value is one of the cryptic looking scene identifiers above.

    activateScene(sceneID) {
        var scene;
        if(sceneID in this.sceneList) {
            var scene = this.sceneList[sceneID];
            var group = scene.group;
            var req = {scene:sceneID};
            return this.setGroupState(group,req );            
        }
    }

Application Startup

To hide some of the events that occur at startup the application has a splash screen. The splash screen is only momentarily present. During the time that it is momentarily shown the application will attempt to reconnect to the last bridge that it had connected to and will query for the available groups and scenes. This is just enough of a distraction to hide the time taken to do this additional setup.

switch
The Application Splash Screen

Installing and Running the Application

If you have downloaded the source code to your local drive, you can add the program to Chrome as an unpacked extension. In a browser instance open the URL chrome://extensions.  In the upper-left corner of this UI is a button labeled Load Unpacked.  Select this option.

unpacket
UI for loading unpacked Chrome extensions

You will be prompted to select a folder.  Navigate to the folder where you have unpacked the source code and select it.  After selecting it you will see the application in the list of installed extensions.

loadedextension

The application will now show up in the Chrome app launcher.  This may be exposed through the regular app launcher that is part of your operating system (such as the Program menu on Windows) and will also appear in Chrome itself.  Close to the address bar is a button labeled “Apps.”

applauncher
The application in the Chrome app launcher

Completing the Application

As I mentioned in the opening,  this is not meant to be a complete application.  It is only an operational starting point, creating something that is functional enough to start testing different functions in the Hue API.

I will close with mentioning some other potential improvements.  For a user running the application for the first time the setup process might be smoothed out by automatically trying to pair with the first bridge seen (if there is only one bridge seen) and prompting the user to press the link button.  This makes the setup process a two step process: start the application and press the link button on the bridge.  There could also be other people that are operating the Hue lighting at the same time that this application is running.  Periodically polling the state of the lights and light groups on the network and updating the UI accordingly would improve usability.  A user may also want to control individual lights within a group or have control over the light color.  For this a light selection UI would also need to be developed.

It took me about an evening to get this far in the development and it was something enjoyable to do during a brief pause between projects.  As such projects go, I’m not sure when I’ll get a change to return to it.  But I hope that in it’s current form that it will be of utility to you.

-30-

 

Basic Hue Lighting Control: Part 1

screenshot
Screenshot of Chrome application for controlling Hue lighting.

Continuing from the post I made on SSDP discovery with Chrome, I’m making an application that will do more than just discovery. For this post I’m going to show the starting point of a Chrome application for controlling your home Hue lighting. I’ve divided this into two parts. In this first part I’m showing the process of pairing with the bridge. In the second part I’ll control the lights.

The features that this application will implement will include bridge discovery and pairing; the power state of the light; and the brightness level of the light. There’s many other features that could still be implemented.  Given the full range of capabilities that the Hue kits support (changing color, timers, response to motion sensors, etc.) this will not be an application that utilizes the full capability of the Hue lighting sets.

Chrome Only

This application is designed to only run in Chrome. If you want to adapt it to run outside of Chrome, you can do so by first disabling SSDP discovery. (Other HTML application platforms might not support UDP for discovery.)

The other discovery methods (querying Hue’s discovery web service or asking the user to enter the IP address) can still work. A non-chrome target will also need to allow CORS to be ignored and allow communication without SSL.

What is Hue Lighting?

Hue Lighting is an automated lighting solution made by Philips. Generally the lighting kits are sold in a package that contains three LED based light bulbs and a bridge. The bridge is a device that connects to your home network with an Ethernet jack and communicates with the light bulbs.

Philips also makes free applications for iOS and Android for controlling the lights. For any Hue light the light’s brightness and whether or not it is turned on can be controlled through the applications. Some lights also allow the color temperature to be changed (adjusting the tint between red, yellow and blue). Some lights support RGB (Red, Green, Blue) parameters so that their colors can be changed.  These settings can be individually adjusted or settings for a collection of the lights can be defined together as a “scene.” When a scene is activated the state of all of the lights that make up the scene are updated. Scenes can be activated through special light switches, through an app, through a schedule, or in response to a Hue motion sensor detecting motion.

Discovery: Review and New Methods

The central piece of hardware for the Hue lighting is the Hue Bridge. At the time of this writing there are two versions of the bridge. For the functionality that this application will utilize, the differences between the two bridges will not matter. The messaging and interaction to both versions of the bridge will be the same. My UI will properly represent the bridge that the system discovers. The first version of the Hue Bridge is round. The second version of the Hue Bridge is square. In either case we must first find the bridge’s IP address before we can begin interaction.

phillipsbridge
Phillips Hue Bridge Version 1 (left) and Version 2 (right)

The Hue bridge can be discovered in multiple ways. It can be discovered using SSDP. The basics of SSDP discovery were previously discussed here. Please refer back to it if you need more detail than what is found in this brief overview.  Devices that support SSDP discovery join a multicast group on the network that they are connected to. These devices generally wait for a request for discovery to be received. An SSDP request is sent as an HTTP over UDP message and every SSDP device that receives it responds with some basic information about itself and a URL to where more information on the device can be found. Examples of some devices that support SSDP are network attached storage; set top boxes like Android TVs and Rokus; printers; and home automation kits.

Two other methods of discovering a bridge include asking the user to enter an IP address and asking for a list of IP addresses of bridges on your network through the Hue discovery service.  If you have a Hue bridge connected to your network right now you can see it’s IP address by visiting https://discovery.meethue.com/ . If you are on a shared network then you may also see IP addresses of other bridges on your network. It is also possible that not all bridges on your network are reachable.  This method is much easier to implement than SSDP based discovery. But on a network for which there is no Internet connection (whether by design or from an outage) this method will not work. The SSDP method is only dependent on the local network.

function discoverBridge() { 
    discovredHueBridgeList = [];
    fetch(' https://discovery.meethue.com')
        .then(response => response.json())
    .then(function (hueBridgeList) {
        console.info(hueBridgeList);
        hueBridgeList.forEach((item)=> {
         // each item processed here has a bridge IP address
         // and serial number exposed through item.id and 
         // item.internalipaddress
       }
     );
}

Once I have a bridge IP address I attempt to query it for more information. If communication succeeds, then I show a representation of the bridge with an icon that matches the version of the bridge that the user has. The UI layout has two images ( one named hueBridgev1 and the other hueBridgev2) I show the appropriate image and hide the other.

Pairing

Now that the bridges have been discovered, it is up to the user to select one with which to pair. After the user selects a bridge, she is instructed to press the pairing button on the bridge. While this instruction is displayed the application is repeatedly attempting to request a new user ID name from the bridge. This should be viewed more as an access token. The Hue documentation uses the term “user name” but the actual value is what appears to be a random sequence of characters. To request a user name a JSON payload with one member named devicetype is posted to the bridge. The value assigned to devicetype matters little. It is recommended that it be a string that is unique to your application. The payload is posted to http://%5Byour bridge IP address]/api. A failure response will result. This is expected. The application must repeatedly make this request and prompt the user to press the link button on the bridge.  The request will fail until the pairing button on the bridge is passed.

function pairBridge(ipAddress) {
   console.info('attempting pairing with address ', ipAddress);
   var req = { devicetype: "hue.j2i.net#browser" };
   var reqStr = JSON.stringify(req);
   var tryCount = 0;
   return new Promise(function(resolve, reject)  {
      var tryInterval = setInterval(function () {
      console.log('attempt ', tryCount);
      ++tryCount;
      if (tryCount > 60) {
        clearInterval(tryInterval);
         reject();
         return;
      }
      fetch(`http://${ipAddress}/api`, {
         method: "POST",
         headers: {
            "Content-Type": "application/json"
         },
         body: reqStr
      })
      .then(function(response)  {
         console.log('text:',response);
         return response.json();
      })
      .then(function(data)  {
          console.log(data);
          if (data.length > 0) {
             var success = data[0].success;
             var error = data[0].error;
             if (success) {
                console.log('username:', success.username);
                var bridge = {
                   ipAddress: ipAddress,
                   username: success.username
                };
                clearInterval(tryInterval);
                 resolve(bridge);
                  return;
               }
               else if (error) {
                  if (error.type === 101) {
                     console.log('the user has not pressed the link button');
                  }
               }
            }
         });
      }, 2000);
   });
}

Once the button is pressed the bridge will respond to the first pairing request it receives with a user name that the application can use. This user name must be saved and used for calls to most of the functionality that is present in the bridge. I save the bridge’s serial number, IP address, and the name that must be used for the various API calls to an indexedDB object store. The access information for multiple paired bridges could be stored in the object store at once. But the application will only be able to communicate with one bridge at a time.

Continued in Part II

SSDP Discovery in HTML

While implementing a few projects I decided to implement them in HTML since it would work on the broadest range of my devices of interest. My projects of interest needed to discover additional devices that are connected to my home network. I used
SSDP for discovery.

SSDPDiscovery

SSDP (Simple Service Discover Protocol ) is a UDP based protocol that is a part of UPnP for finding other devices and services on a network. It’s implemented by a number of devices including network attached storage devices, Smart TVs, and home automation systems. There are a lot of these devices that expose functionality through JSON calls. You can easily make interfaces to control these devices. However, since the standards for HTML and JavaScript don’t include a UDP interface, how to perform discovery isn’t immediately obvious. Alternatives to SSDP include having the user manually enter the IP address of the device of interest or scanning the network. The latter of those options can raise some security flags when performed on some corporate networks.

For the most part, the solution to this is platform dependent. There are various HTML based solutions that do allow you to communicate over UDP. For example, the BrightSign HTML5 Players support UDP through the use of roDatagramSocket. Chrome makes UDP communication available through chrome.udp.sockets. Web pages don’t have access to this interface (for good reason, as there is otherwise potential for this to be abused). Although web apps don’t have access, Chrome extensions do. Chrome Extensions won’t work in other browsers. But at the time of this writing Chrome accounts for 67% of the browser market share and Microsoft has announced that they will use Chromium as the foundation for their Edge browser. While this UDP socket implementation isn’t available in a wide range of browsers, it is largely available to a wide range of users since this is the browser of choice for most desktop users.

To run HTML code as an extension there are two additional elements that are needed: a manifest and a background script. The background script will create a window and load the starting HTML into it.

chrome.app.runtime.onLaunched.addListener(function() {
    chrome.app.window.create('index.html', {
        'outerBounds': {
        'width': 600,
        'height': 800
        }
    });
});

I won’t go into a lot of detail about what is in the manifest, but I will highlight its most important elements. The manifest is in JSON format. The initial scripts to be run are defined app.background.scripts. Other important elements are the permission element, without which the attempts to communicate over UDP or join a multicast group will fail and the manifest_version element. The other elements are intuitive.

        {
            "name": "SSDP Browser",
            "version": "0.1",
            "manifest_version": 2,
            "minimum_chrome_version": "27",
            "description": "Discovers SSDP devices on the network",
            "app": {
              "background": {
                "scripts": [
                  "./scripts/background.js"
                ]
              }
            },
          
            "icons": {
                "128": "./images/j2i-128.jpeg",
                "64": "./images/j2i-64.jpeg",
                "32": "./images/j2i-32.jpeg"
            },
          
            "permissions": [
              "http://*/",
              "storage",
              {
                "socket": ["udp-send-to", "udp-bind", "udp-multicast-membership"]
              }
            ]
          }    

Google already has a wrapper available as a code example chrome.udp.sockets that was published for using Multicast on a network. In it’s unaltered form the Google code sample assumes that text is encoded in the 16-bit character encoding of Unicode. SSDP uses 8-bit ASCII encoding. I’ve taken Google’s class and have made a small change to it to use ASCII instead of Unicode.

To perform the SSDP search the following steps are performed.

  1. Create a UDP port and connect it to the multicast group 239.255.255.250
  2. Send out an M-SEARCH query on port 1900
  3. wait for incoming responses originating from port 1900 on other devices
  4. Parse the response
  5. Stop listening after some time

The first item is mostly handled by the Google Multicast class. We only need to pass the port and address to it. The M-SEARCH query is a string. As for the last item, it isn’t definitive when responses will stop coming in. Some devices appear to occasionally advertise themselves to the network even if not requested. In theory you could keep getting responses. At some time I’d suggest just no longer listening. Five to ten seconds is usually more than enough time. There are variations in the M-SEARCH parameters but the following can be used to ask for all devices. There are other queries that can be used to filter for devices with specific functionality. The following is the string that I used; what is not immediately visible, is that after the last line of text there are two blank lines.

M-SEARCH * HTTP/1.1
HOST: 239.255.255.250:1900
MAN: "ssdp:discover"
MX: 3
ST: ssdp:all
USER-AGENT: Joel's SSDP Implementation
    

When a response comes in, the function that we assign to MulticastScoket.onDiagram will be called with a byte array containing the response, the IP address from which the response came, and the port number from which the response was sent (which will be 1900 for our current application). In the following code sample, I initiate a search and print the responses to the JavaScript console.

const SSDP_ADDRESS = '239.255.255.250';
const SSDP_PORT = 1900;
const SSDP_REQUEST_PAYLOAD =    "M-SEARCH * HTTP/1.1\r\n"+
                                "HOST: 239.255.255.250:1900\r\n"+
                                "MAN: \"ssdp:discover\"\r\n"+
                                "MX: 3\r\n"+
                                "ST: ssdp:all\r\n"+
                                "USER-AGENT: Joel's SSDP Implementation\r\n\r\n";

var searchSocket = null;

function beginSSDPDiscovery() { 
    if (searchSocket)
        return;
    $('.responseList').empty();
    searchSocket = new MulticastSocket({address:SSDP_ADDRESS, port:SSDP_PORT});
    searchSocket.onDiagram = function(arrayBuffer, remote_address, remote_port) {
        console.log('response from ', remote_address, " ", remote_port);
        var msg = searchSocket.arrayBufferToString8(arrayBuffer);
        console.log(msg);        
    }
    searchSocket.connect({call:function(c) {
        console.log('connect result',c);
        searchSocket.sendDiagram(SSDP_REQUEST_PAYLOAD,{call:()=>{console.log('success')}});
        setTimeout(endSSDPDiscovery, 5000);
    }});    
}

Not that parsing the response strings is difficult, by any means it would be more convenient if the response were a JSON object. I’ve made a function that will do a quick transform on the response so I can work with it like any other JSON object.

function discoveryStringToDiscoveryDictionary(str) {
    var lines = str.split('\r');
    var retVal = {}
    lines.forEach((l) => {
        var del = l.indexOf(':');
        if(del>1) {
            var key = l.substring(0,del).trim().toLowerCase();
            var value = l.substring(del+1).trim();
            retVal[key]=value;
        }
    });
    return retVal;
}    

After going through this transformation a Roku Streaming Media Player on my network returned the following response. (I’ve altered the serial number)

{
    cache-control: "max-age=3600",
    device-group.roku.com: "D1E000C778BFF26AD000",
    ext: "",
    location: "http://192.168.1.163:8060/",
    server: "Roku UPnP/1.0 Roku/9.0.0",
    st: "roku:ecp",
    usn: "uuid:roku:ecp:1XX000000000",
    wakeup: "MAC=08:05:81:17:9d:6d;Timeout=10"    ,
}

Enough code has been shared for the sample to be used, but rather than rely on the development JavaScript console,  I’ll change the sample to show the responses in the UI. To keep it simple I’ve defined the HTML structure that I will use for each result as a child element of a div element of the class palette. This element is hidden, but for each response I’ll clone the div element of the class ssdpDevice; will change some of the child members; and append it to a visible section of the page.

        
 <html>
    <head>
        <link rel="stylesheet" href="styles/style.css" />
        http://./scripts/jquery-3.3.1.min.js
        http://./scripts/MulticastSocket.js
        http://./scripts/app.js
    </head>
    <body>
Scan Network

 

</div>

address:
location:
server:
search target:

</div> </div>

</body> </html>

 

The altered function for that will now display the SSDP responses in the HTML is the following.

        function beginSSDPDiscovery() { 
            if (searchSocket)
                return;
            $('.responseList').empty();
            searchSocket = new MulticastSocket({address:SSDP_ADDRESS, port:SSDP_PORT});
            searchSocket.onDiagram = function(arrayBuffer, remote_address, remote_port) {
                console.log('response from ', remote_address, " ", remote_port);
                var msg = searchSocket.arrayBufferToString8(arrayBuffer);
                console.log(msg);
                discoveryData = discoveryStringToDiscoveryDictionary(msg);
                console.log(discoveryData);
        
                var template = $('.palette').find('.ssdpDevice').clone();
                $(template).find('.ipAddress').text(remote_address);
                $(template).find('.location').text(discoveryData.location);
                $(template).find('.server').text(discoveryData.server);
                $(template).find('.searchTarget').text(discoveryData.st)
                $('.responseList').append(template);
            }
            searchSocket.connect({call:function(c) {
                console.log('connect result',c);
                searchSocket.sendDiagram(SSDP_REQUEST_PAYLOAD,{call:()=>{console.log('success')}});
                setTimeout(endSSDPDiscovery, 5000);
            }});    
        }