Encoding, hashing and data Preview

Base64

function Base64Encode(text: string): string;
function Base64Decode(encoded: string): string;

Standard Base64 as defined by RFC 4648: the A-Z a-z 0-9 + / alphabet with = padding. Base64Decode returns an empty string when the input is not valid Base64.

Base64Encode("Hello, World!");        // "SGVsbG8sIFdvcmxkIQ=="
Base64Decode("SGVsbG8sIFdvcmxkIQ=="); // "Hello, World!"

A practical use, building an authorization header:

var credentials = Base64Encode("alice:" + apiPassword);
var hc = new HttpCli();
var res = hc.Url("https://api.example.com/data")
            .Header("Authorization", "Basic " + credentials)
            .Get();

Hashing a string

function HashString(algorithm: string, text: string): string;

Returns a lowercase hexadecimal digest. To hash a file on disk use HashFile instead, which streams it rather than holding it in memory.

Value	Algorithm
md5	MD5. Not for security use
sha1	SHA-1. Not for security use
sha256	SHA-256
sha384	SHA-384
sha512	SHA-512
sha3256	SHA3-256
sha3384	SHA3-384
sha3512	SHA3-512

An unrecognized name falls back to SHA-256 and writes a debug log line, so a typo produces a wrong digest rather than an error. Spell the algorithm carefully.

HashString("sha256", "Hello, World!");
// dffd6021bb2bd5b0af676290809ec3a53191dd81c7f70a4b28688a362182986d

HMAC signatures

function HmacSign(algorithm: string, key: string, data: string): string;
function HmacVerify(algorithm: string, key: string, data: string, signature: string): boolean;

An HMAC proves both integrity and authenticity: only a holder of the shared key can produce a valid one. algorithm is sha256 or sha512. HmacSign returns lowercase hex; HmacVerify compares in constant time, so it does not leak the correct value through timing.

var mac = HmacSign("sha256", signingKey, payload);

if (!HmacVerify("sha256", signingKey, payload, receivedSignature)) {
    Log.Error("signature mismatch, refusing");
    Exit(1);
}

Do not paste an HMAC key into a script

Put it in the secret store and read it with GetSecret(), so the key is not sitting in the script source and in every backup of it:

var key = GetSecret("webhook-signing-key");
var sig = HMACSHA256(payload, key);

GetSecret returns an empty string for a name that is not stored, and an empty HMAC key produces a signature that verifies against nothing, so check it before you use it.

CSV

function ParseCSV(text: string, delimiter?: string): string[][];
function FormatCSV(rows: string[][], delimiter?: string): string;

RFC 4180 handling in both directions: quoted fields, embedded delimiters and embedded newlines are all dealt with. The delimiter defaults to a comma; pass one character to change it. ParseCSV returns null when the input cannot be parsed.

var rows = ParseCSV(ReadTextFile("/var/inbox/data.csv"));
if (rows === null) {
    Log.Error("could not parse the CSV");
    Exit(1);
}

rows[0].push("Processed");
for (var i = 1; i < rows.length; i++) {
    rows[i].push("yes");
}

WriteTextToFile("/var/outbox/data.csv", FormatCSV(rows));

Tab separated data is the same function with a different delimiter:

var rows = ParseCSV(text, "\t");

FormatCSV quotes a field automatically when it contains the delimiter, a double quote or a newline.

XML

function ParseXML(text: string): object | null;

Parses XML into a nested object, or returns null when the input is not valid XML.

XML	In the object
Child element <foo>	Property foo
Several <foo> siblings	Array at property foo
Attribute bar="v"	_attrs.bar on that element's object
Text content	_text on that element's object

<order id="42">
  <customer name="Alice" />
  <item sku="ABC-1">Widget A</item>
  <item sku="ABC-2">Widget B</item>
</order>

var doc = ParseXML(ReadTextFile("/var/inbox/order.xml"));
if (doc === null) {
    Log.Error("XML parse failed");
    Exit(1);
}

var orderId = doc["_attrs"]["id"];      // "42"
var items = doc["item"];                 // an array, because there are two <item> elements
Log.Info("order " + orderId + " has " + items.length + " items");
items.forEach(function (it) {
    Log.Info("  " + it["_attrs"]["sku"] + " " + it["_text"]);
});

One element is not an array

An element that appears once is an object; the same element appearing twice is an array of objects. A document that usually has several <item> elements but occasionally has one will break a script that assumes items.length. Normalize first:

var items = doc["item"] || [];
if (!Array.isArray(items)) { items = [items]; }

ParseXML reads the whole document into memory. For very large files, shell out to a streaming tool with RunCapture.

TCP reachability

function TcpConnect(host: string, port: number, timeoutMs?: number): boolean;

Opens a TCP connection and closes it immediately, exchanging no data. Returns true if it connected within the timeout, which defaults to 5000 ms. IPv6 addresses work.

if (!TcpConnect("sftp.partner.com", 22, 2000)) {
    Log.Warn("partner SFTP unreachable, skipping the transfer this time");
    Exit(0);
}

A reachability check is not a health check

true means a TCP handshake completed. It says nothing about whether the service behind the port is working, whether your credentials are valid, or whether it will still be there a second later. It is useful for failing fast and for a clearer log line, not for deciding that a transfer will succeed. Any real work still needs its own error handling.