The local file system Preview

These functions act on the Connector host's own disk, not on a virtual file system. They see whatever the operating system shows the Connector service account, which is usually the whole machine, and they know nothing about SFTP.cloud users, virtual file system roots or encryption at rest.

These functions are not scoped to anything

A path here is an operating system path. /etc, C:\Windows, a mounted network share: if the service account can reach it, a script can read it, and a script can usually delete it. There is no root to escape from because there is no root.

To touch the files your users actually transfer, use the virtual file system object. Use these functions for staging areas, temporary files and handing data to other software on the host.

Almost none of this works in the Docker image

The published Connector container is a distroless image: no shell, no utilities, and a filesystem that holds the Connector binary and the /data volume and essentially nothing else. /tmp may not exist and GetTempFileName() may return a path you cannot write. If a script needs local scratch space in a container, use a path under /data on your mounted volume, and remember that the process runs as the unprivileged nonroot user. See Run in Docker.

Listing

function ListDir(path: string, mask?: string): DirListItem[];
function ListDirR(path: string, mask?: string): DirListItem[];

ListDirR recurses into subfolders; ListDir does not. mask is an optional glob such as *.csv. Both return an array of DirListItem.

var files = ListDir("/var/inbox", "*.csv");
if (Array.isArray(files)) {
    files.forEach(function (item) {
        Log.Info(item.Name + " " + item.Size + " bytes");
    });
}

Name here is the full path, unlike Stat

ListDir and ListDirR set Name to the fully qualified path, for example /var/inbox/report.csv. StatFileSystemObject sets Name to the bare name, for example report.csv. It is an easy thing to trip over when you feed one into code written for the other. Use ExtractName() and ExtractPath() to get whichever half you meant.

Existence, type and size

function FileExists(path: string): boolean;   // false for a directory
function DirExists(path: string): boolean;    // false for a file
function FileSizeOf(path: string): number;    // bytes, or -1 if it cannot be stat'd
function FileAgeSecs(path: string): number;   // seconds since last modified, or -1
function FileType(path: string): string;      // MIME type from the first 261 bytes, "" if unknown
function StatFileSystemObject(path: string): DirListItem;

FileType sniffs content rather than trusting the extension, which makes it the right tool for catching a renamed file:

var local = "/var/scan/" + ShortUID();
GetCurrentVFS().ExportFile(CtxRelPath(), local);

var mime = FileType(local);
DelFile(local);

if (mime === "application/x-msdownload" || mime === "application/x-executable") {
    Log.Error("executable content uploaded as " + CtxRelPath() + " by " + CtxUsername());
    Exit(1);
}

It recognizes the common image, video, audio, archive, document and font formats. Plain text, CSV, JSON and XML have no magic bytes, so they come back as an empty string; an empty result means "unrecognized", never "not a file".

StatFileSystemObject returns an empty DirListItem when the path does not exist, so check info.Name !== "" rather than expecting an error.

Creating, copying, moving, deleting

function MakeDir(path: string): boolean;
function CopyFile(what: string, toDirectory: string): boolean;
function MoveFile(what: string, toPath: string): boolean;
function DelFile(path: string): boolean;
function DelDir(path: string): boolean;        // must be empty
function DelTree(path: string): boolean;       // recursive
function Chown(path: string, user: string): boolean;

Every one returns true or false and never throws, so an unchecked call fails silently.

Function	Watch out for
CopyFile	The second argument is a directory, not a file path, and it must already exist
CopyFile	It will not overwrite. An existing file with the same name means false
MoveFile	The second argument is a full file path, and it must not already exist
DelDir	Fails on a non empty directory. DelTree is the recursive one
Chown	POSIX only, takes a user name, and needs the Connector to be running as root. Always false on Windows

if (!CopyFile("/var/inbox/report.csv", "/var/archive")) {
    Log.Error("copy failed, likely because the destination already has that name");
}

Reading and writing text

function ReadTextFile(path: string): string;              // "" on any failure
function WriteTextToFile(path: string, text: string): boolean;   // replaces
function AppendTextToFile(path: string, text: string): boolean;  // appends
function ReadFileAsHex(path: string, offset: number, count: number): string;
function GetTempFileName(): string;

Both write functions create the file if it does not exist. Escapes such as \n and \t behave the way you expect.

var tmp = GetTempFileName();
WriteTextToFile(tmp, JSON.stringify(EventCtx()));
// ... hand tmp to something ...
DelFile(tmp);

GetTempFileName() returns a path in the operating system's temporary directory. It does not create the file, and it does not clean up after you. Delete what you create.

ReadTextFile returns "" for a missing file and for an empty one

There is no way to tell those two apart from the return value. When the difference matters, call FileExists() first.

Hashing a file

function HashFile(algorithm: string, path: string): string;

Streams the file, so size is not a problem. Algorithms are the same set as HashString: md5, sha1, sha256, sha384, sha512, sha3256, sha3384, sha3512. Returns lowercase hex, or an empty string on failure.

var digest = HashFile("sha256", "/var/outbox/delivery.zip");
if (digest !== "") {
    WriteTextToFile("/var/outbox/delivery.zip.sha256", digest + "  delivery.zip\n");
}

Waiting for a file

function WaitForFile(path: string, timeoutMs: number, pollIntervalMs?: number): boolean;

Blocks until the file exists or the timeout expires. The poll interval defaults to 500 ms.

if (!WaitForFile("/var/inbox/daily_export.csv", 60000, 1000)) {
    Log.Warn("the export did not arrive in time");
    Exit(0);
}

Existing does not mean finished

A file appears the moment something creates it, which is usually before it has finished writing it. WaitForFile returning true tells you the name exists and nothing more. To wait for it to settle, watch the modification time stop moving:

if (WaitForFile("/var/inbox/upload.dat", 30000)) {
    while (FileAgeSecs("/var/inbox/upload.dat") < 10) {
        Sleep(1000);
    }
    // Now it has been quiet for ten seconds.
}

Both the wait and the sleep spend the handler's timeout, so a loop like this belongs on a background after handler with a generous timeout, never on a before handler.

Secure deletion

function SecureErase(path: string, passes?: number): boolean;

Overwrites the file with cryptographically secure random data before unlinking it. passes defaults to 1.

SecureErase("/var/scan/plaintext.tmp", 3);

On modern storage this is best effort, not a guarantee

Overwriting a file in place assumes the storage rewrites the same physical blocks. SSDs with wear levelling, copy on write filesystems, thin provisioned volumes, snapshots and network storage all break that assumption, and the old blocks may survive. SecureErase is a meaningful improvement over DelFile and it is not a guarantee of unrecoverability. If content must never persist, do not write it to local disk in the first place.

Compression

function Zip(what: string, archive: string, password?: string, method?: string): boolean;
function Unzip(archive: string, toDirectory: string, password?: string): boolean;
function GzipFile(path: string, destPath?: string): boolean;
function GunzipFile(path: string, destPath?: string): boolean;

Zip accepts a wildcard in what and replaces the archive if it already exists.

method	Algorithm
deflate	Default
bzip2, zstd, xz	Alternatives
store	No compression

method is ignored when a password is set; encrypted archives are always deflate. Do not pass both.

Zip("/var/outbox/*.csv", "/var/staging/export.zip");
Zip("/var/outbox/*.csv", "/var/staging/export.zip", zipPassword);      // encrypted
Zip("/var/outbox/*.csv", "/var/staging/export.zip", "", "zstd");       // explicit method

Unzip needs the destination directory to exist. It refuses archives whose entries would escape that directory, the "zip slip" attack, by returning false and stopping partway.

GzipFile compresses a single file, defaulting to path + ".gz". GunzipFile reverses it, stripping .gz when present and otherwise appending .out so it never overwrites its own source.

Zip and Unzip build the whole archive on disk

Compressing a large tree takes proportional time and space, and it counts against the handler timeout. Do it on a background after handler.

Splitting

function SplitFileByLines(path: string, linesPerChunk: number, outputDir: string): number;
function SplitFileBySize(path: string, bytesPerChunk: number, outputDir: string): number;

Both return the number of chunks created, or -1 on failure. Chunks are named after the source with a zero padded counter before the extension: report_0001.csv, report_0002.csv.

var chunks = SplitFileByLines("/var/data/big_report.csv", 1000, "/var/split");
if (chunks < 0) {
    Log.Error("split failed");
} else {
    Log.Info("created " + chunks + " chunks");
}

Use SplitFileByLines for text and SplitFileBySize for anything binary.

Prefer the virtual file system where you can

These functions exist and are supported, and they are the right answer for staging areas and for feeding external tools. But for the files your users transfer, the virtual file system object is better in every way: it works across every storage backend, it handles encryption at rest, it returns errors you can read instead of a bare false, and it does not depend on the Connector host having a usable local disk at all.