Transfers from a script Preview

A script running on a Connector can ask your Head to copy, move or sync files between any two of your storages, including storages served by another Connector. The Connector never moves the bytes itself and never talks to another Connector: it submits a transfer job to the Head, which runs it exactly like a copy or move started in the web client, and reports back.

The automation identity

A job runs as a user. Not the person who uploaded a file, and not the Connector: a transfer user your administrator has marked as an automation identity in the customer portal, under the user's settings. That user's permissions on every Connector decide what the job may read and where it may write, and its namespace (the folders it sees) is where the paths in your script live.

This is deliberate. A Connector holds your data, but it should not be able to act as any of your people; and a script should not need a password to move a file. The automation identity is the one account you designate for this, and you grant it exactly what your automation needs.

The identity never blocks anyone

Marking a user as an automation identity changes nothing about how that user signs in. It only allows your Connectors to run transfers as that user.

The functions

Call	Returns
SubmitTransfer(spec)	The job's first snapshot, once the Head accepted it
WaitTransfer(id, timeoutMs)	The snapshot when the job finished, or when the timeout ran out
GetTransfer(id)	The current snapshot, or null for an unknown id
CancelTransfer(id)	The snapshot after asking the Head to cancel

spec is an object:

Field	Meaning
as	The automation identity's username, for example archive@acme
op	copy (the default), move, or sync (one way, see Keeping a folder in sync)
src	The file or folder to transfer, as a path in the identity's namespace. For sync, the folder that is the truth
dst	The folder it lands in, as a path in the identity's namespace. For sync, the folder made to match src (its counterpart, not its container)
mode	sync only: update (the default) copies what is missing or changed and never deletes; mirror also deletes what the destination has and the source does not
conflict	sync only: what to do with a file that differs on both sides: newer (the default), skip, keep, larger or src
dryRun	sync only: compare and report, move nothing
maxDelete	sync only: refuse the run if a mirror would delete more entries than this; absent means no ceiling
allowEmptySource	sync only: let a mirror run from a source that lists empty over a destination that does not (refused otherwise)
head	The Head to ask, by id. Needed only while a Connector is bound to two Heads during a move; otherwise omit it

A snapshot is a plain object: id, headId, remoteJobId, asUser, op, src, dst, triggeredBy, state, bytesDone, bytesTotal, filesDone, filesTotal, currentPath, error, createdAt, finishedAt, and terminal, which is true once nothing will change. A sync also carries mode, conflict, dryRun, phase (scanning or applying while it runs), plannedCopies, plannedDeletes, copied, deleted, skipped, conflicts, failed, timesPreserved and warning. state is one of queued, running, done, failed, canceled, interrupted (the Head was upgraded mid job), or lost (the link to the Head stayed down too long to learn the outcome).

Every failure throws: a user that is not an automation identity, a Head that is draining, a job that exceeds the Head's limits, a link that is down. Catch what you want to handle.

A nightly copy to the archive

// Runs on a schedule. The archive storage lives on another Connector, mounted at /archive in
// the automation identity's namespace; this Connector's own storage is at its root.
var t = SubmitTransfer({ as: "archive@acme", op: "move", src: "/outgoing/" + Param("folder"), dst: "/archive/" + FormatDateTime("YYYY-MM") });
Log.Info("submitted " + t.remoteJobId);
var done = WaitTransfer(t.id, 3600000);
if (done.state !== "done") {
    throw new Error("archive move ended " + done.state + ": " + done.error);
}
Log.Info("moved " + done.filesDone + " files, " + done.bytesDone + " bytes");

Reacting to an upload

// Bound to after.upload: replicate every file that lands in /inbound to the second site.
if (CtxRelPath().indexOf("/inbound/") === 0) {
    SubmitTransfer({ as: "replica@acme", src: CtxRelPath(), dst: "/mirror/inbound" });
}

Two things to remember in an event handler:

• Do not wait there. A before handler holds the operation while it runs; an after handler holds the response. Submit and return; watch the outcome under Automation, then Transfers, or in a scheduled script.
• Mind the loop. A transfer submitted by automation is a real data movement, so it fires the same events as anyone's, including after.upload at the destination and before.transfer on its source. A handler that reacts to those by submitting more transfers should check CtxUsername() and ignore its own automation identity, or it will keep triggering on its own work.

Keeping a folder in sync Preview

op: "sync" makes one folder match another, one way: src is the truth and dst is changed until it looks the same. Nothing is remembered between runs; each run compares the two folders afresh and does only what the difference calls for. The two folders may sit on one storage, on two storages of one Connector, or on two Connectors; the Head picks the placement of every copy exactly as it does for copy, and every delete goes through the destination's Connector, which enforces it like any other operation.

One difference from copy and move

For copy and move, dst is the folder the source lands in. For sync, dst is the folder that ends up matching src: syncing /outgoing to /archive/outgoing fills /archive/outgoing with the contents of /outgoing. Neither folder may be the root of the namespace, and neither may sit inside the other.

Modes

• update (the default) copies files the destination lacks or holds differently and creates missing folders. It never deletes anything.
• mirror does the same and then removes what the destination has and the source does not, files first and emptied folders after. A mirror whose source lists empty while the destination does not is refused (an unmounted disk or a mistyped path must not wipe the other side); pass allowEmptySource: true when that is what you mean. maxDelete refuses the whole run before anything moves when the plan would delete more entries than you allow.

How files are compared

Two files are the same when their sizes match and their modification times agree within two seconds. When the sizes match but the times differ, the content decides where both storages can compute a checksum (local disks can; SFTP and object storage cannot); where they cannot, the copy is made when the destination keeps file times, and otherwise equal sizes are trusted as unchanged and the snapshot carries a warning saying so. Every copy keeps the source's modification time on the destination where the storage allows it, which is what makes the next run cheap: object storage does not keep times, and a destination the identity may not edit metadata on does not either.

Conflict rules

A conflict is a file that exists on both sides with different content.

conflict	What happens
newer	The source copy replaces the destination when it is newer; a newer destination is left alone and counted
skip	The destination is left alone and counted
keep	The destination copy is renamed aside as name.conflict-<date>-<time>-<id>.ext, then the source copy lands; kept copies are never removed by a later mirror
larger	The source copy replaces the destination when it is larger; otherwise the destination is left alone and counted
src	The source copy always replaces the destination

A name that is a file on one side and a folder on the other follows the same rule: under mirror the source wins (except with skip, and keep sets the destination aside); under update it is counted and left alone, because update never deletes.

Names that differ only by letter case are treated as the same name, so a Windows or macOS destination is never asked to copy A.txt and delete a.txt; such pairs are counted as conflicts and left alone.

What the identity needs

On the source: list and download. On the destination: list, upload and create folders; mirror adds delete for files and folders; keep adds rename; keeping file times, which makes re runs cheap, adds edit metadata. See Permissions.

Events, scripts and partial runs

A sync is real data movement: every copy fires the same events as a copy job and every delete fires the delete events, on the Connectors involved and as the automation identity. A dirList.filter handler that hides entries from the automation identity hides them from the comparison too, so such handlers must exempt it (check CtxUsername()). A run that is canceled, or interrupted by a Head upgrade, leaves the destination part way; deletes only ever run after every copy succeeded, and the next run converges.

// Rehearse first: the counts say what a mirror would do, and nothing moves.
var plan = WaitTransfer(SubmitTransfer({ as: "archive@acme", op: "sync", mode: "mirror", src: "/outgoing", dst: "/archive/outgoing", dryRun: true }).id, 600000);
Log.Info("would copy " + plan.plannedCopies + " and delete " + plan.plannedDeletes);
if (plan.plannedDeletes > 50) {
    throw new Error("too many deletions planned; somebody should look");
}
// Then the real run, with the same ceiling as a belt.
var t = SubmitTransfer({ as: "archive@acme", op: "sync", mode: "mirror", conflict: "newer", src: "/outgoing", dst: "/archive/outgoing", maxDelete: 50 });
var done = WaitTransfer(t.id, 3600000);
if (done.state !== "done") {
    throw new Error("sync ended " + done.state + ": " + done.error);
}
Log.Info("copied " + done.copied + ", deleted " + done.deleted + ", conflicts " + done.conflicts + (done.warning ? " (" + done.warning + ")" : ""));

Two way synchronization, where both sides may change and deletions travel in both directions, is a different thing: it needs a record of the previous run to tell a new file from a deleted one, and it is not offered.

Where the paths live

src and dst are paths in the automation identity's namespace, not in the script's own storage. If the identity sees this Connector's storage at / and the other site's at /archive, then src: "/reports/q1.csv" is a file here and dst: "/archive/2026" is a folder there. The same call works between two folders of one storage, between two storages on one Connector, and between two Connectors; the Head picks the placement, and the bytes stay inside your network whenever both ends are on the same Connector.

Related

• Transfers, the console page where these jobs are listed and canceled.
• Events and when they fire, for the transfer events.
• Schedules, for running a transfer script on a timetable.