Resource and Protocol Integration Guide

This is a testable integration guide covering credential selection, HTTP, MQTT, MQTT over WebSocket, EarthMQTT WebSocket, WebTransport, and the interaction between space, group, scope, and reserved $… topics.

Before testing, open Connect → EarthMQTT Connection and copy the endpoints shown by your deployment. Examples use these placeholders:

Do not copy angle brackets into real configuration.

1. Six concepts to understand first

- is the built-in default space in every domain. If no other space exists, all protocols can begin with it.

scope is not a topic prefix. It is the access ceiling granted to an application credential:

Access narrows at every layer: credential scope sets the ceiling, the connection username selects a domain and space, and the subscription filter selects topics. A topic cannot bypass credential authorization.

2. Application keys and resource keys

Both credential types reveal the plaintext secret once. A lost secret cannot be recovered; create or rotate it.

A new resource key has the form erk_v2.<identity>.<secret>. The identity segment carries workspace and resource IDs so clients do not repeat them. It is base64url encoding, not encryption. Authentication still verifies the hash of the entire key or the factory-seed HMAC, so never truncate, edit, partially store, or log it.

Security guidance:

1. Give every directly connected device its own resource key. Never copy one erk_… to multiple devices.
2. Use a manage application key for automated registration, but give each device only the returned resource key.
3. Give dashboards and message processors only data, narrowed to the required spaces.
4. Rotate a suspected compromised resource key from device details. The old key and sessions become invalid immediately.

3. Create a resource and obtain its unique key

3.1 Create in the console

Open Credentials → Device or resource credentials → Add a device or resource, enter a resource ID, and select Create and generate device key. The erk_… value is shown once.

Resource creation and key generation are one operation. A client must not treat a response without apiKey as successful creation.

3.2 Create through the API

Create an application key with data and manage, then call:

curl -sS -X POST '<BASE_URL>/v1/resources' \
  -H 'Authorization: Bearer <MANAGE_ETK>' \
  -H 'Content-Type: application/json' \
  -d '{
    "tenantId": "<TENANT_ID>",
    "resourceId": "<RESOURCE_ID>",
    "name": "Warehouse sensor 01",
    "spaces": ["-", "production"],
    "metadata": {
      "type": "sensor",
      "site": "Warehouse A",
      "lat": 52.52,
      "lon": 13.405
    }
  }'

HTTP 201 returns a structure like:

{
  "resource": {
    "tenantId": "acme",
    "resourceId": "sensor-01",
    "status": "registered",
    "credentialConfigured": true
  },
  "apiKey": "erk_SHOWN_ONCE"
}

Calling POST /v1/resources again with the same resourceId registers a new key and invalidates the old one. This is key rotation, not an ordinary metadata update.

4. Choosing a protocol

Prefer https://, mqtts://, and wss:// in production. Test mqtt:// or ws:// only when the console explicitly advertises a plaintext endpoint, and never expose plaintext credentials on the public Internet.

5. HTTPS REST

5.1 Report with a resource key

curl -i -X POST '<BASE_URL>/v1/telemetry' \
  -H 'Authorization: Bearer <RESOURCE_ERK>' \
  -H 'Content-Type: application/json' \
  -d '{
    "eventId": "sensor-01-000001",
    "timestamp": 1789948800000,
    "data": {"temperature":23.6,"humidity":48,"online":true}
  }'

HTTP 202 means accepted durably. A newly issued erk_v2.… carries workspace and resource identity, so neither headers nor the body repeat them. eventId makes retries idempotent. When timestamp is absent, the server receive time is used. Legacy erk_… keys remain valid with their original two identity headers; rotate once to receive the new format.

5.2 Report for multiple resources with an application key

An application key does not use X-Resource-Id; specify an already registered resource in the body:

curl -sS -X POST '<BASE_URL>/v1/telemetry' \
  -H 'Authorization: Bearer <DATA_ETK>' \
  -H 'Content-Type: application/json' \
  -d '{
    "tenantId": "<TENANT_ID>",
    "resourceId": "<RESOURCE_ID>",
    "eventId": "gateway-000001",
    "topic": "plant/line-a/state",
    "data": {"temperature":23.8,"online":true}
  }'

Use this when one edge application reports for multiple devices. Do not burn the application key into every device as a shared device secret.

5.3 Query events and issue commands

curl -sS -H 'Authorization: Bearer <DATA_ETK>' \
  '<BASE_URL>/v1/resources/<RESOURCE_ID>/events?tenantId=<TENANT_ID>&afterSequence=0&limit=10'

curl -sS -X POST \
  '<BASE_URL>/v1/resources/<RESOURCE_ID>/commands?tenantId=<TENANT_ID>' \
  -H 'Authorization: Bearer <DATA_ETK>' \
  -H 'Content-Type: application/json' \
  -d '{"eventId":"command-000001","command":{"action":"setInterval","seconds":30}}'

A command goes to the resource's $earth/message by default, as an envelope with eventId and sequence and the command itself under command. For a device that listens on its own topic, add topic to the request (for example "topic":"cmd/light"; no wildcards, no leading $); on that topic it receives just the command object by default ("format":"raw"), as its own protocol would carry it, and "format":"envelope" asks for the envelope with eventId for de-duplication. $earth/resume replays keep the same topic and format. On the globe, "Device controls → Configure controls → Sending" fixes both for a device type or a single device; {{resourceId}} (or {{tenantId}}) in its topic becomes the controlled device's own, so one type profile with cmd/{{resourceId}}/light reaches lamp-1 on cmd/lamp-1/light. scope sets who receives it:

• broadcast: every subscriber of the topic, exactly as an ordinary MQTT publish. It is the default once you name a topic of your own, which suits unmodified legacy devices, including devices that share one application key.
• directed: only sessions connected with that device's own resource credential (username acme/<RESOURCE_ID>); other subscribers of the same topic do not get it. $earth/message is directed only.

Either way the command is recorded on the target device's own stream, so that device can catch up with $earth/resume; other broadcast subscribers are not replayed to. The response's reach says whether it can arrive: with reachable: false, reason explains, for example shared-credential (the device reports with a shared key, so no session is its own for a directed command), connector-commands-off (the device comes from an external broker whose connector does not allow commands) or connector-needs-topic (it comes from an external broker but no topic was named). For devices from an external broker, once the connector allows commands they are published back to the customer's broker on the topic and in the format given.

202 means the command was recorded, not that the device executed it. Commands remain while a resource is offline and can be caught up with $earth/resume after MQTT reconnection.

REST telemetry and resource telemetry over EarthMQTT WebSocket or WebTransport enter default space -. Use an MQTT username to publish directly into another space.

6. MQTT over TLS

The resource key is the device identity, so the server ignores Client ID differences for resource connections. Each resource key keeps one MQTT connection and a later connection always takes over. MQTT username must be the credential's own tenant ID; there is no shared username for all devices. The resource ID is no longer repeated.

An application key also carries its workspace identity, so the default space does not require the workspace ID again. MQTT requires the Username Flag whenever a password is present: use -u '' with Mosquitto or username: '' with MQTT.js. The empty string is a protocol compatibility field, not an account name. Use @production for a named space. acme and acme@production remain supported and are checked as workspace claims against the key. Omitting Client ID selects an ephemeral clean session and lets the client library or broker generate the ID. Set a stable unique Client ID only for durable subscriptions and offline QoS queues. The same explicit Client ID inside one application key takes over the older connection; different application keys remain isolated.

6.1 Minimal publish and subscribe test

mosquitto_pub \
  -h '<MQTT_HOST>' -p '<MQTT_PORT>' --tls-use-os-certs \
  -i 'earthmqtt-common-clientid' -u '<TENANT_ID>' -P '<RESOURCE_ERK>' \
  -q 1 -t 'plant/line-a/state' \
  -m '{"temperature":23.6,"humidity":48,"online":true}'

Subscribe with an application key:

mosquitto_sub \
  -h '<MQTT_HOST>' -p '<MQTT_PORT>' --tls-use-os-certs \
  -u '' -P '<DATA_ETK>' \
  -q 1 -t 'plant/+/state' -v

The subscriber should receive plant/line-a/state. Do not add workspace, resource, or space prefixes to ordinary topics. A plain payload may be any JSON value; numeric 23.6 is stored as {"value":23.6}. Only an {eventId,data} envelope is deduplicated by business event ID.

An MQTT.js application connection may omit Client ID. The library usually creates an ephemeral ID; an MQTT 3.1.1 clean-session or MQTT 5 client capable of sending a zero-length Client ID may instead let the broker assign it:

const client = mqtt.connect('<MQTT_URL>', {
  username: '',
  password: process.env.EARTHMQTT_APPLICATION_KEY,
  clean: true,
});

# subscribes only to the primary space. Publish topics cannot contain + or #. Quote $… topics with single quotes in shell commands.

7. MQTT over WebSocket

This is still MQTT. Topics, username, password, Client ID, QoS, spaces, and $… rules are identical to MQTT over TLS. Only the URL changes to wss://…/mqtt, with WebSocket subprotocol mqtt.

import mqtt from 'mqtt';
import { randomUUID } from 'node:crypto';

const client = mqtt.connect('<MQTT_WS_URL>', {
  clientId: 'earthmqtt-common-clientid',
  username: '<TENANT_ID>',
  password: process.env.RESOURCE_API_KEY,
  reconnectPeriod: 1000,
});

client.on('connect', async () => {
  await client.subscribeAsync('$earth/message', { qos: 1 });
  await client.publishAsync('plant/line-a/state', JSON.stringify({
    eventId: randomUUID(), timestamp: Date.now(),
    data: { temperature: 23.7, online: true },
  }), { qos: 1 });
});
client.on('message', (topic, payload) => console.log(topic, payload.toString()));

npm install mqtt
RESOURCE_API_KEY='<RESOURCE_ERK>' node mqtt-ws-test.mjs

Browsers can use MQTT.js or another MQTT-over-WebSocket library. Never hard-code long-lived keys in a public web page. A signed-in first-party page should request a short-lived one-time ticket.

8. EarthMQTT WebSocket

/v1/ws is a JSON real-time protocol, not MQTT over WebSocket.

Resource report:

{"kind":"telemetry","eventId":"ws-000001","timestamp":1789948800000,"data":{"temperature":23.8,"online":true}}

Node.js test:

import WebSocket from 'ws';
import { randomUUID } from 'node:crypto';

const socket = new WebSocket('<WS_URL>', { headers: {
  authorization: `Bearer ${process.env.RESOURCE_API_KEY}`,
}});
socket.on('open', () => {
  socket.send(JSON.stringify({kind:'telemetry', eventId:randomUUID(), timestamp:Date.now(), data:{temperature:23.8, online:true}}));
  socket.send(JSON.stringify({kind:'subscribe', version:1,
    channel:'space/<TENANT_ID>/-/pub/resource:<RESOURCE_ID>', afterSequence:0}));
});
socket.on('message', (raw) => {
  const message = JSON.parse(raw.toString());
  console.log(message);
  if (message.kind === 'event') socket.send(JSON.stringify({kind:'ack', version:1, channel:message.channel, sequence:message.sequence}));
});

Channels use space/{domain}/{space}/pub/{publisherId}, for example space/acme/-/pub/resource:sensor-01. ACK only after successful processing. The server rejects ACK values above the highest sequence it sent.

A browser first calls POST /v1/ws-tickets with the new resource key as Bearer, then opens new WebSocket('<WS_URL>?ticket='+encodeURIComponent(ticket)). A ticket is short-lived and single-use, keeping the long-lived key out of the URL.

9. HTTP/3 WebTransport

Use WebTransport only when Connect reports it enabled. Each control frame contains a 4-byte big-endian JSON length followed by UTF-8 JSON. Open a bidirectional stream and send auth before the timeout:

{"kind":"request","id":"1","type":"auth","data":{"apiKey":"<RESOURCE_ERK>"}}

Minimal browser framing:

const encoder = new TextEncoder();
const frame = (value) => {
  const json = encoder.encode(JSON.stringify(value));
  const bytes = new Uint8Array(4 + json.byteLength);
  new DataView(bytes.buffer).setUint32(0, json.byteLength);
  bytes.set(json, 4);
  return bytes;
};
const transport = new WebTransport('<WEBTRANSPORT_URL>');
await transport.ready;
const stream = await transport.createBidirectionalStream();
const writer = stream.writable.getWriter();
const reader = stream.readable.getReader();
let buffered = new Uint8Array();
const readFrame = async () => {
  for (;;) {
    if (buffered.byteLength >= 4) {
      const length = new DataView(buffered.buffer, buffered.byteOffset, 4).getUint32(0);
      if (buffered.byteLength >= 4 + length) {
        const body = buffered.slice(4, 4 + length);
        buffered = buffered.slice(4 + length);
        return JSON.parse(new TextDecoder().decode(body));
      }
    }
    const {value, done} = await reader.read();
    if (done) throw new Error('WebTransport control stream closed');
    const next = new Uint8Array(buffered.byteLength + value.byteLength);
    next.set(buffered); next.set(value, buffered.byteLength); buffered = next;
  }
};
await writer.write(frame({kind:'request', id:'1', type:'auth', data:{apiKey:'<RESOURCE_ERK>'}}));
const auth = await readFrame();
if (auth.kind !== 'response' || auth.id !== '1') throw new Error(JSON.stringify(auth));
await writer.write(frame({kind:'request', id:'2', type:'resource.telemetry', data:{eventId:crypto.randomUUID(), timestamp:Date.now(), data:{temperature:23.9, online:true}}}));
console.log(await readFrame());

WebTransport requires HTTP/3, a valid certificate, and client support. If the requirement is simply reliable reporting, validate the business data with HTTPS or MQTT first.

10. Spaces, groups, and MQTT usernames

Resource keys use the workspace ID as a readable label. An application key already carries workspace identity and may omit that prefix. The part right of @ selects the connection location:

Without a colon, alliance is interpreted as a space in the current workspace. A group must use @alliance: or @alliance:space. Resource credentials cannot enter groups. Applications may continue to use the full acme@… form; the server verifies that acme matches the application key.

For acme@-,production, ordinary topics and # address primary space -. Use $space/production/{topic} for the other space, or subscribe to $space/# for every authorized space.

11. Reserved $… topics

Unknown $… publishes are rejected. An ordinary # does not match a first level beginning with $, so subscribe explicitly.

mosquitto_sub -h '<MQTT_HOST>' -p '<MQTT_PORT>' --tls-use-os-certs \
  -i 'earthmqtt-common-clientid' -u '<TENANT_ID>' -P '<RESOURCE_ERK>' \
  -q 1 -t '$earth/message' -t '$earth/errors/#' -v

Send and record a directed command:

mosquitto_pub -h '<MQTT_HOST>' -p '<MQTT_PORT>' --tls-use-os-certs \
  -u '' -P '<DATA_ETK>' -q 1 \
  -t '$earth/deliver/resource:<RESOURCE_ID>/cmd/config' -m '{"interval":30}'

Catch up after reconnecting:

mosquitto_pub -h '<MQTT_HOST>' -p '<MQTT_PORT>' --tls-use-os-certs \
  -i 'earthmqtt-common-clientid' -u '<TENANT_ID>' -P '<RESOURCE_ERK>' -q 1 \
  -t '$earth/resume' -m '{"afterSequence":12,"limit":100}'

Replayed messages arrive on the original business topic with "replay":true.

12. External MQTT broker

Under Connect → External Broker, enter the URL, account, subscription filters, and resource-ID extraction rule.

Topic extraction: filter plant/+/state, rule plant/{resourceId}/state, incoming topic plant/boiler-7/state produces resourceId=boiler-7.

Payload extraction: field path meta.deviceId applied to {"meta":{"deviceId":"boiler-7"},"temperature":71.2} produces boiler-7.

On first observation, the platform creates an internal resource record in the workspace. The external broker account authenticates the bridge; devices do not need EarthMQTT resource keys. This is one authenticated bridge representing multiple resources, not multiple devices sharing one erk_….

Broker connection state and resource availability are independent. Connection history records connects, disconnects, retries, and recoveries. Resource availability requires an explicit field path plus online/offline value mapping generated by AI or saved in a manual data rule. Runtime code does not infer a vendor protocol from topic names such as LWT or status, nor from familiar words such as Online or Offline. Retained messages replayed after reconnect keep the original event time and do not look like fresh device reports.

If a device has no reliable online/offline field, configure Data freshness in its resource details. When its latest real report exceeds that resource's silence window, the status becomes Data stale. This means the data is no longer current; it is not proof that the device is offline. There is no global one-hour default because normal reporting intervals range from seconds to days.

Test by publishing plant/boiler-7/state to the external broker. The connection should report Connected, its message counter should increase, boiler-7 should appear in the resource list, and Live Monitor should identify the external MQTT source.

13. Complete acceptance test

1. Create an application key with manage.
2. Create sensor-01 using POST /v1/resources and save the returned erk_….
3. Send one numeric message over HTTP with the resource key.
4. Subscribe to plant/# over MQTT with the application key.
5. Publish plant/sensor-01/state with the resource key.
6. Subscribe to $earth/events/# and confirm eventId, sequence, and publisher.
7. Send a directed message through $earth/deliver/resource:sensor-01/cmd/config.
8. Disconnect the resource, issue another command, reconnect, and use $earth/resume.
9. Check live data, online state, trends, and AI analysis in the console.

14. Troubleshooting