FireSight OT OT remote access

Install and upgrade FireSight OT

This guide is for the people setting up FireSight OT:

• the Cloud Hub, a server you run;
• Cloud Connectors, the small devices at each site, based on Raspberry Pi;
• the FireSight OT client for the people who connect.

It also covers upgrading all three.

1. Before you start

• A server for the Cloud Hub:
  • Debian 12 (bookworm) or 13 (trixie), 64-bit PC (amd64) or ARM (arm64);
  • 2 CPU cores, 2 GB RAM and 20 GB disk are plenty to start with;
  • a cloud VM, a VM in VMware or Hyper-V, or a physical server.
• A DNS name for it, for example remote.example.com, pointing at the server's public address.
• Firewall:
  • TCP 443 open to the server;
  • TCP 80 as well during installation, if you want a free Let's Encrypt certificate.
• Downloads from the FireSight portal (Downloads → FireSight OT):
  • FireSight OT Cloud Hub: firesight-ot-hub-<version>.tar.gz;
  • FireSight OT Cloud Connector SD card image: firesight-ot-cloud-connector-<version>.img.xz.
• For each site: a Cloud Connector, either one of ours (CC-3, CC-4 or CC-5) or your own Raspberry Pi 3, 4 or 5 bought locally, a microSD card (16 GB or more) and power. Also either an Ethernet cable to the site's network or Wi-Fi details.

To check whether the server is amd64 or arm64, run dpkg --print-architecture on it.

2. Install the Cloud Hub

1. Get the bundle onto the server. Either:

   • download it on the server: open the download link from your email, then copy the curl command from that page into the server's terminal; or
   • copy it from your computer:

     scp firesight-ot-hub-<version>.tar.gz you@your-server:
     

2. Unpack and check it. On the server:

   tar xzf firesight-ot-hub-<version>.tar.gz
   cd firesight-ot-hub-<version>
   sha256sum -c SHA256SUMS
   

   Every line should end in OK.

3. Install. Use your own address and email:

   sudo bash install-hub.sh --deb ./fsot-hub_<version>_amd64.deb \
        --url https://remote.example.com --letsencrypt it@example.com
   

   • ARM server: use the _arm64.deb instead.
   • Without --letsencrypt: the Hub uses its own certificate. Browsers then show a warning, and people check a fingerprint the first time they connect. You can get a Let's Encrypt certificate later from the console instead: Security & settings → HTTPS certificate (the Hub renews it by itself; port 443 must reach the Hub directly).
   • Server shared with other software: for example, a UniFi Network server uses port 8443. The installer checks the ports first and leaves out any that are already in use. Choose ports yourself with --listen :443.
   • Browser remote desktop (0.4.0): the installer also starts guacd, the service that draws RDP and VNC screens in the console. It runs in a container listening only on the server itself, and Docker is installed if missing (set up so it doesn't change the firewall). Add --no-desktop to skip it; people then use the client for RDP and VNC.
   • fail2ban (0.4.0): the installer also installs fail2ban, which blocks addresses that keep failing to sign in (to the Hub and to the server's SSH) at the firewall. Add --no-fail2ban to skip it. Adjusting and turning it off: section 7.
   • Port 443 or 80 already used by a web server on this machine (nginx, Apache, a hosting panel): the installer stops with a message. Use a server of its own, or ask FireSight support about running behind your web server.

4. First sign-in. The installer prints the first admin username and password. They're also in /var/lib/fsot-hub/initial-admin.txt. Then:

   1. Open https://remote.example.com and sign in as admin.
   2. Change the password, and turn on two-factor sign-in under My account.
   3. Follow the getting-started checklist from Ember, the helper in the corner.

5. License. The Hub starts with a 30-day evaluation for up to 3 Cloud Connectors.

   1. Open License and copy the activation request.
   2. Paste it into the FireSight portal.
   3. Paste the license file you get back into License.

   To renew automatically, turn on online renewal on the same page. See FireSight OT licenses.

6. Upload the SD card image to the Hub (optional but handy). Under Cloud Connectors → SD card image, upload the .img.xz as it is; don't unpack it. Your staff can then download it from the Hub.

Ports used:

• TCP 443: the web console, Cloud Connectors and the client.
• TCP 8443: an optional fallback for sites that block outgoing 443.
• TCP 2022 and 9000: only needed if backup software collects Hub files.
• UDP 51820: only for the optional full-network VPN.

The web console can get its own port later, under Security & settings → Web console port.

3. Set up a Cloud Connector

3.1 In the Hub

1. If you don't have one yet, add a company under Companies.
2. Go to Cloud Connectors → New connector. Choose the company, give it a name and site, and answer How does it connect to the internet?:
   • Ethernet cable: one port for both the internet and the machines.
   • Wi-Fi: the machines plug into the Ethernet port. Enter the Wi-Fi name, password and country (for example IT), and the device's address on the machine network.
   • Ethernet cable, machines on a second USB Ethernet port.
   • Tick Use Wi-Fi as a backup if the cable is the main connection and Wi-Fi should take over when it fails.
3. Click Create and get code, then Download firesight.txt. The file contains the Hub address, a one-time enrollment code and the network settings.
   • Keep the file safe until it's on the card: it contains the code and any Wi-Fi password.
   • The code is valid for 3 days and works once.

3.2 Prepare the SD card

1. Flash the image with Raspberry Pi Imager: Choose OS → Use custom, pick the .img.xz, then pick the card.
   • Don't use Imager's own settings for user, Wi-Fi or SSH. FireSight OT sets the network itself.
2. Unplug the card and plug it back in. A drive called bootfs appears.
3. Copy your firesight.txt onto bootfs, then eject the card.

bootfs also has a firesight-example.txt, which the device ignores. To set up a card by hand, copy that file to firesight.txt, remove the # from the lines you need and fill them in. The lines for Wi-Fi are:

hub=https://remote.example.com
code=XXXX-XXXX-XXXX-XXXX
layout=wifi
wifi_ssid=YourWiFiName
wifi_password=YourWiFiPassword
wifi_country=IT
machine_ip=192.168.1.250/24

Put no # and no spaces around =. For a cable connection, use only the hub= and code= lines.

3.3 At the site

1. Put the card in the Pi.
2. Connect the uplink: the Ethernet cable to the site network, unless it uses Wi-Fi. Connect the machine network if it's separate.
3. Power the Pi on.
4. Within a few minutes, the Cloud Connector shows Online in the Hub. The code and passwords are wiped from the card automatically.

Without editing the card: don't add a firesight.txt.

1. Connect the Pi to the network with a cable.
2. From a laptop on the same network, open http://<device-address>:8080 or http://firesight-connector.local:8080.
3. Set a setup password, then enter the Hub address and the enrollment code.

3.4 Give people access

1. Under Resources, add the machines (PLCs, HMIs, engineering stations) behind the Cloud Connector.
   • Serial consoles (routers, switches, drives): plug a USB serial adapter (FTDI, Prolific, Silicon Labs or CH340) into the Cloud Connector and connect the console cable. Add a Serial console resource and pick the adapter; most network equipment uses 9600 8N1. Only one person can use a console at a time. A site can turn serial consoles off with serial_consoles=off on the SD card.
   • Machine VLANs (0.4.0; several identical machine networks on one Cloud Connector): connect the Cloud Connector's machine port to a trunk port on the plant switch, and put each machine or line on an access port of its own VLAN. Under the Cloud Connector's Device network, add each VLAN (up to 20): its tag and the Cloud Connector's own address on it, e.g. VLAN 10 and VLAN 20 both with 192.168.1.250/24. Each VLAN is kept apart, so all of them can use the same addresses. Then choose the VLAN on each resource. Switch settings: Switch settings for Cloud Connectors.
   • Time for the machines (NTP) (0.4.0): under the Cloud Connector's Settings → Time server for the machines, switch it on. Then set the PLCs', HMIs' and PCs' NTP server to the Cloud Connector's machine-side address (e.g. 192.168.1.250). Its own clock follows the site's NTP if that works, otherwise the Hub through the tunnel. Its Status tab shows whether it's synchronised. A site can refuse it on the device itself: untick Allow this device to be the NTP time server on the setup page, or put ntp_server=off on the SD card. The Hub's time comes from its server's own clock (Debian keeps it with systemd-timesyncd if it can reach NTP), or from NTP servers you list under Security & settings → Time (NTP): internet ones such as pool.ntp.org, or your company's own, which then need outgoing UDP 123 from the Hub. A Cloud Connector can instead use its site's own NTP servers (its Settings → Time server for the machines → These NTP servers).
2. Grant access to people, divisions or vendors.
3. People open My access in the Hub:
   • web machines open in the browser;
   • SSH, Telnet and serial consoles open in a terminal window (Open terminal), with a transcript kept for 90 days and linked from the audit log;
   • RDP and VNC open in a desktop window (Open desktop), when the Hub has the browser desktop;
   • everything else, and all of these too, through the FireSight OT client.

See People and access for more.

If a Cloud Connector doesn't come online

• Ask Ember first. Open Ember in the corner of the console and choose A Cloud Connector is offline. It runs the checks for you.
• The site's firewall must allow outgoing HTTPS (TCP 443, or 8443) to the Hub.
• A web proxy: if the site needs one, add proxy=http://proxy.site.local:3128 to firesight.txt.
• Use a new enrollment code each time you retry. A code that was already used, or is older than 3 days, is refused. The device's setup page (http://<device-address>:8080) shows why.
• Wi-Fi: the country code is required. The password must be 8 to 63 characters.
• Software older than 0.2.1: the device sometimes needed one reboot after first boot before Wi-Fi came up, and lost its code on the way. Flash the current image, or enter a fresh code on the setup page.

More in FireSight OT troubleshooting.

4. The FireSight OT client

People download the client from their own Hub, under My access, for Windows, Mac or Linux. It always matches the Hub's version.

On Mac and Linux, the one-line install command shown there installs or updates it. After a Hub upgrade, people should download it again. See The FireSight OT client.

5. Upgrade

Upgrade order:

1. the Cloud Hub;
2. the Cloud Connectors;
3. the clients.

Cloud Connectors and clients keep working while you update them one by one. Check the release notes on the portal for anything special.

5.1 Upgrade the Cloud Hub

1. Download the new firesight-ot-hub-<version>.tar.gz from the portal and get it onto the server, as in section 2.

2. Back up the Hub's data first:

   sudo tar czf ~/fsot-hub-backup-$(date +%F).tar.gz /var/lib/fsot-hub /etc/default/fsot-hub
   

   Keep the backup somewhere safe: it contains the Hub's keys.

3. Unpack and install the new package:

   tar xzf firesight-ot-hub-<version>.tar.gz
   cd firesight-ot-hub-<version>
   sha256sum -c SHA256SUMS
   sudo apt install ./fsot-hub_<version>_amd64.deb
   

   Use _arm64.deb on ARM. Your settings, users, Cloud Connectors and license all stay.

4. Check that it's running:

   systemctl status fsot-hub --no-pager | head -5
   

   • It should say active (running).
   • fsot-hub version prints the new version, and so does the top of Overview in the console.
   • Hubs before 0.2.1 didn't restart by themselves after apt install. When upgrading from one, also run sudo systemctl restart fsot-hub.

You don't need to run install-hub.sh again for an upgrade. Re-run it only to change the address, ports or certificate.

Going back: stop the Hub, then install the previous .deb with sudo apt install --allow-downgrades ./fsot-hub_<old>_amd64.deb. If needed, restore the backup:

sudo systemctl stop fsot-hub
sudo tar xzf ~/fsot-hub-backup-<date>.tar.gz -C /
sudo systemctl start fsot-hub

5.2 Upgrade Cloud Connectors

• Devices already installed: in the Hub, open the Cloud Connector, then Remote management → Update software. The device downloads the signed new version from the Hub, which supplies the version matching itself, and restarts its service. It's back online within a minute. Its settings, enrollment and network stay. Do one site first, then the rest.
• New devices: flash the new SD card image. Also upload it to the Hub under Cloud Connectors → SD card image, so staff get the current one.
• Re-flashing an existing device wipes its enrollment. In the Hub, make a new enrollment code for that connector and set the card up as in section 3.

5.3 Clients

People download the client again from My access, or re-run the install command on Mac and Linux.

6. Where things are on the Hub server

The Let's Encrypt certificate renews by itself, and the Hub restarts to use it.

7. Sign-in protection and fail2ban

New in FireSight OT 0.4.0 (in beta).

Addresses that keep failing to sign in are blocked at two levels.

7.1 In the Hub (console)

Security & settings → Sign-in protection (administrators):

• What counts: failed sign-ins from the console (passwords and two-factor codes), single sign-on, API tokens, and the backup SFTP/S3 accounts.
• Default: 5 failures within 10 minutes block the address from the console and API for 30 minutes, twice as long each further time that day (up to 24 hours). The person sees until when.
• Never block: list your office or VPN addresses (e.g. 203.0.113.0/24).
• Blocked now: each blocked address with an Unblock button; you can also block an address by hand for an hour, a day or a week.
• Off: untick Block addresses after repeated failed sign-ins.

Cloud Connectors are never blocked (they sign in with certificates). Blocks and unblocks are in the audit log and go to SIEMs and webhooks as sign-in events.

7.2 On the Hub server (fail2ban)

The installer also sets up fail2ban, which blocks persistent addresses at the server's firewall (every port), for the Hub's sign-ins and for the server's SSH:

Change the settings in a file of your own (the installer rewrites its file, /etc/fail2ban/jail.d/fsot-hub.conf, when it's run again):

sudo nano /etc/fail2ban/jail.d/zz-local.conf

[DEFAULT]
# never blocked: your office and VPN (space separated)
ignoreip = 127.0.0.1/8 ::1 203.0.113.0/24

[fsot-hub]
maxretry = 10
findtime = 30m
bantime = 2h

[sshd]
maxretry = 5
bantime = 1h

sudo fail2ban-client reload
sudo fail2ban-client status fsot-hub    # settings in use, and who is blocked

Everyday commands:

sudo fail2ban-client banned                                # everything blocked, all jails
sudo fail2ban-client set fsot-hub unbanip 198.51.100.7     # lift a block (also: set sshd unbanip …)
sudo fail2ban-client set fsot-hub banip 198.51.100.7       # block by hand

Turn it off:

• only for the Hub's sign-ins (keep SSH protected): add enabled = false under [fsot-hub] in zz-local.conf, then sudo fail2ban-client reload;
• completely: sudo systemctl disable --now fail2ban;
• at installation: run install-hub.sh with --no-fail2ban.

Locked out? fail2ban blocks at the firewall, so the browser can't reach the Hub at all from that address. Use another connection (a phone hotspot, your cloud provider's web console) to run sudo fail2ban-client set fsot-hub unbanip <your address>, then add your address to ignoreip above and to Never block in the console.

Behind a reverse proxy or load balancer: tell the Hub to trust it (FSOT_TRUSTED_PROXIES in /etc/default/fsot-hub), so it sees people's real addresses; otherwise it would block the proxy.