twitterfacebookgithubgoogle-plusxinglinkedininstagram
Engineering

Liveblog: Starlink setup in our corporate network in Germany

7 minutes reading time
Starlink setup teaser
  • Tobias Hoge
    Tobias Hoge
  • Michael Schaefermeyer
    Michael Schaefermeyer
  • Kevin Scholz
    Kevin Scholz

We report on our experiences with installing and running Starlink on the corporate network. This report will be updated with new findings along the road.

For a (less technical) write-up on how we got started with Starlink in the first place, see our first blog post:

Challenges with our office internet (German only)

How Starlink works

Starlink makes possible what was long considered impossible: High-speed, low-latency, high-bandwidth satellite Internet anywhere in the world. Originally aimed at rural populations, Starlink also solves situations where internet connections of domestic network operators in Germany are unreliable. But how does SpaceX, the provider of Starlink, actually do it?

Roughly speaking, Starlink internet is powered by a high number of low-flying satellites: Low orbits mean that the distance between users and satellites is small, which results in fast ping times. Iridium, for example, the "old hand" in the satellite communications business has satellites at 780km altitude. SpaceX has its satellites flying in Low Earth Orbit from 328 to 614km (at the time of this blog post, most satellites are in a target orbit of 550km orbit altitude). A nice side effect of this is that non-functional satellites can be lowered into Earth's atmosphere where they burn up. This avoids space debris.

The high number of satellites makes the high-speed bandwidth possible. While Iridium operates 75 satellites, SpaceX is expected to launch up to 42,000 (at the time of this blog post, there are already 1,265 satellites in space). And the current stage of the rollout is even the "slower" scenario: when the satellites start communicating with each other using laser beams (which is planned for all satellites launched from 2022 onwards), even faster speeds in the gigabit speed range will be possible. Currently, however, all satellites still communicate with ground stations.

There are various websites that can show the current positions of the Starlink satellites, e.g. satellitemap.space:

Starlink Satelliten

Delivery & Hardware

We ordered Starlink on 02/23/2021, and the hardware was shipped on 03/19/2021. As expected from Elon Musk, SpaceX is up for surprises: the shipment of the hardware started in Long Beach, CA USA and arrives within a day after an adventure with DHL and the German customs after one week at our place.

Starlink delivery

Included in the package is the satellite dish (aka. Dishy McFlatface), a mount for the dish, a WiFi router, and a PoE power supply. The one-time cost for the hardware here is about $500 plus shipping, although SpaceX is paying about $2500 to manufacture it, according to several reports.

Dishy runs on about 100 watts via the PoE power supply and is more than just an antenna. According to initial teardown reports, in addition to the actual array of differently oriented receivers, it also contains an ARM processor for firmware and control of the dish. It can thus be operated independently and connected directly to any kind of Ethernet device. The included WiFi router can handle internal network routing and provide a WiFi network if desired. However, it is not needed to use the satellite Internet. Dishy itself functions as a DHCP server, giving the connected device an IP lease.

Set up

Setting up Dishy is easy. The Starlink app, available in the Google Play Store and Apple App Store, supports the search for a suitable installation site using AR. The only requirement: a clear view of the sky with no obstacles in a 100° field of view.

Starlink App AR Funktion

After that, Dishy needs to be placed correctly and connected to a power outlet via the PoE adapter on the 30m long hard-wired cable. For our first test we connected the included router and configured it via the app.

Starlink app

The configuration via the app was child's play compared to other internet providers and can also be done by non-tech-savvy people. The antenna itself is already configured, so you basically just assign a WiFi name and password here. During the setup McFlatface automatically adjusted itself to the optimal position.

Starlink antenna adjustment

First results

The first speed tests meet our expectations. After a few test runs, we even achieve higher values than previously reported on the Internet:

Speedtest Starlink

A ping of 18ms and a download of 181 Mbps is surprisingly good.

The external IP address we get assigned is part of the 188.95.145.0/24 network, is called "STARLINK-MC-FRA2-IPV4" and is listed in the IP databases in Frankfurt. This suggests that we are either communicating directly via a ground station in Frankfurt or that the ground station in France has direct peering to Frankfurt.

Certainly the biggest limitation of Starlink in the enterprise environment is currently the carrier-grade NAT. So, as known from mobile communications, we do not get an own IPv4 address (not even a dynamic one) from Dishy, but share an IP address with other Starlink customers. This makes it difficult to connect directly to the office internet, for example when using a VPN. IPv6 is also not currently supported. SpaceX seems to be testing IPv6 support at the moment though and is probably going to roll it out in the future.

Now, one can dislike the usage of a VPN to connect to an office network. Especially classic point-to-site connections are now obsolete in our view. We prefer the zero-trust approach because it is much more secure. But for the remaining services that we prefer to use over a secured connection, peer-to-peer VPNs are possible thanks to Wireguard - and thanks to solutions like Tailscale, which can be used well in a corporate context. Nevertheless, it doesn't feel right to no longer have a fixed (or dynamic) IP address.

Especially in the area of VoIP, we will add our experiences here as soon as they are available. We are curious to see how well it performs.

Integration into our network

As described in our other Starlink article, Starlink is not yet suitable as a "primary Internet". However, we are hoping to use Starlink at least as a backup for our primary internet connection.

Zweitag Network

Our network currently consists, if you leave the ISP modems aside, entirely of UniFi components: On the main network, we have the UniFi USG Pro, five UniFi managed switches (3 x 48 port, 2 x 8 port), and 8 active access points. The UniFi controller software runs in a VM on our server in the office. In a backup network, we also run a USG 3P, 1 switch and 3 mesh access points to continue to provide our colleagues with an Internet connection in case the main network goes down. The backup network's UniFi controller software runs on a Mac Mini.

The main uplink is powered by a Vodafone Cable offering (↓ approx. 1000 MBit/s ↑ approx. 50 MBit/s) and a Telekom DSL line (↓ approx. 50 MBit/s ↑ approx. 10 MBit/s). The backup network's uplink is powered by a Telekom LTE modem (↓ approx. 100 MBit/s), which books an LTE dayflat in case of failure.

In order to test Starlink in a real office scenario, we are recreating our network as close as possible.

Zweitag Starlink Network

For this we use a UniFi USG Pro, as well as two UniFi access points. A Raspberry PI runs the UniFi controller software and automatically performs tests for long-term measurements. We therefore don't need the Starlink router that comes with Dishy. Instead, we connect the "Power Brick" directly to our switch, which passes the data to our USG Pro via a dedicated VLAN. We distribute the "Starlink Internet" to the two APs via a second VLAN. This way, our employees can already test the "Internet from space", as they affectionately called it when setting it up.

Zweitag server rack

Long-term measurements

In order to make a decision about a possible switch to Starlink as our main ISP, we continuously collect metrics. We use Telegraf to do so, which runs as a Docker container on the Raspberry PI within the Starlink network. The agent regularly executes commands such as speed tests, pings, DNS queries, etc. and pushes them to the Influxdata Cloud.

You can find the corresponding Dockerfile and Telegraph configuration in this gist.

We then generate our dashboard based on this data:

Starlink Dashboard.

After about 24 hours, we can already see that Starlink constantly provides a decent bandwidth. We even reached download speeds of over 200 Mbit/s in parts, which surprised us. However, there are also some outliers in the ping and DNS requests. This is also noticeable when using Starlink from a laptop. There are occasional disconnections (sometimes once per hour for 1-3 minutes), which make some tasks like video calls impossible. However, this should be caused by the current satellite constellation and should improve in the future.

We will monitor how Starlink performs in the long run over the next few weeks. We'll also update this section when we have more data.

(Interim) Conclusion

The first test results are very promising. The Internet is usable and reaches good speeds, partly exceeding our expectations. However, the occasional disconnections do not yet make Starlink a full-fledged replacement for other ISPs. However, since "only" about 1,300 of the planned 42,000 satellites are available so far, we can assume that these disconnections will soon no longer occur.

We will add a final conclusion in the coming weeks. For the first test run, however, we are already very pleased with our new colleague "Dishy McFlatface" on our roof terrace.

Are you interested in new technologies like we are?

At Zweitag, we're always looking for curious people with whom we can set up something really cool.

If you're cloud, networking, or ops savvy and want to work on a wide variety of projects in a fun environment, check out our careers page.

Infrastructure Developer:in (DevOps) wanted

3/23/2021

Spread the news…

You are looking for the right partner for successful projects?
We'd love to help – let's get in touch!
Contact us