I was privileged enough to get onto the internet from home in 1995 courtesy of my parents returning from Hong Kong with a cutting edge 28.8k dial up modem. I’m not sure if that modem had different standards to what we used in South Africa, but it was a nightmare, it constantly disconnected without explanation. I believe this is what caused my quest for better internet to begin. Over the years I moved to a 33.6k leased line, one way satellite downlink (InfoSat) and to what I’ve been using for the last 10 years, ADSL, all while watching the rest of the world move ahead without us. I’ve never been satisfied as the bandwidth always lagged the bandwidth requirements of modern technologies, movies, cameras, software sizes, tools and so forth. Then came Octotel.
Octotel announced in 2015 that they would be laying fibre in my area, Sea Point, I was immediately thrilled. In 2016 I kept a close eye on the progress and the pricing started being released, with a gigabit option no less, the light was at the end of the tunnel. After attending a community meeting I was told my area would only be live in early 2017. 2017 rolled around and I could see the trenching progressing slowly towards my neighbourhood.
I started scouting out the ISPs on Octotel. I definitely wanted gigabit, no compromises at this stage of the game. At the time there were only two ISPs offering gigabit on Octotel, one being Vanilla, I had never dealt with them before but they were responsive to my mails, I liked their philosophy as per their website and as I consider myself a ‘power user’ I think it was a good fit. Oh, and they support eSports teams (I enjoy the odd CS:GO tournament). Order placed. The total cost of the service would be R1668 including VAT (R969 for the line, R699 for uncapped bandwidth from Vanilla).
The first step in the Octotel process is the trenching of the road and laying the conduits down in your street, once the trenching is done you basically have a network of empty pipes/conduits in your neighbourhood. My road was trenched roughly 5 weeks before my scheduled install date. As far as I can tell Octotel works in large areas. My road was at the end of the area they were working on at that stage so I would imagine trenching to install date could be anything from 5 weeks to 10 weeks depending on where you are in that area. Once trenched Octotel runs a link cable down the main roads and installs splitters in all the manholes linking the side streets. In my case this was done last, but should usually be done before your installation.
The next day, install day, the Octotel installation team came to splice up the fibre. I had prepared the point of entry, UPS and so on for the Fibre gear so all in all the install took about 1 hour to get the fibre from the wall box to the location of my router.
Once you’re spliced up and installed you’ll have what is known as an ONT (optical network terminal) on your premises connected to the GPON network.
As you can tell by the glaring red light, I was not connected at this stage. I consulted with Octotel and Vanilla and it turns out they hadn’t actually installed the main road fibre cable to the data center yet. I wasn’t thrilled to hear this of course.
In the following days of impatiently waiting to try and figure out when I would be live I tried to read more about the optical network, how it works, how it connects and so forth.
Disclaimer: Anything in this paragraph is inferred by material I have available to me, corrections welcome.
Octotel is not an ISP, all they do is provide the last mile connectivity from their data center where ISPs can peer with their infrastructure. They are known as an ‘Open Access Network’ which means they don’t compete with their own ISPs like Telkom/SAIX and Telkom Internet which can blur the lines and lead to unfair competition against ISPs.
They make use of gigabit passive optical network (GPON) technology to connect customers. This is a preferred technology for FTTx deployments because of the ability to connect multiple premises off a single fibre using splitters, the splitters are passive and do not require power so can be stored in places like manholes without cooling, power, etc.
I’m not sure how many houses Octotel would put on a single fibre or splitter but as I understand it, the limit is 64. GPON is limited to 2.488 gigabit per second downlink and 1.244 gigabit per second on the uplink side, so if you had a congested network, contention would come into play, if all users were using their maximum allocation you would only receive 38 megabit per second down, 19 megabit per second up.
My best guess is that Octotel runs a Layer 2 network service for their ISPs, which basically means that when you plug your LAN cable into the ONT device, you are plugging in a network cable directly into your ISP network switch, just bridged by fibre.
Unfortunately going live took quite a bit longer than anyone anticipated due to the missing cable in the main road, I was eventually live 17 days after my install date, I believe this is close to a ‘worst case’ as I was the first person installed in my area/zone as far as I know.
Having an internet connection faster that was actually as fast as my hardware could handle was a new paradigm for me. After some research I found the router I had (EdgeRouter X / ER-X) couldn’t actually do network address translation (NAT) at full gigabit wire speeds, only around 350 megabits per second, not good enough 🙂 I was recommended by Vanilla to try the Mikrotik RB3011UiAS-RM and after reading online this device seemed to be capable of handling the speed.
I used iperf3 on my network to determine what the best case switching speed was on my home LAN. I tested multiple switches and the lowest the throughput was 920 megabits per second with consistent sub 0.5ms local equipment latency. This is useful so we’ll know if we are hitting the limits of my own hardware.
The speed tests are more to illustrate the speed of Vanilla’s network and not Octotel, so keep that in mind when interpreting the results. Your ISP using Octotel may have different routing and performance which could produce better or worse results.
In order to eliminate any possible delay my router would have on the connection I connected directly to the GPON ONT for these tests via LAN, so no switch, firewall etc.
For the following tests I made use of the speedtest-cli tool which is essentially the same as the in browser tool but produces information in an easier to digest format. Each result is based on an average of three tests.
|Server||Location||Ping||Download (Mb/s)||Upload (Mb/s)|
The test to Vanilla is essentially the highest achievable speed on this line as it is almost directly connected, doing a trace to the server I found only 3 hops in between me and the Vanilla speed test server. We can see here that the GPON network is basically as fast as my local gigabit LAN. The speeds only dropped roughly 20 megabits per second from the iperf3 tests. The fastest test was only 5 megabits per second slower than local LAN.
It is clear to me that my router, even though it is a high quality router, it still can’t cope with full gigabit speeds if you want to use all its features. After moving the ONT from direct LAN to the router the top download speed remained basically the same, however, in order to do QoS on a Mikrotik router you need to disable the hardware NAT acceleration feature known as fasttrack. This dropped my speed down to around 650 megabits per second. I did some digging and could see that on a single connection test only one core of the router’s CPU is in use (of 2 cores), when doing a parallel connection test both cores get used and the performance then gets back up close to wire speeds. This single connection ‘slowdown’ is caused by my router’s CPU unable to process the firewall and QoS rules fast enough, not the connection itself. It’s nice to have these types of problems now 🙂
The basic throughput tests are a bit synthetic and may not really show the true experience a user has on the internet. One of my personal priorities has always been the latency of a connection and how it copes when under load. On ADSL people often assume lag or high latency spikes are caused by downloads but more often than not it is saturating the upload bandwidth that causes lag spikes and delays. This is the worst issue for gamers and people operating on remote servers requiring quick feedback.
I made use of the DSLReports speed test tool to test my line and lag.
The initial results were a bit of a surprise, I thought at 25 megabit per second upload buffer bloat would be a thing of the past but it turns out even this can be saturated and cause lag, here was my initial result.
Coming from ADSL, I have had to solve this problem before and am relatively experienced in doing so, which basically just involves configuring a QoS queue on my router and specify the maximum upload link speed and queuing algorithm of choice (I used PCQ in this case). Here is the next result with QoS enabled.
These tests are not very scientific I must warn you. I fired up Counter Strike Global Offensive (CS:GO) and played a few games. When matched on Cape Town servers my reported ping was between 3 and 8 milliseconds and when on Johannesburg servers between 22 and 30. I felt that the responsiveness (eg when a shot was registered by the server) was basically like a LAN game, where as on ADSL previously it had a minor delay before registering a shot.
In normal households you can’t stop your family from using the internet while you game. This can cause some lag and then family feuds 🙂 I started up some downloads and uploads while playing to see what the impact on latency was. In short, I had no noticeable increase in my ping at any stage.
I also downloaded a game from Steam just to test that aspect. I downloaded the older Counter Strike: Source which is 3.4 GB and it was completed in roughly 3 minutes and 20 seconds!
I don’t have a 4K TV at home so my tests were limited to 1080p on my streaming devices.
To keep it short and sweet, all video on Netflix, Amazon Prime Video and YouTube all loaded almost instantaneously and quickly upgraded themselves to 1080p video, even 1080p @ 60 fps in some cases where it was available. I never had an issue with mid-video buffering.
To test this a bit better I moved to my PC which has a near 4K display and tested some 4K streaming. This tested the connection a bit more but it was still able to play the video without any buffering which was great.
DSTV Catch up plus also had no problems and a 1.8GB HD movie took about 3 minutes to download.
While my time with fibre is only just beginning, so far I’m extremely pleased with the results. Things like uptime and contention may appear later which I will be keeping an eye on and monitoring using some automated tools.
After using it for some time I rarely see a download (drivers, ISOs, updates, etc) go above 20 megabytes per second. It is still extremely fast but nowhere near saturating the gigabit capacity of the line or even the 650 megabit limit of my router. I would presume this is because most servers, peering links and so forth don’t really cater for these speeds as the norm.
I do however often saturate the upload speed of 25 megabits per second uploading application archives (Java WAR files) and videos so I may adjust my service to cater for this. I’d like to see Octotel offering a 1000 / 200 package as they offer a 200 / 200 package. Given than the upload and download channels on a GPON network are separate this should be easily possible. Even better would be a service similar to CyberSmart 400Mbps service, allowing users to change their speeds on the fly 500/500, 700/300, etc 🙂 wishful thinking perhaps 🙂
It seems my long, 22 year+, journey to a proper internet connection is finally coming to an end. Thanks Octotel and Vanilla for getting me across the line. When the XGPON upgrade comes, I’ll be signing up I’m sure 🙂
I hope this post has provided you with some useful information, if you have any questions about this post please comment or contact me and I will try and answer them for you.