Systems & Stories

Next couple of blogs are going to be centered around Embedded Linux boards because that’s something that has always fascinated me. One of the best boards in the market with huge adoption, excellent documentation, open DTBs, detailed datasheets, and a very active open-source ecosystem is the BeagleBone Black — so naturally, I bought one.

It has always been fun messing around with boards, kernels, bootloaders and random hardware bring-up experiments. But what truly goes on from the instant power is applied to a fully operational Linux system is far more complex than most people realize. There’s an entire chain of events involving ROM code, SRAM, DDR initialization, PLL stabilization, memory calibration, bootloaders, kernel loading, and finally userspace startup.

The entire flow on the BeagleBone Black looks roughly like this:

At a very high level, power is first applied to the board, after which the AM335x Boot ROM begins executing. The ROM loads the SPL/MLO into SRAM, which then initializes clocks, DDR, PMICs and other low-level hardware. Once external DDR becomes usable, full U-Boot is loaded into memory, followed by the Linux kernel, device tree and finally the root filesystem before Linux userspace comes alive. The Boot ROM itself is burned directly into the AM335x SoC by Texas Instruments and is immutable by design. Since TI manufactures only the SoC and not the final board, the ROM has no knowledge about the exact DDR layout, PMIC configuration, GPIO routing, peripherals or memory timings used by a specific board like the BeagleBone Black. It intentionally performs only minimal initialization required to locate and load a secondary bootloader into SRAM.

That responsibility falls onto the board-specific SPL/MLO and U-Boot code provided by projects like BeagleBone. In other words, the actual usable boot sequence is really ROM → MLO/SPL → U-Boot → Linux Kernel → RootFS

My current development setup looks something like this:

The SD card only contains the earliest bootloader stages (MLO and u-boot.img). The actual Linux kernel (uImage) and Device Tree are served over TFTP from the host machine, while the root filesystem is mounted over NFS. UART logs are captured through a USB-UART adapter connected to the host machine so that every stage of the boot process can be monitored live.

This setup is incredibly convenient because kernel iterations become extremely fast. Device Trees can be modified instantly without reflashing SD cards, rootfs changes appear immediately over NFS, and complete boot logs are always visible over UART. Once the Boot ROM loads the SPL into SRAM, the real board bring-up begins. At this stage, the system is still operating in an extremely primitive state. DDR is not initialized yet, most peripherals are unavailable, the normal serial subsystem does not exist yet, and only a tiny amount of SRAM is available for execution. That means debugging this stage is not as simple as adding a printf() somewhere in the code. To enable very early UART debugging support inside SPL, I enabled the following U-Boot configuration options:

CONFIG_DEBUG_UART=y
CONFIG_DEBUG_UART_OMAP=y
CONFIG_DEBUG_UART_BASE=0x44e09000
CONFIG_DEBUG_UART_CLOCK=48000000
CONFIG_DEBUG_UART_SHIFT=2

These enable a minimal UART backend that works before the full serial driver stack is initialized. Without this, early SPL debugging would effectively be blind. After enabling these and sprinkling debug logs throughout the SPL code, the actual early boot flow became visible:

This was probably the most fascinating part of the entire experiment because you can literally watch the SoC slowly “wake up” in real time. First the watchdog gets disabled so the board does not reset during bring-up. Then UART pinmuxing appears, clocks become active, PLLs stabilize, PMIC voltages get configured, DDR PHY calibration begins and eventually external RAM becomes usable. One thing that immediately caught my attention in this sequence was the DDR initialization path. A lot of questions started appearing naturally: – How is time even measured this early? – What exactly is a DPLL? – How are DDR clocks generated? – Where do DDR timing values come from? – Why are timings so board specific?

The BeagleBone Black starts with a very simple 24 MHz crystal oscillator physically present on the board:

But DDR memory obviously cannot operate at just 24 MHz. To generate the required high-speed clock, the AM335x uses a DPLL (Digital Phase Locked Loop), which multiplies the reference clock frequency. In the case of the BeagleBone Black DDR subsystem: 24 MHz × 16.67 ≈ 400 MHz So the DDR subsystem ultimately operates at roughly 400 MHz. This explains the DDR PLL and clock generation part. But the far more interesting topic is what happens after the clock becomes stable because having a 400 MHz clock alone is not enough. The controller still needs to know: – how long rows need to stay active – when reads are allowed – how frequently refreshes happen – how long writes take – how quickly rows may be switched – when precharge operations complete

That entire behavior is controlled through the EMIF timing registers. For the BeagleBone Black, the DDR timing configuration structure looks something like this:

These are not arbitrary hexadecimal constants copied from somewhere on the internet. Each register contains multiple tightly packed DDR timing parameters encoded into specific bitfields.

For example: sdram_tim1 = 0x0aaad4db contains fields for: – tRCD – tRP – tRAS – tWR – tRRD – and several other DDR timing constraints

One particularly interesting register is the refresh controller. DRAM cells store information as tiny electrical charges inside capacitors. Those charges naturally leak away over time, which means memory rows must periodically be refreshed or the stored data disappears. On the AM335x EMIF controller, refresh timing is configured through ref_ctrl.

The stock BeagleBone Black configuration uses `ref_ctrl = 0x00000c30`.
The programmed refresh counter value becomes:
0x0c30 = 3120 cycles
Since DDR operates at 400 MHz:
1 cycle = 2.5 ns
Therefore:
3120 × 2.5 ns = 7.8 us

meaning the EMIF performs a refresh operation roughly every 7.8 microseconds. Naturally, the next thought was what happens if I intentionally break these timings? Surprisingly, increasing the refresh interval significantly did not immediately crash the board. I pushed the refresh timing as high as 161.4 us and the system still continued operating. I stress-tested memory using both mtest and memtester, but the board remained surprisingly stable.

So I moved on to a much more critical timing parameter: tRCD. tRCD stands for RAS-to-CAS Delay and defines how long the controller must wait after activating a DRAM row before issuing a READ or WRITE command. Inside SDRAM_TIM_1, the tRCD field is encoded in bits [24:21]. From 0x0aaad4db the encoded value becomes 5. The AM335x EMIF timing fields follow this encoding rule: actual value = encoded value + 1.

Therefore:
tRCD = 5 + 1 = 6 cycles
At 400 MHz DDR:
1 / 400,000,000 = 2.5 ns
which means:
tRCD = 6 × 2.5 ns = 15 ns

So the EMIF waits roughly 15 ns after opening a DRAM row before attempting to access data from it. At first, I reduced this from 6 cycles → 5 cycles which changes the delay from 15 ns → 12.5 ns and surprisingly the board still booted successfully. But once I reduced it further 6 cycles → 3 cycles which becomes 15 ns → 7.5 ns the board finally became unstable and crashed shortly after full U-Boot started. That behavior makes perfect sense because reducing tRCD too aggressively causes the EMIF controller to issue READ or WRITE commands before the DRAM row has fully stabilized internally. At that point, memory accesses become timing-unsafe and corruption starts appearing. One interesting observation was that SPL itself still continued functioning even with broken DDR timings. The reason is fairly simple: SPL barely uses DDR compared to full U-Boot. SPL primarily executes from SRAM, uses very small buffers and performs relatively few memory accesses. Full U-Boot, however, relocates itself entirely into DDR, enables networking, loads kernels and starts stressing memory much more heavily. That increased memory pressure is what finally exposes unstable DDR timings.

Stay Tuned for More :)

By – Shaurya TW: AI Assistance has been used for image gen and some paraphrasing of texts.

Whatsapp and iMessage are a duopoly in the instant messaging space which is probably not new to anyone. Meta is known for making money from user data. Living in India means Whatsapp is the primary means of communication. (iMessage not used as much due to the abundance of SMS spams and lack of iPhone users). To my surprise on my PiHole setup one day I noticed a lot of requests being blocked to this domain – dit.whatsapp.com

Looking at the networking tab in my browser while using whatsapp web –

This kind of sent me down the rabbit hole of understanding what this exactly was and made me aware of the DIT metadata system meta engineering blog whatsapp faq post.

It's of course impressive the amount of engineering it takes to deploy such a system at scale. Also is a bit weird that this much engineering effort was put to just collect user data. Even though its anonymous as they repeatedly claim in the blog, and the intention is just to improve the working of the app and do analysis on events such as app crashes, they could change this at anytime since its not open sourced. And as mentioned in various posts such as this and this, its very clear that the anonymous claims aren't very promising either.

If the privacy angle isn't enough to migrate, I think this graph can drive another point home.

For the unaware, Whatsapp Business API is the cash cow which gave Meta multifold returns on their $19B buyout of the business back in 2014. A direct effect of this is very noticeable now after number of years of free use. This is how most people's archived chats look like (if they use this feature like me to basically shove all the businesses spamming ads).
Tired of all this I went looking into various threat models for instant messaging services and found a couple of attack vectors I had concerns over: – Identity registration service – Cloud backup service – Media upload endpoints – Messages service – Audio/Video call service – Push notification service

If one takes the case of Whatsapp or any centralized messaging apps, the claims of end to end encryption can maybe put your worries to rest when it comes to the messaging service but MITM attacks are still a concern as concluded in this white paper.

When it comes to the cloud backup system, that is a major concern due to it being unencrypted by default. In this white paper it is also discussed about some issues with the password attempt count being modifiable –

Tons of folks in the privacy space recommend Signal being the go to with the highly battle tested Signal protocol, but I did want to go full decentralized due to the elimination of any central trust anchor and see how that would work. I think Signal should be sufficient, but now that I am in this rabbit hole, might as well go full psycho.

Now there are quite a lot of great comparisons online such as this, and I was curious to try SimpleX chat out based on its move away from any user identifiers at all. One great thing about this when compared to something like Session is that I am not too fond of the idea of onion routing just because you would be adding your server as a node in the tor network. I would much rather prefer my server just being used for communication amongst my friends and family (one can argue this is less private but I would prefer it this way)

Five out of the six main issues which are mentioned earlier are all basically solved if I go with SimpleX. Why push notifs is still kind of an issue I will get into later. For the more curious folks wanting to do a deep dive into its architecture, please check out their protocol spec.

  1. Starting off with identity registration being the main problem with most apps like Signal/Whatsapp/Telegram, I never understood why phone numbers have to be involved. My idea of an instant messenger would involve sharing some sort of link/QR code with someone you met and can now keep in touch with. This idea of phone numbers being involved in any messaging app will eventually lead them to capitalize on it by sending you ads. (Since your phone number is basically linked to all other services)
  2. Cloud backups as mentioned earlier are a major issue by design because you are basically sending all your chat logs somewhere less secure. Now end to end backups are a thing, but I still would want control over where my backups get pushed to / what I can do with them. A simple export and import option would be great but whatsapp still doesn't allow an easy way to import the exported data.
  3. Support for self hosted messaging service (SMP) and media file transfer service (XFTP) is one of the biggest advantage with SimpleX and removes reliance on any centralized authority.
  4. Audio and video in most modern apps use Web-RTC based communication and in this case we again have the option to use our own.
  5. Push notification is a major problem due to Apple enforcing a closed ecosystem requiring all notifications to pass through the Apple Push Notification service (APNs), which have very strict rules. Unlike other platforms, iOS does not allow apps to maintain direct, persistent connections to custom servers to receive messages, making a custom backend to APNs bridge necessary. On Android, its very much possible thanks to UnifiedPush and other such implementations.

How I Setup SimpleX Chat with custom servers

Now that we have discussed the concerns and understood other options out there, let's set this up so people can use it with their peer groups. I want to keep the setup fairly simple like this –

If you or your peer has this setup already then all you would need is the addresses for the SMP, XFTP and the WebRTC servers and you should be good to go. If you are the first one in your group, then I'm just going to provide some links related to hosting these which you can follow - – Simplex Chat Self Hosting DocumentationSimplex as your private messengerCoturn Docker ImageSmp-server Docker ImageXFTP-Server Docker Image

In my case, due to resource limitations in my vps, I didn't go ahead with standard docker images but instead pulled the binaries out of the docker images and just created systemd services to avoid docker overhead.

Once the servers are setup, can follow the steps below for setting up the client app -

  1. When one installs the app and has created their profile, you are given a choice to choose either SimpleX or Flux servers, can choose either for now, we would not be using these later anyway.
  2. Notification privacy options can be confusing. This is the only service in the app which is not decentralized, due to restrictions enforced by Apple as mentioned before but they do give an option to not use any push servers. I would probably go with the instant option for now since its as secure if not better than other messaging apps.
  3. Once the initial setup is done, we dive into Settings > Network & servers > Your servers and add both the custom SMP and XFTP servers. If using port other than 5224 for xftp then should be mentioned in the address. Here the format is usually: xftp://<server-identifier>@<domain-name>:<custom-port> If using port other than 5223 for smp then should be mentioned in the address. Here the format is usually: smp://<server-identifier>@<domain-name>:<custom-port> Do test out both servers and enable use for new connections
  4. After adding both servers just disable the SimpleX Chat preset server and Save the custom servers. Set the WebRTC ICE servers by adding details for your TURN and STUN servers. The format is usually -stun:<domain>:<port>turn:<username>:<password>@<domain>:<port>
    5. These are some advanced settings that can be applied

    And voila you should have your custom completely private setup running. Now all you have to do is share the qr codes with friends and enjoy super private messaging! More on that here
    By – Shaurya No AI involved in writing these :)