Notes from Terry Crowley on Layering
Layering: Any complex system needs to be designed in layers, where higher layers have more capability but are also heavier. When building an app on top of a layered platform, each app needs to choose at what layer to integrate. Most apps use the higher layers. For example, iOS offers a UIKit with views like buttons and checkboxes, multiple layout options, and screen transitions. Most apps use UIKit to build their UI. This is the right level of abstraction if you’re building a typical iOS app like a notes app. You want to think in terms of buttons and checkboxes. However, if you’re building Flutter, it bypasses UIKit, and renders its own UI components as pixels, giving them to a lower layer within iOS. Its interaction with iOS is in terms of pixels, not components. The lower layer within iOS is rarely used directly but equally critical. This shows that one API does not fit all apps. That’s why any complex system needs to be designed in layers so that each app can hook in at the right layer.
Expose lower layers: The lower layer should be independent of the semantics of the higher layer. For example, UIKit on iOS is built on top of Core Animation. Each view in UIKit corresponds to a unique layer in Core Animation. UIKit offers two things on top of Core Animation: layout and event handling (like touch or keyboard events). Core Animation is designed so that you can implement any layout system on top, say CSS. That’s layering done properly. If that weren’t the case, Core Animation would merely be an implementation detail of UIKit and not a separate logical layer in its own right. If you’re hesitating to expose the lower layer, that’s a smell of poor layering.
End-to-end principle: The lower layers should not force functionality onto higher layers that they don’t want or can implement better. For example, we typically use HTTPS, but plain HTTP should be available when needed. When you update an iOS device, the OS image is downloaded over plain HTTP. This is secure because the image is itself cryptographically signed, and the signature verified before installation, as part of iOS’s secure boot. If HTTP had in-built encryption that can’t be disabled, the resulting double encryption would be inefficient. Such a design violates the end-to-end principle: the lower layer is forcing security onto higher layers.
Passing through functionality from lower layers: Unix has the sendfile() API that transfers data from a file to a socket in the kernel, without copying it. If you’re building a web server and the user wants to download a file, and you’re building your server in C, you can invoke sendfile(). But imagine you’re building your web server in Java, which didn’t support sendfile for 15 years. So you had to do it yourself, slower than the kernel could, and while consuming more memory for temporary buffers. The moral of the story is that higher layers should expose the power of lower layers when possible.
Another example of hidden functionality is an asynchronous layer built on top of synchronous APIs 1, which are in turn built on top of async APIs. This conversion from async to sync to back again is inefficient. The solution is for the higher layer to expose a synchronous API if the underlying layer supports it, asynchronous if the underlying layer supports it, and both if the underlying layer supports both. The higher layer can also offer an adaptation — sync → async or async → sync — but use of this adapter should be optional.
Layers get replaced: If you’re building an app, you might start by using layers provided by the underlying platform, but over time, your functionality may evolve in a different direction from that of the layer. Or your requirements may turn out to require only a small fraction of the layer’s functionality. Or it may be core to your value proposition. In all these cases, you may later decide to replace the platform layer with your own layer. The opposite situation can also happen: you start with your own layer, because the platform did not provide an appropriate layer at that time. But it does, later, and you migrate. So migrations in both directions — platform layer → your own layer, and your own layer → platform layer — are possible.
Layers in apps: So far we discussed platforms. Apps can also have layers, but the lower layers shouldn’t be exposed. For example, the Microsoft Exchange email system runs on top of a database, but that’s not exposed, since exposing it will mean that third parties can insert data into the database that’s semantically invalid and breaks Exchange. This is the right decision since Exchange is an app and not an OS.
By using a worker thread.