The Google Privacy Sandbox: The Big Picture and the Privacy Sandbox Browser Elements

Welcome back! This article will cover two topics

1. An overview of the “big pieces” of the Google Privacy Sandbox.
2. A first drill down into the browser-based elements/structures that are core to the sandbox.  

We are not going to get into flows or “connective tissue” for how the various pieces talk to each other (e.g. OHTTP) for now.  We are going to build up the main static elements first, bit-by-painful-bit, exploring all the underlying concepts and technologies that reside within and drive the functionality of these “big pieces”.  Then we will show how all the pieces fit together.  

Another aspect of my approach that differs from the documentation is that I am going to attempt to provide a holistic view across all the APIs.  The existing documentation on Google’s site is organized around each API.  Makes sense.  Each has their own product manager who is responsible for the API and understands its workings.  The problem is each of these APIs runs across the different pieces of infrastructure (i.e. servers, browsers) in different ways: sometimes uniquely, sometimes in parallel but without interaction, and at other times with interactions and dependencies on one or more of the other APIs.  

The Google Privacy Sandbox is a comprehensive privacy-preserving advertising system that involves contextual targeting, behavioral targeting and reporting.  It depends for its success on another series of APIs – such as the Private State Tokens API – each of which has its own product owner.  Looking at it from the perspective of any single API makes it impossible to understand how all the parts work to form a functioning whole.  Looking at it instead from a systems design perspective, starting with each core piece of hardware/software and identifying how the APIs run and interact on that part of the system, and then building up until the core pieces are connected allows us to understand the dynamics of the entire system rather than its piece parts.

An Overview of the Complete Sandbox Architecture and Timeline

So, let’s jump in.  Figure 1 shows the core browser elements, server-side applications, and “on-the-wire” improvements to web connectivity of the Google Privacy Sandbox as currently envisioned when it is fully implemented across the Web.  This is not intended as a diagram representing the technical architecture.  It is strictly intended to call out the various components that will be discussed in subsequent articles.

Figure 1- Overview of The Google Privacy Sandbox System

‍

I say “currently envisioned” because we are a long way from having stable releases of any component of the architecture, some of these are only proposals and it is possible there will be significant changes over the next one-two years as these technologies evolve. They are:

• The browser (client), with a number of elements, of which several are new (shown in dark grey) that we will cover in more detail in the next few articles.
• An SDK runtime for mobile apps.
• A Trusted Execution Environment (TEE).  This is a secure, attested cloud-based environment running Protected Audience API services centrally versus in-browser, usually for performance and scale reasons.
• At least two Key Management Servers run by legally and physically separate entities that provide cryptographic keys needed to support the Sandbox – especially the Key/Value service that is part of the Protected Audiences API.  The reason there is a need for two servers is  because the Protected Audience API uses split private keys as part of multi-party computation to provide better security, as no single party will host any service’s private keys.  The companies that run these servers are called Coordinators.
• K-anonymity servers run by Google to provide real-time, algorithmically-derived k-anonymity thresholds to provide protection from microtargeting.
• A publisher’s server(s) where auction logic files are stored and called by the client’s browser to run an ad auction.  These server(s) will also contain code for data collection for reporting purposes
• A “buyer’s” ad server(s) where creatives are stored, bidding logic resides that is called from within the browser, and/or source event registration and data collection occur for reporting purposes.
• “On the wire” technologies to better protect browsers from being fingerprinted or identified, including DNS over HTTPS, Network State Partitioning, and IP Protection.

Some of these elements are in testing today in what are called Origin Trials.  We described one of these -the FLEDGE Origin Trial, in the last article.  Others are in the process of being specified.  Others have been specified and there are demo or alpha code bases that are available for testing.  This last category is particularly true of the server-based applications needed to complete the platform.  Figure 2 shows the history of the development of core APIs of the Sandbox, the current plan for deployment to testing and general availability, and an overview of the current status of core technologies for the platform.

‍

Figure 2 - Google Privacy Sandbox API Roadmap

‍

Source: https://privacysandbox.com/open-web/#the-privacy-sandbox-timeline

There are similar roadmaps for the server-side technologies, which will be provided in the articles on each of those applications.  In the meantime, they can be found on Google’s Privacy Sandbox site here.

Over the next few posts, I will be drilling down into each of these core elements and examining them in extensive detail.  Today we will start a series of posts about the client browser and what happens there.  There are two reasons for this:

1. Cookies were a browser-based technology for AdTech, and most of the changes needed to replace the functionality of cookies with a privacy-preserving replacement are occurring in the browser.
2. The current FLEDGE Origin Trial #1 is strictly focused on running auctions in the browser, so focusing on the browser elements helps to discuss the current mechanics of the sandbox in the first half of 2024 before the server-side elements come into play.  

A Quick Review of Browser Components

Figure 3 shows some core components of a typical client browser before the Google Sandbox.  There are three key groupings we need to be concerned about from this diagram:

1. The browser header
2. The main browser frame, also known as the Body Element
3. Browser storage.  

‍

Figure 3 - Key Components of Browsers for Google Sandbox Discussion

‍

The true technologists will argue that I am skipping a great deal of detail necessary to understand the workings of key components of the Sandbox, such as networking.  I agree.  But these articles are intended for technically-oriented senior product leaders and AdTech executives who want to understand the Sandbox, so I am only going to discuss those browser components at a level of detail needed to achieve that goal for the specific aspect of the Sandbox I am discussing in a specific article.  For those readers who desire a more complete, holistic view of browsers as a way of understanding the Sandbox, this article by Deepak Puttarangaswamy provides an excellent summary.

HTTP Headers

HTML headers allow the web browser and a server to transfer specific configuration or other details needed to fulfill a request from or response to the browser.  When a user clicks to a URL, the browser sends a request that includes a header. This request HTTP header contains additional information for the web server. The web server then responds by sending specific data back to the client to be rendered on the user’s screen. A response header is attached that contains information on whether the initial request went through, type of connection, encoding etc. If the request did not go through, then HTTP response headers will contain an error code.

End-to-End Headers

End-to-end headers apply to both requests and responses, but don’t apply to the content itself. These headers must be present in any HTTP message transmitted to the final recipient.  Intermediate proxies must retransmit these headers unmodified and caches must store them.  The most common are Connection, Keep-Alive, Cache Control, and Date.

Request Headers

Request headers are important because websites tailor their layouts and design in accordance to the type of machine, operating system and application making the request. Otherwise, content might be displayed incorrectly. One important header is the user agent header.  It provides critical information on the software and hardware of the source browser as shown in the example in Figure 4.  

‍

Figure 4 - Example of HTTP Request Headers (with user agent header highlighted)

‍

We will come back and revisit the user agent header when discussing the User Agent Client Hints API.  This is because the user agent header provides a substantial amount of information that can be used to fingerprint a browser and uniquely identify it, thus violating privacy.   As part of the Privacy Sandbox, Google has proposed the Client Hints API to enables sites to request the information they need while limiting the information shared about an online user.

Response Headers

A response header is an HTTP header that can be used in an HTTP response and that doesn't relate to the content of the message. Response headers, like Age, Location or Server are used to give a more detailed context of the response.

Representation and Payload Headers

Representation headers contain information about the representation of the resource sent in an HTTP message body. Clients specify the formats that they prefer to be sent during content negotiation (using Accept-* headers), and the representation headers tell the client the format of the selected representation they actually received. 

Payload headers describe payload information related to safe transport and reconstruction of the original resource representation, from one or more messages. This includes information like the length of the message payload, which part of the resource is carried in a specific payload (for a multi-part message), any encoding applied for transport, and message integrity checks, among other elements

Each entity tag is represented as a pair.  Representation headers include Content-Language, Content Encoding, and Content Location.  Payload headers include Content-Length, Content-Range, and Transfer Encoding.

‍

The Main Browser Frame

The main browser frame is where the browser’s rendering engine takes all the HTML, CSS, JavaScript and other information about the web page and displays it in the browser window.  Many of the adaptations for the Privacy Sandbox to the client-side architecture occur in the main browser frame.  We will not drill down into all the sub-elements of the main browser frame here.  However, I do want to focus on a few elements/concepts that will be critical to understand which are leveraged/extended to support functionality in the Privacy Sandbox

Web Workers

As the web evolved, developers being developers wanted to execute “bigger and better ideas” in browsers.  Often these web-based applications performed more resource intensive tasks, such as  complex calculations for image generation. So, a solution was needed to allow resource-intensive tasks to run in their own threads, in parallel, to ensure browser response times remained tolerable.

The outcome was a specification for web workers.  Workers are background threads in JavaScript that execute code independently of the main thread. Introduced in HTML5, they are designed to offload tasks that can be time-consuming or resource-intensive and to overcome the limits of single-threaded JavaScript execution.  Workers are relatively heavy-weight, and are not intended to be used in large numbers. They are intended to be long-running scripts that are not expected to respond to clicks or other interactions.  Generally, workers have a high start-up performance cost, and a high per-instance memory cost.

Worklets

Worklets are a new concept introduced in the CSSS Houdini specification and, as we will discuss in later articles, are critical to the Privacy Sandbox.  Worklets are a lightweight version of web workers that allow developers to extend the CSS rendering engine to handle custom CSS properties, functions and animations.  Worklets are similar to web workers in that they can run scripts independent of the main JavaScript execution environment.  However, there are some significant differences,  Worklets:

• Are thread-agnostic. That is, they are not designed to run on a dedicated separate thread, like each worker is. Implementations can run worklets wherever they choose (including on the main thread).
• Are able to have multiple duplicate instances of the global scope created, for the purpose of parallelism.
• Do not use an event-based API. Instead, classes are registered on the global scope, whose methods are invoked by the user agent.
• Have a reduced API surface on the global scope.
• Have a lifetime for their global object which is defined by other specifications, often in an implementation-defined manner.

These differences, especially those that allow user agents to parallelize work over multiple threads with global scope or to move work between threads as required, as well as the security features that prevent one worklet from accessing user data from another worklet, make worklets a perfect vehicle for implementing ad auctions and bid generation in the browser.

iFrames

The term iFrame stands for “inline frame”.  iFrames are an HTML element that loads a second HMTL element within the main browser frame of another web page while allowing the user to interact with the content.  They are commonly used to embed specific content or other interactive elements into a page.  Documents rendered in an iFrame are encapsulated within their own browsing context with separate CSS, JavaScript, and HTML content.

Calling an iFrame requires a single line of HTML:

‍

<iframe src=“https://www.example.com/” width=”200” height=”200” sandbox=“allow-downloads”></iframe>

‍

No doubt you have heard of  iFrames as they are often used to display ads since they provide more flexibility than an inline script.  But from the perspective of the Google Sandbox they are problematic because they can provide a browser fingerprinting surface where they can identify an individual based on the ads they have been presented over a period of time.  So Google Sandbox had to create a new concept, Fenced Frames, to replace the standard iFrame.  We will be talking more about Fenced Frames in the next article.   

Browser Storage

One of the most confusing topics for me even as I worked in AdTech for many years was browser storage.  I imagined there was actually some space in the browser itself, similar to HMTL elements in a .html document, where the data used by the browser was stored.  Ultimately, I realized that this was not the case.  That, in fact, browser storage was nothing more than a set of encrypted files in different subdirectories holding different kinds of data under my c:\users directory (on Windows).  These storage elements include:

• Local Storage
• Session Storage
• Index DB
• Cookies

The contents of these types of storage can be examined in the Chrome Developer Console (Figure 5).

‍

Figure 5 - Display of Browser Storage Contents in the Chrome Developer Console

‍

Local Storage

Local storage is used to hold persistent data that is needed across browser sessions or when a page refreshes for a specific domain.  Each domain basically has its own encrypted file within the storage folder the browser uses on the local hard drive.  One domain cannot gain access to data stored in local storage by another domain.  Stored data is not automatically sent to the server with every HTTP request. This means that the server will not have access to the data unless it is specifically requested.  

Session Storage

Session storage is similar to local storage with one key difference: the data stored in session storage is automatically deleted when the user closes the browser tab or window where the data is stored.

IndexDB

IndexDB is a large-scale, NoSQL storage system that allows storage of just about anything in the user's browser. In addition to the usual search, get, and put actions, IndexDB also supports transactions. Each IndexDB database is unique to a site domain or subdomain, meaning it cannot access or be accessed by any other domain. Data storage limits are usually quite large, if they exist at all, but different browsers handle limits and data eviction differently.

Cookies

Do I really need to explain these to someone in AdTech?  

Figure 6 provides a summary of these different types of storage.

‍

Figure 6- Summary of Different Types of Browser Storage (needs update)

 Source: https://www.geeksforgeeks.org/difference-between-local-storage-session-storage-and-cookies/ 

‍

Permissioning

Permissions Policy allows the developer to control the browser features available to a page, its iframes, and subresources, by declaring a set of policies for the browser to enforce. With these tools, the  top-level site owner can define what it and its third parties intend to use, and removes the burden from an end-user of determining whether the feature access request is legitimate or not. For example, by blocking the geolocation feature for all third parties via Permissions Policy, the developer can be certain that no third party will gain access to the user's geolocation.

Many of the Privacy Sandbox APIs use the browser’s permissions policy to control access to various capabilities. With third-party cookies, the page owner has no granular control over how the cookies are used by third-party iframes.  WIth the Privacy Sandbox APIs and Permissions Policy, a page can allow or deny Privacy Sandbox APIs from being used by the page itself and third-parties on the page. For example, a page owner, such as a publisher, can use Permissions Policy to allow specified third-parties to run an ad auction, or deny all third-parties from reading the user's topics.

We will cover permissions policy and how it impacts the Privacy Sandbox in detail in a later article.

The Browser and Google Sandbox

That overview of the browser, while somewhat basic, was important because it provides a baseline of how browsers operated before the Google Privacy Sandbox was conceived.  Figure 7 is an updated architectural overview of the Chrome browser with the new Google Sandbox elements.  Clearly, a lot of changes are being made to reengineer Chrome, and online advertising as well, to adapt to the new privacy-centric view of the Web.   We will delve more deeply into those changes in the next couple of  articles.

Figure 7- The Browser with Updates for Google Privacy Sandbox

‍