More specifically, we are tagetting the morning Tuesday, January 27th (US/Pacific timezone). The bsky.network firehose will be switched from the narelay.pop2.bsky.network relay instance to the relay1.us-west.bsky.network relay instance.

This transition was previously announced but postponed until now. The majority of Bluesky internal services transitioned to the new relay instances in the past week.

How will this impact relay consumers?​

If you are connected to the bsky.network hostname, your websocket connection may drop, and the cursor sequence will jump forward by a significant delta. If your client implements auto-reconnection, no manual intervention should be necessary, and impact should be minimal.

It is likely that some events will be duplicated with new (higher) sequence numbers. The per-account revision (rev) field can be used to de-duplicate events at the account level. For many use-cases (such as bots, feed generators, etc), re-processing a small number of events should not be problematic.

The backfill window of the bsky.network instance should reflect what is seen on the firehose. That is, reconnecting with an "old" (lower) cursor value should not result in replaying the full "new relay" 72 hour backfill window. This is because an intermediate service (rainbow) is inserted between the relay backend and the public endpoint.

Operators can implement the transition on their own schedule by connecting directly to specific relay instances:

• narelay.pop2.bsky.network is the current relay behind bsky.network. It has been in operation since Fall 2024 and runs the older "bigsky" implementation. The current sequence is around 17502400000 (aka, 17 billion). This instance will be shut down a week or two after the transition.
• relay1.us-west.bsky.network will be the new relay behind bsky.network. The current sequence is around 26653242499 (aka, 26 billion).
• relay1.us-east.bsky.network is an alternative relay running in a separate data center. The current sequence is around 26653242087 (aka, 26 billion).

Note that the east/west sequence numbers are similar (because they started at the same time), but are not identical. The content is very similar (they are both full-network relays), but are not seamlessly interchangeable.

What about Jetstream Consumers?​

Jetstream uses timestamp cursors, so no observable cursor jump will happen. Consumers may see a small number of duplicate/replay events around the transition.

How will this impact PDS hosts?​

There should not be any noticeable impact from this transition. The new relay instances have already been consuming from all PDS hosts for many months, "request crawl" messages are mirrored, and rate limits have been synchronized between old and new relay instances.

The new relay implementation does include support for string verification of "MST inversion proofs", part of the Sync 1.1 protocol changes. However, strict verification will still be disabled for now.

We do intent to enable strict MST verification in the near future. The reference PDS implementation has included the required changes for some time. We will monitor which PDS instances and implementations might be impacted and work with developers and operators before making that change, and will make public announcements ahead of time.

Why are we doing this?​

The new relay includes support for Sync 1.1 protocol features, such as the #sync message type and additional metadata supporting "MST inversion". It also fixes a handful of bugs that have caused operational problems with individual PDS implementations.

The new implementation also removes a huge amount of deprecated code related to "archival" relay operation and indexing of record data in the relay itself. It should be easier to maintain and develop new features for the relay going forward, including smarter rate-limiting and resolving connection issues with upstream PDS instances.

We recently released Tap, a tool designed to handle the hard parts of repo synchronization, so you can focus on building your application.

The Challenge of AT Sync​

AT provides two primary mechanisms for syncing repositories: the Relay firehose of real-time events, and direct PDS access to full repos via com.atproto.sync.getRepo. Most applications consume events directly from an upstream Relay or Jetstream, and this presents a new set of challenges:

Backfill is tricky. When you start tracking a new repo, you likely need to fetch at least some of the current records in that repo, possibly the full repo. Neither the Relay nor Jetstream provides a built-in utility for doing this, leaving it as an exercise for the developer. And once you do have the full backfill, you then need to cut over to fetching live events, requiring careful cursor management to make sure you’re not duplicating or missing data.

Recovery is fragile. If your repo gets into a desynchronized state for whatever reason, it’s often hard to recover gracefully. Without some explicit recovery logic, you’ll simply be missing data forever. Cursor management is a tricky affair given the stateless design of the repo event stream making it easier for repos to land in a desynchronized state.

WebSocket connections aren’t always the right solution. Currently, syncing requires maintaining a stateful WebSocket connection, which on its own can be hard to manage and requires custom infrastructure separate from most web application stacks. And most serverless platforms don’t have an easy solution for deploying these types of long-lived connections.

The protocol itself is demanding. Proper AT involves binary encodings, identity resolution, signature verification, and the semantics of Sync 1.1. Jetstream was built to make sync more approachable, but required developers to implicitly trust the output from Jetstream and clients end up trading off the benefits of doing actually authenticated sync.

What Tap Does​

Put simply, Tap is the all-in-one tool for synchronizing subsets of the Atmosphere, or even creating a full copy of the Atmosphere if you need it. Tap is a single-tenant service, written in Go, that subscribes to a Relay and outputs filtered, verified events of repositories. Think of it as a specialized synchronization tool that sits between the full network firehose and your application.

The goal of Tap is to handle the complexity of repo synchronization, with features that manage:

Automatic backfill. When you begin tracking a repository, Tap fetches its complete history before delivering live events

Verification. Tap handles MST integrity checks, identity signatures, and all the cryptographic validation required by the protocol.

Recovery. If a repo becomes desynchronized, Tap automatically resyncs from the authoritative PDS.

Flexible delivery and connection. Choose between a WebSocket connection with acknowledgements, fire-and-forget mode, or webhooks for serverless applications.

Filtered output. Receive events only for the repositories and collections you care about, delivered as simple JSON.

Here’s a basic workflow to track a single repo:

# Start Tap (defaults to port 2480, SQLite backend)
go run ./cmd/tap --disable-acks=true

# Connect to receive events
websocat ws://localhost:2480/channel

# Add repositories to track
curl -X POST http;//localhost:2480/repos/add \
  -H "Content-Type: application/json" \
  -d '{"dids": ["did:plc:ewvi7nxzyoun6zhxrhs64oiz"]}'

Once you add the repository via the repos/add API method, Tap begins backfilling its history automatically. Historical events arrive marked with live:false, followed by any buffered live events that occurred during the backfill, and then new events streamed in time marked with live: true. Your application receives a complete, ordered view of each repository it subscribes to, without gaps.

Delivery Guarantees and Ordering​

Tap takes on the responsibility of cursor management and provides clear delivery guarantees. Events are delivered at least once — if Tap crashes before receiving an event acknowledgment, events will be re-delivered after Tap is restarted.

Ordering guarantees are per-repo rather than global. For live events from a particular repo, Tap will wait until the previous event is acked before sending the next event. When backfilling or resyncing a repo, Tap will send all processed records concurrently. This design allows a historical backfill to run quickly, while ensuring live events maintain strict ordering.

Network Boundaries and Filtering​

Tap can operate in three modes for determining which repos to track:

Dynamically configured (default): Start with zero tracked repositories and add specific DIDs via the repos/add endpoint as needed.

Collection signal mode: Track all repositories that contain at least one record in a specified collection. Set TAP_SIGNAL_COLLECTION=com.example.nsid to automatically discover and track repositories when they create records in your application’s namespace. A common strategy that Atmosphere apps use is to have a “profile” or “declaration” which suggests that a given repo uses that application. For example, Bluesky creates an app.bsky.actor.profile record for all Bluesky users.

Full network mode: Track all repositories across the entire network with TAP_FULL_NETWORK=true. This is a very resource intensive mode and should only be used if you actually want a complete mirror of the entire Atmosphere! Expect terabytes of data and days to weeks for the initial backfill. Our benchmarks suggest Tap can handle 35-45k events per second during a full network backfill.

When to Use Tap​

Tap isn’t meant to replace Jetstream for all use cases – if Jetstream is working fine for you, you may not even need Tap. Jetstream isn’t going anywhere! Since Tap does require more operational overhead, it’s important to understand when you might want to use it with your app:

• Automatic historical backfill when tracking new repositories
• Native webhook support for serverless architectures
• Full network backfill and mirroring capabilities for research and analysis
• Guaranteed delivery with acknowledgement mode
• For tracking specific subsets of repositories without needing to process the full firehose

Tap is particularly valuable for applications that need precise control over which data they sync, applications that scale up repository tracking (such as social discovery tools), and research or infrastructure projects that require a complete mirror of either the entire or some well-defined subset of the Atmosphere.

Full Atmosphere Mirroring as a Principle​

One design goal for Tap worth highlighting is that we consider it a failure of AT as an open network if any third party with adequate resources cannot backfill the entire network. This stands in contrast to centralized, monolithic social networks that would typically only offer a real-time event stream, if any firehose access at all, making any kind of longitudinal analysis difficult (or impossible) without special agreements.

Tap makes this philosophical principle practical. The ability to provision an app and maintain your own complete mirror of the Atmosphere enables research, verification, and entire new classes of applications that require historical context. Adding real-time synchronization enables new services built on both historical depth and current data.

Deployment and Client Integration​

Tap is a single Go binary backed by a relational database for managing repo metadata (SQLite by default for simple deployments or Postgres for larger scale apps). Once received, records are handed off to the app to be indexed.

For TypeScript developers, we’re releasing the @atproto/tap client library to make consuming Tap events in your TypeScript app even easier. We’re planning a “typed indexer” that merges this work with the recent Lexicon SDK rewrite, providing end-to-end type safety from protocol and Lexicon definitions to event stream handers.

Getting Started​

Tap is in beta and we welcome you to try it out with your application and let us know if you run into any issues. We’ve included comprehensive documentation in the project README, a 3-minute deployment guide for Railway, and a TypeScript library showcasing event handlers.

TL;DR: This is a secure way to find people you know by sharing your phone’s contacts with Bluesky. Don’t want to use this? No sweat! A key feature of this design is that it’s double opt-in. If you never use this feature, you will never be findable using your phone number.

Anyone building a social app has to solve the “cold start” problem of bootstrapping dense social graphs. Although Bluesky is a lively place with over 40 million users, we want to connect even more people with their friends.

A common industry solution to friend-discovery is to allow users to upload their phone contact lists to find and follow other users. Unfortunately, these systems have been shown to leak users’ phone numbers. Bluesky is unwilling to ship any feature that exposes PII in that way.

Another concern is consensual usage. The contact upload should only be used for the purpose of finding friends. And then, what if a friend uploads your phone number without you knowing? You shouldn’t show up in these phone-number matches without okaying it yourself.

These issues have meant that until now, we have foregone this common utility.

We believe we have now found a solution to the security risks of contact-list imports, which we will describe in this blog post. We offer this initial write-up to solicit critical feedback from the community. If you have questions or concerns about the design of this scheme, please raise them here.

Requirements​

Handling user contact information has security and privacy implications. If we offer a phone-number matching service, we want it to have these properties:

• Truly “opt-in”. This feature isn’t for everyone, and those who don’t want it can ignore it without hassle.
• Revocable consent. If you change your mind later, all uploaded data can be removed from our servers.
• Specificity. Uploaded data will only be used to help you find your contacts on Bluesky, and for no other purposes.
• Security. We want to keep uploaded phone numbers confidential, even in worst-case scenarios, like our servers getting hacked. Enumeration attacks should likewise not be feasible.

In terms of threat modeling, we want to protect the following information:

• Whether a particular phone number belongs to any Bluesky user at all
• The mapping of specific phone numbers to specific users
• The uploaded contacts list of each user

And we want to protect against the following types of threat actors:

• An attacker with access to the external API interface
• An attacker with unauthorized access to internal Bluesky infrastructure

Threat 1: Enumeration attacks​

The most likely threat is an attacker with access to the external API interface trying to enumerate which phone numbers belong to which users. This might sound easy to defend against, but it’s deceptively tricky.

Imagine we have a magical, perfectly secure “black box” computer system. Users upload their contact lists to this system, and nobody can take them back out again. But, the uploader gets back a list of accounts whose phone numbers were on the list in the form of “suggested follows”.

An attacker uploads a contact list filled with 10,000 random phone numbers. There’s only a finite number of phone numbers in the world (on the order of 8 billion), so eventually their random number list will include the phone number of a legitimate user. The attacker gets a notification telling them that a friend has been found, giving them the opportunity to follow this user.

At this point, the attacker knows that this user’s phone number is somewhere on their list of 10,000 numbers, but they don’t know which one. The attacker creates a new account and re-uploads half of the initial list, 5,000 phone numbers. If the same user is found again, they know their phone number is in that half of the list; if not, it’s the other half. This process is repeated $O(\log n)$ times, and eventually the attacker will have reduced the list of phone numbers to just one - and has successfully inferred the phone number of a random Bluesky user.

By scaling this attack up (say, with a bot farm), the attacker could eventually deduce the phone number of every user. Hashing and rate limiting are often used to “solve” this problem, but because there are finitely many phone numbers in the world, none of these solutions is effective. Not good!

Solving enumeration attacks​

We solve the enumeration attack vector with two steps:

1. Limit discovery to mutual contacts, and
2. Verify phone number ownership before using it in matches.

We have two users, Alice and Bob. They each verify their own phone numbers before uploading their respective contact lists. For either user to get a follow recommendation, Bob’s phone number must be in Alice’s submitted contacts list, and Alice’s phone number must be in Bob’s submitted contacts list. An attacker’s phone number(s) will never be in the contacts list of any legitimate users, so the attacker will never discover any mutual contacts.

Verifying phone number ownership both increases the overall work for an attacker and prevents them from lying about their phone number.

The downside of this scheme is that it will reduce the overall number of matches, since both sides must participate in the contact import, but we think this is a worthwhile trade for protecting people’s privacy. In fact, we would argue that making people feel safe to use the import is pretty critical to gaining any value at all.

In addition to phone number verification, we need to be careful to rate-limit the number of contact numbers that can be imported by an individual user. This prevents undue load on our systems and also ensures that a user cannot attempt to enumerate who has their phone number as a contact.

Threat 2: Database leaks​

Nobody ever wants to experience a data breach, but good security practices mean planning for the worst. The scenario is simple: somehow the service is breached, and the phone number registry is leaked. What now?

Securing against database leaks​

The “obvious” approach is to store the cryptographic hash of each phone number, similarly to password-handling best practices. The problem here is that there are simply not very many possible phone numbers, somewhere on the order of 8 billion. A modern GPU could crack a SHA-256-hashed phone number in under a second.

Brute-force-resistant hash algorithms like Argon2 are a big improvement on that, but 8 billion phone numbers is still a small search space ($2^{33}$), and attackers could construct a Rainbow Table, i.e., a reverse lookup table for every possible phone number. Unlike with password hashing, we cannot use a random salt for each phone number because then we wouldn’t be able to find matches between two users (because mismatched salts would produce different hash outputs).

Since we’re only trying to discover mutual contacts, we can store phone numbers in hashed unordered pairs, such that hash(x, y) == hash(y, x). This order-independence can be achieved by sorting the pair of inputs before hashing them. Assuming there are $2^{33}$ possible phone numbers, then there are $2^{65}$ possible pairs of phone numbers. This makes brute-force attacks much more difficult, and makes Rainbow Tables much less feasible.

Note: The practical brute-force difficulty will be lower than $2^{65}$ because real-world pairs of phone numbers are correlated (e.g. more likely to be from the same region). However, it is strictly more difficult to brute-force a hash of a pair of numbers than to brute-force a hash of either number individually.

Argon2 is a brute-force-resistant hash function, so that’s what we’ll be using (specifically, Argon2id). Since we cannot use a randomized salt (as would be typical of a password hashing use case), we will use a fixed salt across all hashes. This still provides a security benefit because the salt value can be stored separately from the database. In the event of a database compromise, the data would be completely useless to attackers without knowledge of the salt. Using a fixed salt like this is often referred to as a “pepper”

In an ideal world, the pepper would be stored securely inside an HSM. However, HSM support for the Argon2 hash function is not ubiquitous, so it will have to be computed outside of an HSM. To achieve an HSM-equivalent level of overall security, we’ll apply a final HMAC to the Argon2 output, using a secret that is stored in an HSM (using the HMAC as a keyed hash function, rather than a MAC)

Argon2 provides brute-force-resistance, while the final HMAC binds the output hash to a secret stored in an HSM. The overall hash function could be summarised in pseudocode as:

def hash(x, y):  
    return HMAC(Argon2id(sort([x, y]), salt=ARGON_SECRET), key=HMAC_SECRET)

If user A with phone number pA submits their contacts list [pB, pC, pD], then we’d store the following in our database:

hash(pA, pB) -> A
hash(pA, pC) -> A
hash(pA, pD) -> A

(where A is user A’s user ID)

Later, user B submits their contacts list [pE, pA]

The mapping of hash(pB, pE) -> B is stored, but hash(pB, pA) matches with the earlier inserted hash(pA, pB), associated with user A.

At this point, we’ve established that user A and user B are mutual contacts, and we can send follow recommendation notifications to each of them. hash(pA, pB) -> A can be removed from the database now, since it’s no longer needed.

A Complication: Phone Number Recycling​

There’s an annoying feature of phone infrastructure that causes trouble for anyone working with it, including us: after some time, disused phone numbers get reclaimed and assigned to new users. Mobile network operators are forced to do this because otherwise there wouldn’t be enough phone numbers to go around. When you set up a phone with a new SIM, there’s a good chance your “new” phone number used to belong to someone else.

Because of our bidirectional verification system (only discovering mutual contacts), the only way this could impact our system is if the old-owner and the new-owner of a phone number both use Bluesky, both use the contact import feature, and both have a mutual contact (i.e., there is a phone number that is common to both of their contact lists). In such a case, the new-owner would discover the old-owner as a false-positive match (and vice versa).

On the face of it, this is a very unlikely scenario, but it could become more statistically significant if the “mutual contact” is some widely-known phone number, like that of a popular restaurant.

To mitigate this, we will store an additional boolean alongside each hash, denoting whether the phone number of the inserting user is sorted first in the pair or not. So in the earlier example, we would actually store hash(pA, pB) -> (A, True) (assuming pA < pB), and so on.

When user B is submitting their contacts and the matching hash is discovered, the service checks that the boolean matches the expected value. If not, it ignores the match, but updates the stored user ID associated with the hash to match B.

The downside of this approach is that the information in the database would leak statistical information about the relative magnitude of the submitter’s phone number. For example, if user A imports 1000 phone numbers and all the boolean field values are True, we can be confident that user A’s phone number has a low magnitude (i.e., it probably starts with a 1). This is not useful information on its own, but it could be used to accelerate a brute-force attack.

We mitigate this by sorting according to a custom comparison function that does not produce a Total Order. In pseudocode:

def intransitiveCompare(a, b):
    assert(a != b)
    return sha256(a || SEP || b) > sha256(b || SEP || a)

Where || denotes concatenation and SEP is a separator string that is guaranteed not to be contained within the inputs. Note that this comparison function preserves antisymmetry i.e., intransitiveCompare(a, b) == !intransitiveCompare(b, a) for all non-equal values of a and b.

With this comparison function, even a hypothetical phone number of “0” would still be expected to compare greater than a random other phone number with 50% probability. And so, storing these comparison result booleans in the database does not leak any relevant information about the input phone numbers.

Summary​

The following features all work together to construct a contact discovery system that is highly resistant to data misuse:

• Verification of the user’s own phone number
• Only revealing mutual contacts
• Brute-force-resistant pairwise hashing, bound to an HSM-stored secret

We resist API-level enumeration, while also mitigating the consequences of a full database compromise, all while keeping the system (and its deployment) simple. Deployments of this design could be additionally hardened through the use of trusted computing environments, but this is out of scope for our initial plans.

As stated in the introduction, we offer this write-up with the hope of receiving any critiques that might highlight issues with the construction. If you have any concerns to share, please provide them here.

Fortunately, we have more capacity on the team for protocol work than we did even a couple months ago. Expect to see a lot of work start to land in the coming months.

The Atmosphere is Thriving​

Before we dive into what we’re up to, let’s take a moment to celebrate what's happening in the Atmosphere. Our little network is really starting to hit its stride. The energy is incredible and growing by the day! We're seeing new projects pop up constantly, and there's a new level of maturity across the board.

What's really amazing is watching developers help other developers. Development is happening over on Tangled. Devs are sharing updates through Leaflet. Projects like Slices, Microcosm, PDSls, and Graze are making it easier for everyone to build. The AT Protocol Community just announced the second AtmosphereConf this March in Vancouver. This is what decentralized development looks like. Remember: on an open protocol we can just do things.

Big Picture​

We’re close to a big milestone for the protocol. Think of it as the “AT 1.0 moment” (even if we don’t literally call it that). As we wrap up our protocol work on auth scopes and Sync1.1, we believe that we’ve fleshed out a complete set of primitives for working with online identities and public broadcast data. This doesn’t mean that development on new features (i.e. private data) isn’t happening. But we think it’s important that we land and mature the work that we’ve already done around public broadcast data before we move on to the next big chunk of work.

With that in mind, our current focus is on adding a layer of maturity and polish to AT to make it the obvious choice when building public sphere social apps.

We’re pursuing this through three main tracks:

• Developer Experience: Making AT fun and easy to work with. Product-focused devs should be able to build a new social app in a weekend.
• Governance: Ensuring that AT is something larger and longer-lived than Bluesky
• Hard Decentralization: Encouraging the emergence of more stakeholders in the network for a more resilient Atmosphere

Developer Experience​

We know there are rough edges in the developer experience, but we’ve been hard pressed to find the time to smooth them out while also adding new protocol functionality. With a bit of polish, we’re confident that AT can be fun and easy to build on.

OAuth Cookbooks & Tutorials​

OAuth is one of the trickiest parts of building on the protocol — tricky enough that Nick Gerakines is selling out courses on how to do it! OAuth in general is unfortunately complicated in itself, and the decentralized bits of AT only add to that complexity, but that doesn’t mean that it needs to be unapproachable.

We're taking inspiration from Auth0's approach and putting together some comprehensive examples and tutorials that we hope will make getting started with OAuth way easier. Expect to see these published in the next week or two.

We recently wrapped up our dev work on auth scopes and permission sets. Expect an announcement and guides on how to make use of those shortly.

Lexicon SDK​

Lexicons are the coordination points for schematic data in the network. As more and more applications are publishing new Lexicons, it’s important that developers can actually make use of them to build native integrations between apps.

The current Lexicon SDK was really a prototype that ended up living on way longer than it probably should have. It doesn’t have to be like this. We’re putting together a new SDK that takes inspiration from Buf’s approach to codegen and schema management. This SDK should make it a breeze to pull in schemas from across the Atmosphere into your application.

Sync Tool​

Repository synchronization is at the core of AT. Every app has to do it. And unfortunately it’s difficult to do and even more difficult to do correctly.

We’re continuing to roll out Sync1.1 on the producer (Relay) side, but a fully spec-compliant consumer for it still doesn’t exist. Most developers currently rely on Jetstream to drive their application, but that only helps with live data. There’s no easy way to perform backfill and sync full subsets of the network.

We’re working on a tool that should smooth over the tricky parts of doing sync in AT. This tool will likely take the form of a small service you can run that handles backfill, cutover, cursor management, Sync1.1, and dynamic filtering by DID and collection. That service will do all the tricky stuff and then hand off a nice clean set of record-level operations to your application. It will offer several interfaces including a websocket interface and the ability to translate the firehose into webhooks — meaning AT can work with serverless architectures!

Website​

We’re going to be giving atproto.com a facelift in the coming months. You can expect all the work mentioned above to make its appearance there. We’ll also be overhauling the information architecture and publishing new tutorials and guides to make AT more approachable.

Governance​

As the Atmosphere matures and more devs are putting time and resources into building companies/projects in the ecosystem, we believe it’s our responsibility to ensure that the protocol has a neutral long-term governance structure around it. The governance of the protocol should outlive Bluesky and be resilient to shifts in the incentive structure that could compromise a future Bluesky PBC.

To that end, we have 3 major developments:

Patent Pledge​

We announced our Patent Non-Aggression Pledge at the start of October. Our SDKs and reference implementations are all open source and licensed under permissive software licenses. This patent pledge takes it a step further and establishes additional assurance around patent rights.

Independent PLC Organization​

We announced in September that we were working to establish an independent organization to operate the PLC (Public Ledger of Credentials) Directory. PLC is the most common identity method for accounts in the Atmosphere.

Currently this directory is owned and operated by Bluesky PBC. We’re working to establish a Swiss association that will operate the directory and own all assets related to PLC (such as the domain name registration). We’re working with lawyers now to get this done right. Expect an update soon on our progress here.

IETF​

We’re hoping to take pieces of AT to the IETF. We've submitted Internet Drafts on the IETF Datatracker and established a mailing list. We're hoping to establish a working group and towards that end, have requested a Birds of a Feather session in Montreal the first week of November. Some folks from the community will be attending and getting together informally. Leave a comment in the community forum if you’ll be around. If you're interested in shaping the future of the protocol at the standards level, we encourage you to get involved!

Hard Decentralization​

Hard decentralization refers to the emergence of a resilient and multi-stakeholder Atmosphere that relies less on Bluesky PBC’s existence. There's some overlap with the other two goals here. Improving things like sync make it easier to run alternate infrastructure like Relays and Appviews, and our governance work should help build confidence that the protocol is a genuine public good that’s larger than Bluesky.

To support the goal of hard decentralization, we're also tackling some specific technical challenges.

Improving Non-Bluesky PDS Hosting​

The decentralization guarantees of AT come from the locked-open substrate of data hosted by Personal Data Servers (PDSes). One of our current goals to increase the resilience of the network is to encourage more non-Bluesky PDS hosting.

We recently enabled account migration back to bsky.social. We hope this will give users the confidence to experiment with other hosts in the network, knowing they can always migrate back if they need to. Already we’re seeing an uptick in users posting from non-Bluesky PDSes.

Some developers in the network have launched account migration tools that make it easier for non-technical users to migrate between hosts. Examples include PDS MOOver and Tektite. We believe that the next step is to introduce an account migration UI into the PDS itself.

We also intend to make running a PDS more approachable for mid-size hosts. This includes adding auto-scaling rate limits to the Relay reference implementation so that hosts can scale up organically without needing permission or approval. We’re also looking at ways to improve the PDS distribution to make it easier to run and administer with thousands of users.

Technical Improvements to PLC​

While we’re working to move PLC out into an independent organization, we’re also planning some technical improvements to PLC to make it more auditable.

Specifically, we want to make it easier to mirror the directory. We intend to introduce a new WebSocket route to the directory that allows new PLC operations to go out in real time. With this, we’ll also publish a PLC Mirror Service reference implementation. This improves both the auditability of PLC and has operational benefits for developers that may wish to run a PLC Mirror closer to their infrastructure.

There are also some legacy non-spec-compliant operations in the directory that make it difficult to write an alternate implementation of PLC that interoperates with the directory. Upon investigation, these have all been traced back to developers probing the PLC system itself, not regular network accounts. We plan to clean those up and harden the directory against similar operations.

This work is building towards the introduction of Transparency Logs (tlogs). Check out Sunlight to see where we’re heading. This probably won’t land in the next six months, but it’s the clear next step for improving trust in PLC.

Alternate Infrastructure​

We’re excited to see that more and more devs are experimenting with running alternate infrastructure in the network. Blacksky currently runs a full-network relay (using the rksy-relay implementation!), and is working on a full-network Bluesky appview. Futur showed us all that it was possible with Zeppelin, a full network appview that is now decommissioned. And Red Dwarf is a new Bluesky client that doesn’t use an Appview but rather drives the experience via direct calls to the PDS and generic indices provided by Microcosm.

Please reach out to us if you’re working on running alternate infrastructure. We’re eager to lend a hand.

Private Data​

We believe that group-private data is absolutely necessary for the long-term success of the protocol and the Atmosphere. Every day, we wish that we had this figured out and developed already. But as mentioned earlier, we believe that we need to land and mature the existing protocol around public broadcast data before we move on to the next big chunk of work.

We continue to have internal discussions around private data. Paul shared some leaflets that give a sense of the approaches that we’re considering and the rubric by which we’re judging them. The AT Protocol Community is also coordinating a Private Data Working Group to explore some designs for how the system could work.

In the meantime, if you’re building an Atmosphere app, please don’t let the lack of a private data protocol prevent you from building the features that you need to build. Our advice is to publish identities and public data through AT and store any private data that you need on your own server. The semantics of private data will likely look very similar to public data (CRUD operations over a key-value store of typed records, sorted by collection), so if you want to get ahead of the ball, model your data accordingly.

For E2EE DMs, Germ has put together a lovely app built on MLS (Messaging Layer Security) with an AT integration that’s in beta.

Keep up with the Atmosphere​

The Atmosphere is getting bigger every day, and it’s starting to get tough to keep up with everything that’s happening! Here are some ways to stay in the loop:

• Follow the official @atproto.com Bluesky account.
• Follow the community-run @atprotocol.dev account
• Contribute to or read discussions on Github
• Check out the Atmosphere report, an independent newsletter (now on Leaflet!)

Today, we’re removing that restriction and allowing returning users to migrate back to the Bluesky PDS. We hope this gives more users the confidence to explore other PDSs in the network, knowing they can return if needed. That being said, account migration remains a potentially destructive operation, so users should be aware of the risks before migrating between hosts.

This does not yet allow users who have never had a bsky.social account to migrate to our PDS. The migration flow works the same as described in the Account Migration documentation, but instead of creating a new account, you’ll log into your existing bsky.social account, import your repo, and reactivate. The Bluesky PDS will automatically handle the diff and index any changes to the repository since you were last active on bsky.social.

We want to see more PDSs in the network and more users on non-Bluesky PDSs. Future work includes an account migration tool hosted at bsky.social, improvements to the PDS distribution - especially for mid-sized deployments, and auto-scaling rate limits at the Relay.

In particular, we’ve submitted two Internet-Drafts:

Authenticated Transfer Repository and Synchronization: a proposed standard that specifies the AT repository format and sync semantics

Authenticated Transfer: Architecture Overview: an informational draft that goes over the architecture of the broader network and describes how the repository fits into it.

Just today, we were approved for a Birds of a Feather (BOF) session at IETF 124 in Montreal from November 1-7. Details on the BOF can be found Here.

A BOF is a part of the formal IETF process for forming a working group. It involves pulling together interested parties in order to determine if the IETF is a good fit for chartering a working group to work on a particular technology.

This is a “non-working group forming” BOF. The intention is to get feedback on both the charter for the WG and the drafts that we’ve submitted. If things go well, then we’d likely do an interim BOF between IETF 124 and 125 to actually form a working group.

What We’re Planning to Bring (and What We’re Not)​

We’re specifically focusing on the repository and sync protocol. We’re not planning to bring Lexicon, AT’s particular OAuth profile, Auth scopes, PLC, the handle system, or other AT components to the IETF right now.

A few reasons for the narrow scope:

• Working groups need focused charters, especially when bringing a new protocol to the IETF
• The repo and sync protocol is the most foundational part of AT and is therefore the most impactful to have under neutral governance
• The repo and sync protocol is the most “IETF-flavored” part of the stack, especially with its reliance on CBOR and WebSockets (both IETF specifications)

If things go well for both sides, we may consider rechartering the working group later. Whether or not a working group forms will not impact how new AT features such as private state are designed or rolled out.

Why IETF?​

This is part of an ongoing effort to mature the governance of AT. (See also: the parallel work that we’re pushing forward on moving PLC to an independent organization)

We want AT to have a neutral long-term home, and the IETF seems like a natural fit for several reasons. It’s the home of many internet protocols that you know and use every day: HTTP, TLS, SMTP, OAuth, WebSockets, and many others. The IETF has an open, consensus-driven process that anyone can participate in. And importantly, the IETF cares about both the decentralization of the internet while also keeping it functioning well in practice. This balance between idealism and pragmatism matches how we’ve approached the challenges of building a large-scale decentralized social networking protocol.

What You Can Do​

Read the drafts! Take a look at what we’ve submitted and see what you think. We have a public GitHub repo where you can comment on the drafts and provide feedback. We’re hoping to iterate on the drafts at least once before the BOF and already have a few issues noted that we need to address.

If you’re planning to attend IETF 124 in Montreal, let us know! We’d love to connect with folks who are interested in this work.

Until now.

As the next step of maturing governance of the PLC identity system, Bluesky Social PBC is supporting the creation of an independent organization to operate the directory. The organization will set policies and rate-limits, hold any related intellectual property, and coordinate future evolution and development of the system. While Bluesky and the AT network are the largest use case for the PLC system today, it is a general-purpose technology, and will be developed and operated as a vendor- and application-neutral public good.

After considering several jurisdictions, legal structures, and potential parent organizations, the new entity will form as a Swiss Association. In a period of international uncertainty around Internet governance, Switzerland provides a credibly neutral and stable global home. This entity will have the independence and flexibility to transform itself into another legal structure as it evolves. Initial board members and other logistical details are still being finalized.

We do not expect this organization to be the final governance structure for PLC, and we do not expect a single global directory to be the final technical architecture for the system. But as the AT network grows and diversifies, it becomes increasingly important for the identity system to have a clear path toward independence. It is also our hope that a more neutral and independent PLC identity system will find use by other projects in the broader open web ecosystem.

You can read more about the PLC identity system at web.plc.directory, and more about its use in the AT network in the Identity Protocol Documentation.

Email Transitional Scope​

One of the few gaps between password authentication and the initial ("transitional") set of OAuth scopes was access to an account's email address (and email confirmation status). We have implemented a new OAuth scope, transition:email that clients can request for access to the account's email. The email itself can be accessed via the com.atproto.server.getSession endpoint on the PDS, using an OAuth access token.

Public versus Confidential Clients​

The difference between "public" and "confidential" OAuth clients is one of the more complex tradeoffs for application developers. Tradeoffs in security, privacy, trust, application architecture, session lifetime, and user experience are all entangled.

We have a new essay that tries to untangle this knot, explaining the security concerns and trade-offs involved: "OAuth Client Security in the Atmosphere".

We also have a proposal to enable a new architecture: "Client Assertion Backend for Browser-based Applications". In this architecture, in-browser web apps (SPAs) would communicate with a simple and generic backend server to generate client attestations for token requests. The backend could "veto" client sessions (to address security incidents), but would not directly mediate token requests. This would be distinct from the Token Mediating Backend (TMB) architecture. Note that with both architectures, the server can not hijack the session or make "offline" authenticated requests on the user's behalf, assuming the DPoP keys are held securely on-device by the web app.

Session Lifetimes​

Many developers are encountering friction with OAuth session lifetimes, especially using in-browser "public" clients. The existing lifetime limitations mean that un-refreshed sessions would time out after just two days.

There will always be shorter session lifetimes for public clients, but we think the session lifetimes can be increased without a significant security trade-off. We are planning to roll out the following session lifetimes:

• Public Clients
  • increase overall session lifetime (after which tokens can not be refreshed) from one week to two weeks
  • increase the lifetime of individual refresh tokens from two days to two weeks (the same as the overall session lifetime; refreshing tokens does not extend the overall session)
• Confidential Clients
  • increase the lifetime of individual refresh tokens from one month to three months (if tokens are not refreshed, the session ends before the overall session lifetime limit)
  • increase overall session lifetime (assuming tokens are refreshed) from one year to two years; including existing sessions

Remember that the maximum lifetime of OAuth access tokens is much shorter: the specification recommends a limit of 30 minutes, and the reference implementation uses 15 minutes.

There is a tracking issue here: bluesky-social/atproto#3883

Auth Scopes​

Protocol design on Auth Scopes is wrapping up. This functionality will allow app developers to request granular permissions to specific atproto resources, like records (by NSID), remote API endpoints (using service proxying), account hosting status, and identity updates. The system will also allow Lexicon designers to define higher-level "bundles" of permissions for atproto resources in the same NSID namespace, to make end-user permission requests simpler.

You can learn more by reading an earlier draft of this protocol feature from March 2025.

We will begin server-side implementation and integration work on this feature soon after a final proposal is published.

Other Changes and Bug Fixes​

The DPoP JWTs created by the @atproto/oauth-client TypeScript package incorrectly included query parameters as part of the htu request URL field, which goes against the DPoP specification. The client package has been fixed as of version 0.3.18.

The Authorization Server used to require a DPoP proof during Pushed Authorization Requests when the parameters contained a dpop_jkt. This was not valid per spec and is thus no longer the case.

The OAuth Client Implementation Guide previously instructed developers to include the Auth Server Issuer (host URL) in DPoP JWTs included on authenticated requests to the PDS, in the iss field. The example Python code and TypeScript OAuth client implementation also included this field. This is not required by the DPoP specification, and should not be implemented by clients or SDKs. We have updated the guide and implementations to remove this field.

The TypeScript OAuth client SDK was incorrectly sending HTTP POST requests using JSON bodies, instead of form-encoding (the server was flexible to either encoding). This has been corrected, and requests that should use form-encoding (like PAR) now do.

A couple of subtle bugs with the TypeScript OAuth client SDK and DPoP nonce caching have been fixed.

If a client app redirects to the reference PDS without a login_hint, and the user was already logged in, the auth flow now proceeds directly to an account selector, instead of displaying a "Create Account or Log In" page. This makes the flow smoother and reduces a click.

The client app redirect URL no longer needs to be on the same host origin as the client metadata document.

Deprecation notice​

The following invalid behaviors were detected in our reference implementation. As of now, these are considered as deprecated and might change in the future. Make sure you use the latest version of our SDKs, and if you implemented your own, please check that it is compliant.

• DPoP proofs that contain a query or fragment in the htu claim should be rejected.

• JWT for the client attestation using the private_key_jwt authentication method MUST contain an exp claim. This is currently not enforced by the reference implementation.

• When performing a Pushed Authorization Request, the client must either provide a dpop_jkt authorization request parameter, or provide a DPoP proof header. The reference implementation does not enforce this, allowing the DPoP proof header to be provided only during the token exchange.

Remaining Limitations​

The account management interface on the reference PDS implementation allows revoking OAuth client sessions. On a free-standing PDS, this has an immediate effect: both access and refresh tokens stop working immediately. But the Bluesky-hosted PDS instances ("mushroom servers") use an "entryway" service as an OAuth Auth Server. A side-effect of this is that revoking client sessions could take up to 15 minutes to take effect: refresh tokens immediately stop working, but access tokens do not. This is a relatively common design tradeoff in auth systems involving many servers but is not currently well communicated in the interface.

The TypeScript OAuth Authorization server implementation (`@atproto/oauth-provider`) currently does not correctly validate that the initial client attestation signing key is present in the client metadata document over the full lifetime of an OAuth session. Instead, it simply validates each attestation (eg, on token refreshes) against the current JWKs in the client metadata. In other words, if a client removes a public key from the JWK set in the client metadata document, the auth server is supposed to reject future token refresh requests for sessions that started by using that specific key; but the current implementation does not do this. Similarly, our OAuth client implementation did not enforce the use of the same key whenever refreshing sessions. The tracking issue for this is https://github.com/bluesky-social/atproto/pull/3847. Note that fixing this bug might result in sessions becoming invalid if clients do not adapt their implementation to use the same key over the session’s lifetime. Clients only using a single key should not be impacted.

We recently shipped new functionality to the PDS reference implementation (and Bluesky's hosting service) which provides a web interface to create and manage accounts directly on the PDS itself. This post describes the new functionality, and what this means for independent app developers and service providers.

OAuth Account Sign-Up Flow​

The PDS reference implementation has included a web interface for authorization flows since September 2024, which allowed users to login to OAuth client apps. However, users needed to already have an account. One work-around was to have users create an account using the Bluesky app first, but this was a very awkward onboarding flow, and gave Bluesky Social (the app) an outsized role in the ecosystem.

The PDS now allows users to create new accounts during the OAuth flow where client apps can initiate an auth request to a specific PDS instance without having an account identifier for the user. When they are redirected to the PDS (the OAuth "Auth Server"), they are given the option to create a new account. This includes the full account creation flow: selecting a handle, agreeing to any terms of service, and passing any anti-abuse checks. The user is then prompted to complete the client app authorization flow and will be redirected back to the app once they have signed up.

Web interface for "pds.example.com", with buttons "Create a new account", "Sign in", and "Cancel". There is a language-selection drop-down menu in the lower right corner.

"Create Account" web interface, prompting user to "Choose a username" as "Step 1 of 2"

On an independent or self-hosted PDS instance, this process is unbranded and independent of Bluesky Social by default. PDS operators can customize the interface with the PDS_LOGO_URL, PDS_SERVICE_NAME, and other configuration variables. When using Bluesky's hosting service (bsky.social), the flow will indicate that Bluesky (the company) is the hosting provider.

You can experiment with this flow by entering a PDS hostname (or https://bsky.social) using the Python Flask OAuth demo at: https://oauth-flask.demo.bsky.dev/oauth/login

Account Management Interface​

The reference implementation's account management interface is at the path /account. This specific path does not need to be a protocol norm or requirement, and we intend to make any account management URL machine-discoverable in the future. For an independent PDS at pds.example.com, the full URL would be https://pds.example.com/account. For an account hosted on Bluesky's hosting service, the URL is https://bsky.social/account.

Web Sign-In interface, asking for "Identifier" and "Password". There is a "Forgot Password?" link, and a language selector drop-down.

Users will already be signed in if they have done an OAuth approval flow for an app. If not, they should sign in using their full account password (not an app password). Multiple accounts on the same PDS can be signed in at the same time, with an account selector dialog.

Web interface for "Your Account". Shows a list of "Connected apps" (Statusphere React App and OAuth Flask Backend Demo), with "Revoke access" buttons for each. Also has a list of "My Devices", showing "Linux - Firefox" and "Linux - Chrome", the latter indicated as "This device", both with "Sign Out" buttons.

The account management view currently allows users to view authorized OAuth applications, and a list of devices/browsers which are signed in to the host itself. It does not show password-based auth sessions to apps.

We expect to implement more functionality in this interface in the future. For example, email updates, password changes, and account deactivation, deletion, or migration actions are all application-agnostic and could be implemented here. Additional 2FA methods (such as passkeys, OTP, or hardware dongles) could all be configured through this interface.

The specific design and features for account management are largely left to implementations, and are not specified or required by the protocol. We expect that there will be a common overlap in features provided, but PDS hosting providers are free to innovate and differentiate. Providers might bundle other network services (such as email, calendaring, or web hosting), or implement atproto hosting as a component of existing services (such as blog hosting).

Other OAuth Progress​

The protocol team released an initial Auth Scopes design sketch in March, and expects to have a working version in the network before long. In the meanwhile, we are planning to introduce an email-specific transitional OAuth scope, which will let OAuth clients access account email addresses and email verification status.

App developers in the ecosystem have been implementing atproto OAuth clients of both the "public" and "confidential" types. While the "public" client type can be easier to implement, we need to emphasize that long-lived auth sessions (eg, more than a couple days or weeks before re-authentication) are only possible with "confidential" clients. This requires some degree of auth offload to an app server which holds cryptographic private keys and signs token refresh requests. You can read more about this in the "Confidential Client Authentication" section of the atproto OAuth specifications.

As always, we encourage you to check in on both official and community channels for updates on AT Protocol. An increasing share of development is happening out in the ecosystem, with more projects and organizations getting started by the week:

• @atproto.com account on Bluesky
• Discussions on Github
• The Fediverse Report includes weekly AT Protocol coverage
• atprotocol.dev Working Groups

Updated Relay Implementation​

The new version of the relay is available now in the indigo go git repository, named simply relay. It is a fork and refactor of the bigsky relay implementation, and several components have stuck around. The biggest change is that the relay no longer mirrors full repository data for all accounts in the network: sync v1.1 relays are now "non-archival". We have actually been operating bsky.network with a patched non-archival version of bigsky for some time, but this update includes a number of other changes:

• the new #sync message type is supported
• the deprecated #handle, #migration, and #tombstone message types are fully removed (they were replaced with #identity and #account)
• the sync v1.1 changes to the #commit message schema are supported
• message validation and data limits updated to align with written specification
• account hosting status and lifecycle updated to match written specification
• #commit messages validated with MST inversion (controlled by "lenient mode" flag)
• new com.atproto.sync.listHosts endpoint to enumerate subscribed PDS instances
• com.atproto.sync.getRepo endpoint implemented as HTTP redirect to PDS instance
• simplified configuration, operation, and SQL database schemas

The relay can aggregate firehose messages for the full atproto network on a relatively small and inexpensive server, even with signature validation and MST inversion of every message on the firehose. The relay can handle thousands of messages per second using on the order of 2 vCPU cores, 12 GByte of RAM, and 30 Mbps of inbound and outbound bandwidth. Disk storage depends on the length of the configurable "backfill window." (A 24-hour backfill currently requires a couple hundred GByte of disk.)

Staged Rollout​

The end goal is to upgrade the bsky.network relay instance and for all active PDS hosts in the network to emit valid Sync v1.1 messages. We are rolling this out in stages to ensure that no active services in the network break.

We have two new production relay instances:

• relay1.us-west.bsky.network
• relay1.us-east.bsky.network

These both run the new relay implementation and attempt to crawl the entire atproto network. The two instances are operationally isolated and subscribe to PDS instances separately, though configuration will be synchronized between them periodically. They have sequence numbers which started at 20000000000 (20 billion) to distinguish them from the current bsky.network relay, which has a sequence around 8400000000 (8.4 billion). Note that the two new relays are independent from each other, have different sequence numbers, and will have different ordering of messages. Clients subscribing to these public hostnames will technically connect to rainbow daemons which fan-out the firehose.

We encourage service operators and protocol developers to experiment with these new instances. They are running in "lenient" MST validation mode and should work with existing PDS instances and implementations. PDS hosts can direct requestCrawl API requests to them. If hosts don't show up in listHosts, try making some repo updates to emit new #commit messages. Bluesky intends to operate both of these relays, at these hostnames, for the foreseeable future.

For the second stage of the roll-out (in the near future), we will update the bsky.network domain name to point at one of these new relay instances. If all goes smoothly, firehose consumers will simply start receiving events with a higher sequence number.

As a final stage, we will re-configure both instances to enforce MST inversion more strictly and drop #commit messages which fail validation. To ensure this does not break active hosts and accounts in the network, the relays currently attempt strict validation on every message and log failures (even if they are passed through on the firehose in "lenient" mode). We will monitor the logs and work with PDS operators who have not upgraded. If necessary, we can "ratchet" validation by host, meaning that new hosts joining the network are required to validate but specific existing hosts are given more time to update.

Other Sync v1.1 Progress​

We have a few resources for PDS developers implementing Sync v1.1:

• the goat CLI tool has several --verify flags on firehose consumer command, which can be pointed at a local PDS instance for debugging
• likewise the relay implementation can be running locally (eg, using sqlite) and pointed at a local PDS instance
• interoperability test data in the bluesky-social/atproto-interop-tests git repository, both for MST implementation details, and for commit proofs
• the written specifications have not been updated yet, but the proposal doc is thorough and will be the basis for updated specs

While not strictly required by the protocol, both of the new relay hostnames support the com.atproto.sync.listReposByCollection endpoint (technically via the collectiondir microservice, not part of the relay itself). This endpoint makes it much more efficient to discover which accounts in the network have records of a given type and is particularly helpful for developers building new Lexicons and working with data independent of the app.bsky.* namespace.

The need for ordered repository CAR file exports has become more clear, and an early implementation was completed for the PDS reference implementation. That implementation is not performant enough to merge yet, and it may be some time before ordered CAR files are a norm in the network. The exact ordering also needs to be described more formally to ensure interoperation. Work has not yet started on the "partial synchronization" variant of getRepo, which will allow fetching a subset of the repository.

Metamorphosis is the process where a caterpillar forms a chrysalis, liquifies its own body, and emerges as an imago, or butterfly. It is a pretty nifty trick. Anyways, trees are budding, Lexicons are blossoming, spring is happening, and it is time for an update to the AT Protocol roadmap.

We recently summarized progress on the protocol in 2024. This blog post will be forward looking, covering our protocol goals for the next 6-7 months. As a high-level summary:

• Updates to the relay, firehose, and public repo sync semantics (Sync v1.1) are starting to roll out
• Design work on Auth Scopes has started, which will improve atproto OAuth
• PDS will get a web interface for generic account management and signup
• Shared data (eg, group privacy) will likely be the next major protocol component, with E2EE DMs following that

We also have a quick section on deprecated developer patterns; please give those a look!

Sync v1.1​

We are iterating on the core public data synchronization components of the protocol. Relays will become much cheaper to operate, and we’re clarifying the process for fully validating the firehose. The full proposal gets into all the details, but to summarize:

• Efficient mechanism for validating MST operations in individual repository commits ("inductive firehose")
• Adding a new #sync message type, and removing the tooBig flag on commits
• New desynchronized and throttled account statuses, to communicate temporary failures
• New com.atproto.sync.listReposByCollection endpoint to help with backfill

Auth Scopes​

We are updating OAuth for AT Protocol with a way to request and grant granular permissions. For example, it should be possible to give a client permission to read and write posts on Bluesky, but not insert arbitrary block records or access DMs. This is obviously important for user control, privacy, and account security. The system will allow application designers to declare their own auth scopes, as part of the Lexicon system. PDS implementations will be able to enforce these permissions in an interoperable way, at runtime. We will share more details soon.

In addition to completing OAuth for existing apps, Auth Scopes will be necessary for upcoming protocol features, like group-private data and on-protocol DMs.

PDS Account Management​

More and more folks are building independent apps on atproto. While they can use OAuth to authenticate users from any PDS instance, account signup is more complicated. In theory it is possible to implement account creation using the com.atproto.* Lexicons, but in practice this is difficult (or impossible) to implement in independent apps, because of anti-bot measures. This results in developers directing new users to sign up with Bluesky, which is a bad user experience, and conflates having an account on the AT Protocol with having a Bluesky account.

To improve this situation, we are implementing a web interface in the PDS reference distribution which will give users a less-branded account sign-up experience. The PDS technically already has a web interface, used for the OAuth authorization flow, and this simply extends that. Over time, we expect the web interface to provide generic account management capabilities, such as password recovery flows, additional 2FA mechanisms, management of active auth sessions, account deactivation, etc.

The details of the web interface will be implementation-specific. Other PDS implementations might provide different functionality, or make different design choices.

Privately Shared Data and E2EE DMs​

We believe that robust support for group-private data will be necessary for the long-term success of the protocol (and for apps built on the protocol). Similarly, the ability to share private content with a specific group or audience continues to be a top feature request for both the AT Protocol and the Bluesky app. Just as we’re currently doing with public conversation on the Bluesky app and the AT Protocol, we also want to co-design the protocol specification for private data in tandem with specific real-world product features: this results in better outcomes for both. Designing for privacy is pretty different from designing for global broadcast, and we think the data architecture will probably look pretty different from the MST + firehose system.

Shared data will depend on Auth Scopes, and we don't expect to start design work until that is complete.

Looking forward, we continue to have plans to implement on-protocol DMs and E2EE group chat. However, we don’t expect to start work on this until after shared data is implemented. Meanwhile, there has been exciting progress in the broader tech world around the Messaging Layer Security (MLS) standard, and we are optimistic that we will be able to build on reusable components and design patterns when the time comes. It is also possible (and exciting!) that the atproto dev community will experiment and build E2EE chat apps off-protocol before there is an official specification.

Deprecations​

There are a few protocol features and API endpoints which were supported in early days of atproto development. They have been deprecated for some time, but have continued to function. As the protocol stabilizes, we want to ensure developers are building against the current protocol, and will start to remove this functionality more aggressively.

A simple deprecation are the #tombstone, and #handle, and #migrate events on the firehose. These were replaced with #identity and #account early last year (2024), and have been deprecated since then. We will remove them from the atproto Lexicons entirely soon.

Client apps should resolve user login identifiers (handles or DIDs) to PDS instances, and should not hardcode the bsky.social domain for API requests. In the early days, all API requests could be made to this server, and we have continued to proxy requests to avoid breakage. Most clients and SDKs have been updated, and we may stop proxying in the near future.

When making proxied requests to a PDS, clients can specify a remote service to forward to via the atproto-proxy header. To date, the reference PDS implementation has automatically forwarded app.bsky.* endpoints to the Bluesky API server (api.bsky.app). No other services or Lexicon namespaces in the network have this sort of default forwarding. To keep the network more provider-neutral, clients should not rely on this default, and should always specify a service in the proxy header. The service DID reference for the Bluesky AppView is did:web:api.bsky.app#bsky_appview; you can see more example service DIDs in the API Hosts and Auth docs.

Keep up with Ecosystem​

The AT Protocol developer ecosystem continues to grow at a fast pace, with more developers launching new projects and organizations by the week. Here are some ways to stay updated or get involved:

• Follow the @atproto.com Bluesky account
• Contribute to or read discussions on Github
• Subscribe to the The Fediverse Report, an independent newsletter that includes weekly AT Protocol coverage
• Attend atprotocol.dev’s independently-run Tech Talks or the ATmosphere Conference this month in Seattle

This release is a big step forward, significantly improving the type safety of our @atproto/api package. Let’s take a look at the highlights:

• Lexicon derived interfaces now have an explicitly defined $type property, allowing proper discrimination of unions.
• Lexicon derived is* utility methods no longer unsafely type cast their input.
• Lexicon derived validate* utility methods now return a more precise type.

Context​

Atproto is an "open protocol" which means a lot of things. One of these things is that the data structures handled through the protocol are extensible. Lexicons (which is the syntax used to define the schema of the data structures) can be used to describe placeholders where arbitrary data types (defined through third-party Lexicons) can be used.

An example of such a placeholder exists in the Lexicon definition of a Bluesky post (app.bsky.feed.post), which enables posts to have an embed property defined as follows:

  "embed": {
    "type": "union",
    "refs": [
      "app.bsky.embed.images",
      "app.bsky.embed.video",
      "app.bsky.embed.external",
      "app.bsky.embed.record",
      "app.bsky.embed.recordWithMedia",
    ]
  }

The type of the embed property is what is called an "open union". It means that the embed field can basically contain anything, although we usually expect it to be one of the known types defined in the refs array of the Lexicon schema (an image, a video, a link, or another post).

Systems consuming Bluesky posts need to be able to determine what type of embed they are dealing with. This is where the $type property comes in. This property allows systems to uniquely determine the Lexicon schema that must be used to interpret the data, and it must be provided everywhere a union is expected. For example, a post with a video would look like this:

{
  "text": "Hey, check this out!",
  "createdAt": "2021-09-01T12:34:56Z",
  "embed": {
    "$type": "app.bsky.embed.video",
    "video": {
      /* reference to the video file, omitted for brevity */
    }
  }
}

Since embed is an open union, it can be used to store anything. For example, a post with a calendar event embed could look like this:

{
  "text": "Hey, check this out!",
  "createdAt": "2021-09-01T12:34:56Z",
  "embed": {
    "$type": "com.example.calendar.event",
    "eventName": "Party at my house",
    "eventDate": "2021-09-01T12:34:56Z"
  }
}

Revamped TypeScript interfaces​

In order to facilitate working with the Bluesky API, we provide TypeScript interfaces generated from the lexicons (using a tool called lex-cli). These interfaces are made available through the @atproto/api package.

For historical reasons, these generated types were missing the $type property. The interface for the app.bsky.embed.video, for example, used to look like this:

export interface Main {
  video: BlobRef
  captions?: Caption[]
  alt?: string
  aspectRatio?: AppBskyEmbedDefs.AspectRatio
  [k: string]: unknown
}

Because the $type property is missing from that interface, developers could write invalid code, without getting an error from TypeScript:

import { AppBskyFeedPost } from '@atproto/api'

const myPost: AppBskyFeedPost.Main = {
  text: 'Hey, check this out!',
  createdAt: '2021-09-01T12:34:56Z',
  embed: {
    // Notice how we are missing the `$type` property
    // here. TypeScript did not complain about this.

    video: {
      /* reference to the video file, omitted for brevity */
    },
  },
}

Similarly, a Bluesky post’s embed property was previously typed like this:

export interface Record {
  // ...
  embed?:
    | AppBskyEmbedImages.Main
    | AppBskyEmbedVideo.Main
    | AppBskyEmbedExternal.Main
    | AppBskyEmbedRecord.Main
    | AppBskyEmbedRecordWithMedia.Main
    | { $type: string; [k: string]: unknown }
}

It was therefore possible to create a post with a completely invalid "video" embed, and still get no error from the type system:

import { AppBskyFeedPost } from '@atproto/api'

const myPost: AppBskyFeedPost.Main = {
  text: 'Hey, check this out!',
  createdAt: '2021-09-01T12:34:56Z',

  // This is an invalid embed, but TypeScript
  // does not complain.
  embed: {
    $type: 'app.bsky.embed.video',
    video: 43,
  },
}

We have fixed these issues by making the $type property in the generated interfaces explicit. The app.bsky.embed.video interface now looks like this:

export interface Main {
  $type?: 'app.bsky.embed.video'
  video: BlobRef
  captions?: Caption[]
  alt?: string
  aspectRatio?: AppBskyEmbedDefs.AspectRatio
}

Notice how the $type property is defined as optional (?:) here. This is due to the fact that the schema definitions are not always used from open unions. In some cases, a particular schema can be referenced from another schema (using a "type": "ref"). In those cases, there will be no ambiguity as to how the data should be interpreted.

For example, a "Bluesky Like" (app.bsky.feed.like) defines the following properties in its schema:

  "properties": {
    "createdAt": { "type": "string", "format": "datetime" },
    "subject": { "type": "ref", "ref": "com.atproto.repo.strongRef" }
  },

As can be seen, the subject property is defined as a reference to a com.atproto.repo.strongRef object. In this case, there is no ambiguity as to how the subject of a like should be interpreted, and the $type property is not needed.

const like: AppBskyFeedLike.Record = {
  $type: 'app.bsky.feed.like',
  createdAt: '2021-09-01T12:34:56Z',
  subject: {
    // No `$type` property needed here
    uri: 'at://did:plc:123/app.bsky.feed.post/456',
    cid: '[...]',
  },
}

Because the $type property of objects is required in some contexts while optional in others, we introduced a new type utility type to make it required when needed. The $Typed utility allows marking an interface’s $type property non-optional in contexts where it is required:

export type $Typed<V> = V & { $type: string }

The embed property of posts is now defined as follows:

export interface Record {
  // ...
  embed?:
    | $Typed<AppBskyEmbedImages.Main>
    | $Typed<AppBskyEmbedVideo.Main>
    | $Typed<AppBskyEmbedExternal.Main>
    | $Typed<AppBskyEmbedRecord.Main>
    | $Typed<AppBskyEmbedRecordWithMedia.Main>
    | { $type: string }
}

In addition to preventing the creation of invalid data as seen before, this change also allows properly discriminating types when accessing the data. For example, one can now do:

import { AppBskyFeedPost } from '@atproto/api'

// Say we got some random post somehow (typically
// via an API call)
declare const post: AppBskyFeedPost.Main

// And we want to know what kind of embed it contains
const { embed } = post

// We can now use the `$type` property to disambiguate
if (embed?.$type === 'app.bsky.embed.images') {
  // The `embed` variable is fully typed as
  // `$Typed<AppBskyEmbedImages.Main>` here!
}

$type property in record definitions​

While optional in interfaces generated from Lexicon object definitions, the $type property is required in interfaces generated from Lexicon record definitions.

is* utility methods​

The example above shows how data can be discriminated based on the $type property. The SDK provides utility methods to perform this kind of discrimination. These methods are named is* and are generated from the lexicons. For example, the app.bsky.embed.images Lexicon used to generate the following isMain utility method:

export interface Main {
  images: Image[]
  [x: string]: unknown
}

export function isMain(value: unknown): value is Main {
  return (
    value != null &&
    typeof value === 'object' &&
    '$type' in value &&
    (value.$type === 'app.bsky.embed.images' ||
      value.$type === 'app.bsky.embed.images#main')
  )
}

That implementation of the discriminator is invalid.

• First, because a $type is not allowed to end with #main (as per AT Protocol specification).
• Second, because the isMain function does not actually check the structure of the object, only its $type property.

This invalid behavior could yield runtime errors that could otherwise have been avoided during development:

import { AppBskyEmbedImages } from '@atproto/api'

// Get an invalid embed somehow
const invalidEmbed = {
  $type: 'app.bsky.embed.images',
  // notice how the `images` property is missing here
}

// This predicate function only checks the value of
// the `$type` property, making the condition "true" here
if (AppBskyEmbedImages.isMain(invalidEmbed)) {
  // However, the `images` property is missing here.
  // TypeScript does not complain about this, but the
  // following line will throw a runtime error:
  console.log('First image:', invalidEmbed.images[0])
}

The root of the issue here is that the is* utility methods perform type casting of objects solely based on the value of their $type property. There were basically two ways we could fix this behavior:

1. Alter the implementation to actually validate the object's structure. This would be a non-breaking change that has a negative impact on performance.
2. Alter the function signature to describe what the function actually does. This is a breaking change because TypeScript would start (rightfully) returning lots of errors in places where these functions are used.

Because this release introduces other breaking changes, and because adapting our own codebase to this change showed it made more sense, we decided to adopt the latter option.

For example, this is the case when working with data obtained from the API. Because an API is a "contract" between a server and a client, the data returned by Bluesky's server APIs is "guaranteed" to be valid. In these cases, the is* utility methods provide a convenient way to discriminate between valid values.

import { AppBskyEmbedImages } from '@atproto/api'

// Get a post from the API (the API's contract
// guarantees the validity of the data)
declare const post: AppBskyEmbedImages.Main

// The `is*` utilities are an efficient way to
// discriminate **valid** data based on their `$type`
if (isImages(post.embed)) {
  // `post.embed` is fully typed as
  // `$Typed<AppBskyEmbedImages.Main>` here!
}

validate* utility methods​

As part of this update, the signature of the validate* utility methods was updated to properly describe the type of the value in case of success:

import { AppBskyEmbedImages } from '@atproto/api'

// Aliased for clarity
const Images = AppBskyEmbedImages.Main
const validateImages = AppBskyEmbedImages.validateMain

// Get some data somehow
declare const data: unknown

// Validate the data against a particular schema (images here)
const result = validateImages(data)

if (result.success) {
  // The `value` property was previously typed as `unknown`
  // and is now properly typed as `Image`
  const images = result.value
}

These methods perform data validation, making them somewhat slower than the is* utility methods. They can, however, be used in place of the is* utilities when migrating to this new version of the SDK.

New asPredicate function​

The SDK exposes a new asPredicate function. This function allows converting a validate* function into a predicate function. This can be useful when working with libraries that expect a predicate function to be passed as an argument.

import { asPredicate, AppBskyEmbedImages } from '@atproto/api'

// Aliased for clarity
const Images = AppBskyEmbedImages.Main
const isValidImages = asPredicate(AppBskyEmbedImages.validateMain)

// Get an embed with unknown validity somehow
declare const embed: unknown

// The following condition will be true if, and only
// if, the value matches the `Image` interface.
if (isValidImages(embed)) {
  // `embed` is of type `Images` here
}

// Similarly, the type predicate can be used to
// infer the type of an array of unknown values:
declare const someArray: unknown[]

// This will be typed as `Images[]`
const images = someArray.filter(isValidImages)

Removal of the [x: string] index signature​

Another property of Atproto being an "open protocol" is the fact that objects are allowed to contain additional — unspecified — properties (although this should be done with caution to avoid incompatibility with properties that are added in the future). This used to be represented in the type system using a [k: string]: unknown index signature in generated interfaces. This is how the video embed used to be represented:

export interface Main {
  video: BlobRef
  captions?: Caption[]
  alt?: string
  aspectRatio?: AppBskyEmbedDefs.AspectRatio
  [k: string]: unknown
}

This signature allowed for undetectable mistakes to be performed:

import { AppBskyEmbedVideo } from '@atproto/api'

const embed: AppBskyEmbedVideo.Main = {
  $type: 'app.bsky.embed.video',
  video: {
    /* omitted */
  },

  // Notice the typo in `alt`, not resulting in a TypeScript error
  atl: 'My video',
}

We removed that signature, requiring any unspecified fields intentionally added to be now explicitly marked as such:

import { AppBskyEmbedVideo } from '@atproto/api'

const embed: AppBskyEmbedVideo.Main = {
  $type: 'app.bsky.embed.video',
  video: {
    /* omitted */
  },

  // Next line will result in the following
  //  TypeScript error: "Object literal may only
  // specify known properties, and 'atl' does not
  // exist in type 'Main'"
  atl: 'My video',

  // Unspecified fields must now be explicitly
  // marked as such:

  // @ts-expect-error - custom field
  comExampleCustomProp: 'custom value',
}

Other considerations​

When upgrading, please make sure that your project does not depend on multiple versions of the @atproto/* packages. Use resolutions or overrides in your package.json to pin the dependencies to the same version.

Recap​

We hope this release helps you build better codebases with improved type safety. During our own migration, we found and fixed a few small bugs, and we believe these changes will benefit the entire developer community.

Migration TL;DR:​

• Need to be absolutely sure of your data? Use asPredicate or validate* utilities.
• Using data from the Bluesky app view? You can use is* utilities.
• Building lex objects for writing? Make sure you use $Typed when building those.

Happy coding!

In the big picture, most of the public data aspects of the protocol have now been designed and implemented. The last missing pieces are nearing completion, and we do not foresee disruptive changes or additions which would impact interoperability. Now is a great time to start building on the protocol and assembling independent infrastructure.

A lot of progress was made in 2024! Some large protocol milestones include:

• Open PDS federation in the live network: At the time of roll-out, Bluesky initially required pre-registration and placed a limit on the number of hosted accounts. These limits have since been removed, and now any PDS can participate in the live network without prior coordination. We encourage the growth of independent PDS instances of any size. There are some rate-limits in place to prevent bot farms, but we will increase them when needed to accommodate PDS growth.
• Labeling: We launched the stackable moderation system with labels and reports.
• Account Migration: Users can migrate their account to alternate PDSes using command-line tooling.
• Generic Service Proxying: Client XRPC requests to the PDS can be proxied on to arbitrary service providers, using inter-service auth, via a client-controlled HTTP header.
• Flexible Record Schemas: Data records of any schema can be written to atproto repositories, with "eager" Lexicon validation controlled by query parameter.
• Account Deactivation: This was implemented at the protocol layer, along with an overhaul of the #identity and #account firehose events.
• Initial OAuth Support: This was launched in production, with extensive documentation.
• Jetstream: We shipped an alternative WebSocket API for the firehose, which uses simple JSON and record-level operations (as opposed to "commits"). This makes it easier for independent developers to process the Bluesky firehose.

Lexicon Resolution​

One of the more recently-completed components is Lexicon resolution. This is the mechanism for looking up the schema of new data by NSID. After feedback on a public proposal, we settled on a design that uses DNS TXT records that map to a DID, then schemas stored as records in an atproto repo. We have some early implementations which prove out the design.

We plan on writing this up as a formal specification, and would like to build tooling to support publishing, mirroring, discovery, and integration of Lexicons.

Auth and OAuth​

The OAuth announcement blog post gives a good overview of the progress made in 2024, and links out to developer resources. Server-side support has been implemented in the Bluesky PDS distribution, and several independent projects are using it for login in the live network. We iterated on our design to align with the Web Auth Working Group of the IETF, and will make small changes as needed to stay in alignment.

One of the missing pieces is Auth Scopes, which will allow more granular and flexible Authorization grants. We have made a fair amount of design progress on this mechanism but still have a few issues to resolve. Keep an eye out for a public summary of this work soon.

Based on the OAuth Roadmap, we are still in the first phase. Apart from Scopes, we have feedback from developers that token lifetimes may need tuning.

Apart from OAuth, we are increasingly interested in a more powerful and flexible auth token mechanism, but have not started any design or planning work yet.

Sync, Firehose, and Backfill​

As the overall network has scaled to over 26 million accounts, resource costs around Relays and the firehose mechanism have become more urgent.

The initial Relay design required a full mirror of all repository data (records and MST nodes) to fully verify each commit message. This meant that disk consumption increased both with the number of accounts, and the amount of data each account stored. This was considered relatively affordable, and further scaling could be achieved with infrastructure sharding.

However, we are currently exploring a "non-archival" relay design which is significantly cheaper to operate, even at full network scale. This will simplify the role of relays in the network, replace the tooBig mechanism for large commits, and clarify what a downstream service needs to do to synchronize data reliably.

We are also working on improved ergonomics for working with subsets of data in the network. In particular, new small applications (Lexicon schemas) should be able to start small. It is important to be able to backfill only the relevant data already in the network and then subscribe to just a subset of data downstream of the firehose.

Specifications and Ecosystem​

We refreshed the AT Protocol website, and the written specifications were expanded to cover the Firehose, Blobs, Account Hosting, and more.

Bluesky sent a representative to IETF 120 in Vancouver, Canada. We have started participating in the OAuth and DNSOP working groups, and have been discussing timelines and strategy for the standards process with members of the community.

DASL is an independent effort to define a coherent subset of the IPLD specifications which projects can build upon without the full complexity and large implementation surface of those systems. These align very tightly with the atproto data model! It should be possible for atproto software to build on top of DASL implementation libraries (which have fewer dependencies than full IPLD implementations), and there is potential for collaboration within formal standards bodies.

lexicon.community is an independent effort to develop reusable atproto Lexicon schemas in a collaborative manner. They have a defined governance and contribution model, and an initial schema declared for bookmarks.

What Else?​

A few important areas did not see much change in 2024. The PLC identity system has been operating reliably, but the plc.directory service needs to be decentralized through technical and governance improvements. Protocol features to support private content in the network continue to be a top external request, and are likewise a priority within the team. E2EE DMs are the planned successor to the initial DM system, but we have not begun work on them.

All the above are important and will take time to get right. We will share a forward-looking protocol roadmap in the near future, covering these projects, decentralization efforts, and more.

In the meanwhile, we encourage you to check in on both official and community channels for updates on AT Protocol. An increasing share of development is happening out in the ecosystem, with more projects and organizations getting started by the week:

• @atproto.com account on Bluesky
• Discussions on Github
• The Fediverse Report includes weekly AT Protocol coverage
• atprotocol.dev runs Tech Talks

Update January 2025: Many of the operational changes described here were not implemented. We will give an update on Relay infrastructure early this year.

Summary: We are making a couple of changes to the Bluesky relay servers this week. The wire protocol and semantics are not impacted, but the changes may impact firehose consumers:

• One way or another, the event stream sequence will change for all subscribers. Most consumers can probably just reset to the new cursor; see details below.
• The bsky.network hostname will be swapped to a new relay instance (with new sequence)

The atproto network is growing rapidly! This is exciting and positive, but it means we need to move up some of our network scaling plans. One particular resource bottleneck is the relay firehose. We are seeing sustained traffic of over 2,000 events per second, and we have hundreds of active consumers. Multiplied, the overall throughput is hard for any one server to keep up with.

We saw this problem coming from the beginning and we have several plans and options to mitigate it. Details for each of these are linked or in sections below.

First, we are simply upgrading the size of server that we run our primary relay instances on. We expect to do this in coming days. The main impact here will be a reset of the firehose sequence (including cursor values), which will impact all downstream consumers.

Second, we are introducing the concept of "firehose fan-out" services, and releasing our implementation Rainbow. These are servers which re-broadcast the event stream firehose to many clients, reducing the bandwidth load on relays themselves. Rainbow is now in production, as an implementation detail.

Third, we recently released Jetstream as a more informal and light-weight option for event stream consumption. We encourage developers to check out Jetstream, and switch over if it works for their use case.

We encourage folks to take advantage of Rainbow and Jetstream. You can run your own, or make use of our hosted instances. Some additional longer-term options are discussed at the bottom of this article.

We don't have a hard time or date estimate for the cursor sequence change, but we expect it this week, possibly tomorrow (Tuesday). We generally try to give more time and make changes less disruptive but we're not able to do so in this situation. Please follow the @atproto.com account, this blog, and in Github Discussions for announcements.

Relay Upgrade​

The current Bluesky relay instance is available at wss://bsky.network. A more specific hostname for this relay instance will soon be relay1.us-west.bsky.network (not yet configured).

Instead of upgrading the relay in-place, we are going to start a new relay from scratch. This means that event stream ordering will be different and sequence numbers will not align with the current relay. This new instance will soon run at relay2.us-west.bsky.network (not yet configured).

There will be at least a short window when both relays are running, so downstream consumers can update their services to "cut over". At some point soon after the new relay is deployed, we will switch the generic bsky.network hostname to point at it.

What firehose consumers need to do about the cut over depends on how careful they need to be about processing every single event on the firehose.

Services which always connect to the firehose from the current offset don’t need to do anything. This includes developer tools like goat, and real-time sampling like Firesky.

Best-effort consumers can likely be configured to consume from the current firehose offset, and simply restart, missing at most a few seconds of events. This probably makes sense for feed generators, analytics, automated labeling, etc.

Services which want to really minimize the number of missed events can take a different approach. They can be programmed to connect to the new relay with no cursor, detect the current sequence number, then reconnect with a cursor a couple of minutes back (at current rates this means a few hundred thousand events back). This results in a time overlap window between the two relays (with some events processed twice), which helps minimize the chance of missing any events.

Firehose Fan-out and Rainbow​

Firehose fan-out servers have a single upstream connection to a relay firehose, and re-broadcast that stream to multiple subscribing clients. They maintain a local backfill window (allowing re-connection with a cursor), but do not implement endpoints like getRecord or getRepo, only com.atproto.sync.subscribeRepos. They do not re-validate the event stream (eg, they don't verify signatures). They maintain the exact sequence numbers of the upstream relay, meaning that they are interchangeable with the relay and with sibling fan-out servers, and can be placed behind a load-balancer.

Today we are releasing Rainbow, a fan-out service implemented in Go. The source code is available in the indigo git repository, with the same MIT/Apache licensing as our other open source projects.

Rainbow is already running as part of our production relay deployment (wss://bsky.network), with our load balancer (haproxy) distributing WebSocket subscriptions to a separate server running Rainbow.

In the future, we may run additional Rainbow instances outside our primary data centers, similar to how we offer Jetstream instances today. This distributes network traffic over more routers, and could improve connection reliability and throughput for subscribers in regions outside North America.

Going forward, we recommend that software which consumes from firehoses (including atproto SDKs) support HTTP redirects for WebSocket connections. This enables "pooling" behavior where a single hostname could route clients to multiple distinct servers, without use of a load-balancer.

What Else?​

As a reminder, the relay synchronization API is the same as the PDS synchronization API, including the firehose. Relays are essential to the functionality of the atproto network: they are an operational and developer convenience, allowing consuming services to skip keeping track of which PDS instances are active.

A classic way to scale services like the firehose is sharding, where the stream is split into multiple parallel streams. atproto has a natural sharding key (the account DID), which means related events can be consistently routed to the correct shard. This will allow near-indefinite scaling of the firehose event rate, assuming compute and network resources are available.

Part of what makes relays difficult to operate at scale is that they function as both a "relay" (rebroadcasting events from PDS instances) and a full-network mirror (storing all repo contents). In the current protocol, it is necessary to combine both functions to fully verify repository operations, especially deletion of events. We think that clever implementations could make this work less resource intensive (for example, storing just record CIDs instead of full data). It is also possible to validate most aspects of the event stream without a full copy of the repo tree, and there could be a role in the network for "non-mirroring" relays.

Lastly, there is a particular need for efficient consumption and backfill of full-network content for only content and accounts which make use of specific record types. For example, “only whtwnd blog post records”, or “only accounts with labeling service declarations”. We are planning new features to make this type of network subscription and backfill much more efficient.

But the firehose wire format is also one of the more complex parts of atproto, involving decoding binary CBOR data and CAR files, which can be off-putting to new developers. Additionally, the volume of data has increased rapidly as the network has grown, consistently producing hundreds of events per second.

The full synchronization firehose is core network infrastructure and not going anywhere, but to address these concerns we developed an alternative streaming solution, Jetstream, which has a few key advantages:

• simple JSON encoding
• reduced bandwidth, and compression
• ability to filter by collection (NSID) or repo (DID)

A Jetstream server consumes from the firehose and fans out to many subscribers. It is open source, implemented in Go, simple to self-host. There is an official client library included (in Go), and community client libraries have been developed.

Jetstream was originally written as a side project by one of our engineers, Jaz. You can read more about their design goals and efficiency gains on their blog. It has been successful enough that we are promoting it to a team-maintained project, and are running several public instances:

• jetstream1.us-east.bsky.network
• jetstream2.us-east.bsky.network
• jetstream1.us-west.bsky.network
• jetstream2.us-west.bsky.network

You can read more technical details about Jetstream in the Github repo.

Why Now?​

Why are we promoting Jetstream at this time?

Two factors came to a head in early September: we released an example project for building new applications on atproto (Statusphere), and we had an unexpectedly large surge in traffic in Brazil. Suddenly we had a situation where new developers would be subscribing to a torrential full-network firehose (over a thousand events per second), just to pluck out a handful of individual events from a handful of accounts. Everything about this continued to function, even on a laptop on a WiFi connection, but it feels a bit wild as an introduction to the protocol.

We knew from early on that while the current firehose is extremely powerful, it was not well-suited to some use cases. Until recently, it hadn’t been a priority to develop alternatives. The firehose is a bit overpowered, but it does Just Work.

Has the Relay encountered scaling problems or become unaffordable to operate?

Nope! The current Relay implementation ('bigsky', written in Go, in the indigo git repo) absorbed a 10x surge in daily event rate, with over 200 active subscribers, and continues to chug along reliably. We have demonstrated how even a full-network Relay can be operated affordably.

We do expect to refactor our Relay implementation and make changes to the firehose wire format to support sharding. But the overall network architecture was designed to support global scale and millions of events per second, and we don't see any serious barriers to reaching that size. Bandwidth costs are manageable today. At larger network size (events times subscribers), bandwidth will grow in cost. We expect that the economic value of the network will provide funding and aligned incentives to cover the operation of core network infrastructure, including Relays. In practical terms, we expect funded projects and organizations depending on the firehose to pay infrastructure providers to ensure reliable operation (eg, an SLA), or to operate their own Relay instances.

Tradeoffs and Use Cases​

Jetstream has efficiency and simplicity advantages, but they come with some tradeoffs. We think it is a pragmatic option for many projects, but that developers need to understand what they are getting into.

Events do not include cryptographic signatures or Merkle tree nodes, meaning the data is not self-authenticating. "Authenticated Transfer" is right in the AT Protocol acronym, so this is a pretty big deal! The trust relationship between a Jetstream operator and a consuming client is pretty different from that of a Relay. Not all deployment scenarios and use-cases require verification, and we suspect many projects are already skipping that aspect when consuming from the firehose. If you are running Jetstream locally, or have a tight trust relationship with a service provider, these may be acceptable tradeoffs.

Unlike the firehose (aka, Repository Event Stream), Jetstream is not formally part of the protocol. We are not as committed to maintaining it as a stable API or critical piece of infrastructure long-term, and we anticipate adopting some of the advantages it provides into the protocol firehose over time.

On the plus side, Jetstream is easier and cheaper to operate than a Relay instance. Folks relying on Jetstream can always run their own copy on their own servers.

Some of the use cases we think Jetstream is a good fit for:

• casual, low-stakes projects and social toys: interactive bots, and "fun" badging labelers (eg, Kiki/Bouba)
• experimentation and prototyping: student projects, proofs of concept, demos
• informal metrics and visualizations
• developing new applications: filtering by collection is particularly helpful when working with new Lexicons and debugging
• internal systems: if you have multiple services consuming from the firehose, a single local Jetstream instance can be used to fan out to multiple subscribers

Some projects it is probably not the right tool for:

• mirroring, backups, and archives
• any time it is important to know "who said what"
• moderation or anti-abuse actions
• research studies

What Else?​

The ergonomics of working with the firehose and "backfilling" bulk data from the network are something we would like to improve in the protocol itself. This might include mechanisms for doing "selective sync" of specific collections within a repo, while still getting full verification of authenticity.

It would be helpful to have a mechanism to identify which repos in the network have any records of a specific type, without inspecting every account individually. For example, enumerating all of the labelers or feed generators in the network. This is particularly important for new applications with a small initial user base.

We are working to complete the atproto specifications for the firehose and for account hosting status.

One of the major components of the protocol is the concept of "Lexicons," which are machine-readable schemas for both API endpoints and data records. The goal with Lexicons is to make it possible for independent projects to work with the same data types reliably. Users should be able to choose which software they use to interact with the network, and it is important that developers are able to call shared APIs and write shared data records with confidence.

While the Lexicon concept has been baked into the protocol from the beginning, some aspects are still being finalized, and best practices around extensions, collaboration, and governance are still being explored.

A recent incident in the live network brought many of these abstract threads into focus. Because norms and precedent are still being established, we thought it would be good to dig into the specific situation and give some updates.

What Happened?​

On October 10, Bluesky released version 1.92 of our main app. This release added support for "pinned posts," a long-requested feature. This update added a pinnedPost field to the app.bsky.actor.profile record. This field is declared as a com.atproto.repo.strongRef, which is an object containing both the URL and a hash (CID) of the referenced data record.

📢 App Version 1.92 is rolling out now (1/5)Pinned posts are here! Plus lots of UI improvements, including new font options, and the ability to filter your searches by language.Open this thread for more details. 🧵

[image or embed]

— Bluesky (@bsky.app) Oct 10, 2024 at 3:24 PM

All the way back in April 2024, independent developers had already implemented pinned posts in a handful of client apps. They did so by using a pinnedPost field on the app.bsky.actor.profile record, as a simple string URL. This worked fine for several months, and multiple separate client apps (Klearsky, Tokimeki, and Hagoromo) collaborated informally and used this same extension of the profile record type.

やっていることは簡単で、 app.bsky.actor.profile に pinnedPost というカスタムフィールドを作り、これにポストのAT URIを設定しているだけ…なんですが getProfile がカスタムフィールドを返してくれない（それはそう）のがちょっとあれでまだ調整中です

— mimonelu 🦀 みもねる (@mimonelu.net) Apr 29, 2024 at 8:45 AM

One of the interesting dynamics was that multiple independent Bluesky apps were collaborating to use the same extension field.

Blueskyクライアントの一覧を更新しました！🆕Features!PinnedPost (3rd party non-official feature)・Klearsky・TOKIMEKI・羽衣-Hagoromo-

[image or embed]

— どるちぇ (@l-tan.blue) May 3, 2024 at 9:36 AM

Which all worked great! Until the Bluesky update conflicted with the existing records, causing errors for some users. Under the new schema, the previously-written records suddenly became "invalid". And new records, valid under the new schema, could be invalid from the perspective of independent software.

Analysis​

The issue with conflicting records was an unintentional mistake on our part. While we knew that other apps had experimented with pinned posts, and separately knew that conflicts with Lexicon extension fields were possible in theory, we didn't check or ask around for feedback when updating the profile schema. While the Bluesky app is open-source and this new schema had even been discussed by developers in the app ahead of time, we didn't realize we had a name collision until the app update was shipped out to millions of users. If we had known about the name collision in advance, we would have chosen a different field name or worked with the dev community to resolve the issue.

There has not been clear guidance to developers about how to interoperate with and extend Lexicons defined by others. While we have discussed these questions publicly a few times, the specifications are somewhat buried, and we are just starting to document guidance and best practices.

At the heart of this situation is a tension over who controls and maintains Lexicions. The design of the system is that authority is rooted in the domain name corresponding to the schema NSID (in reverse notation). In this example, the app.bsky.actor.profile schema is controlled by the owners of bsky.app – the Bluesky team. Ideally schema maintainers will collaborate with other developers to update the authoritative schemas with additional fields as needed.

There is some flexibility in the validation rules to allow forwards-compatible evolution of schemas. Off-schema attributes can be inserted, ignored during schema validation, and passed through to downstream clients. Consequently it’s possible (and acceptable) for other clients to use off-schema attributes, which is the situation that happened here.

While this specific case resulted in interoperability problems, we want to point out that these same apps are separately demonstrating a strong form of interoperation by including data from multiple schemas (whtwnd.com, linkat.blue, etc) all in a single app. This is exactly the kind of robust data reuse and collaboration we hoped the Lexicon system would enable.

🌈 TOKIMEKI UPDATE!!!(Web/Android v1.3.5/iOS TF)🆕 プロフィール画面に Atmosphere スペースを追加！- AT Protocol では Bluesky 以外にも様々なサービスを自由に開発することができ、実際にいくつかの便利なサービスが公開されています。- ユーザーが利用しているBluesky以外のサービスへのリンクを見ることができます。- 現在は、Linkat (リンク集) と WhiteWind (ブログ) の2つに対応。- 設定→全般から非表示にできます。Web | Android

[image or embed]

— 🌈 TOKIMEKI Bluesky (@tokimeki.blue) Oct 10, 2024 at 10:50 PM

Current Recommendations​

What do we recommend to developers looking to extend record schemas today?

Our current recommendation is to define new Lexicons for "sidecar" records. Instead of adding fields to app.bsky.actor.profile, define a new record schema (eg com.yourapp.profile) and put the fields there. When rendering a profile view, fetch this additional record at the same time. Some records always have a fixed record key, like self, so they can be fetched with a simple GET. For records like app.bsky.feed.post, which have TID record keys, the sidecar records can have the same record key as the original post, so they also can be fetched with a simple GET. We use this pattern at scale in the bsky Lexicons with app.bsky.feed.threadgate, which extends the post schema, and allows data updates without changing the version (CID) of the post record itself.

There is some overhead to doing additional fetches, but these can be mitigated with caching or building a shim API server (with updated API Lexicions) to blend in the additional data to "view" requests. If needed, support could be improved with generic APIs to automatically hydrate "related records" with matching TIDs across collections in the same repository.

If sidecar records are not an option, and developers feel they must add data directly to existing record types, we very strongly recommend against field names that might conflict. Even if you think other developers might want to use the same extension, you should intentionally choose long unique prefixes for field names to prevent conflicts both with the "authoritative" Lexicon author, and other developers who might try to make the same extension. What we currently recommend is using a long, unique, non-generic project name prefix, or even a full NSID for the field name. For example, app.graysky.pinnedPost or grayskyPinnedPost are acceptable, but not pinnedPost or extPinnedPost.

While there has been some clever and admirable use of extension fields (the SkyFeed configuration mechanism in app.bsky.feed.generator records comes to mind), we don't see inserting fields into data specified by other parties as a reliable or responsible practice in the long run. We acknowledge that there is a demonstrated demand for a simple extension mechanism, and safer ways to insert extension data in records might be specified in the future.

Proposals and discussion welcome! There is an existing thread on Github.

Progress with Lexicons​

While not directly related to extension fields, we have a bunch of ongoing work with the overall system.

We are designing a mechanism for Lexicon resolution. This will allow anybody on the public internet to authoritatively resolve the schema for a given NSID. This process should not need to happen very often, and we want to incorporate lessons from previous live schema validation systems (including XML), but there does need to be a way to demonstrate authority.

We are planning to build an aggregator and automated documentation system for Lexicons, similar to package management systems like pkg.go.dev and lib.rs. These will make it easier to discover and work with independent Lexicons across the ATmosphere and provide baseline documentation of schemas for developers. They can also provide collective benefits such as archiving, flagging abuse and security problems, and enabling research.

We are writing a style guide for authoring Lexicons, with design patterns, tips and common gotchas, and considerations for evolution and extensibility.

The validation behaviors for the unknown and union Lexicon types have been clarified in the specifications.

The schema validation behavior when records are created at PDS instances has been updated, and will be reflected in the specifications soon (a summary is available).

Generic run-time Lexicon validation support was added to the Go SDK (indigo), and test vectors were added to the atproto interop tests repository.

Finally, an end-to-end tutorial on building an example app ("Statusphere") using custom Lexicons was added to the updated atproto documentation website.

Overall, the process for designing and publishing new schemas from scratch should be clearer soon, and the experience of finding and working with existing schemas should be significantly improved as well.

OAuth is a framework of standards under active development by the IETF. We selected a particular "profile" of RFCs, best practices, and draft specifications to preserve security in the somewhat unique atproto ecosystem. In particular, unlike most existing OAuth deployments and integrations, the atproto network is composed of many independent server instances, client apps, developers, and end users, who generally have not cross-registered their client software ahead of time. This necessitates both automated discovery of the user’s Authorization Server, and automated registration of client metadata with the server. In some ways this situation is closer to that between email clients and email providers than it is between traditionally pre-registered OAuth or OIDC clients (such as GitHub apps or "Sign In With Google"). This unfortunately means that generic OAuth client libraries may not work out-of-the-box with the atproto profile yet. We have built on top of draft standards (including "OAuth Client ID Metadata Document") and are optimistic that library support will improve with time.

We laid out an OAuth Roadmap earlier this summer, and are entering the "Developer Preview Phase". In the coming months we expect to tweak the specification based on feedback from developers and standards groups, and to fill in a few details. We expect the broad shape of the specification to remain the same, and encourage application and SDK developers to start working with the specification now, and stop using the legacy App Password system for new projects.

What is Ready Today?​

OAuth has been deployed to several components of the atproto network over the past weeks. The Bluesky-developed PDS implementation implements the server component (including the Authorization Interface), the TypeScript client SDK now supports the client components, and several independent developers and projects have implemented login flows. At this time the Bluesky Social app has not yet been updated to use OAuth.

We have a a few resources for developers working with OAuth:

• The TypeScript SDK (@atproto/api) has been updated to support OAuth. READMEs are available for a browser-specific package and a Node.js-specific package. There is also a browser-based example project. You can read more about upgrading in the Typescript API Package Auth Refactor blog post.
• OAuth Client Implementation Guide, a developer-oriented document.
• A Python (Flask) Web Service Demo which shows how to implement a client "the hard way", without using a supporting SDK. A live version of the demo is running at https://oauth-flask.demo.bsky.dev/ (it may not be operated indefinitely).
• The AT Protocol OAuth Specification, which is the authoritative reference.

The current OAuth profile does not specify how granular permissions ("scopes") work with atproto. Instead, we have defined a small set of "transitional" scopes which provide the same levels of client access as the current auth system:

• transition:generic the same level of permissions as an App Password
• transition:chat.bsky is an add-on (must be included in combination with transition:generic) which adds access to the chat.bsky.* Lexicons for DMs. Same behavior as an App Password with the DM access option selected.

Next Steps​

There are two larger components which will integrate OAuth into the atproto ecosystem:

• The first is to expand the PDS account web interface to manage active OAuth sessions. This will allow users to inspect active sessions, the associated clients, and to revoke those sessions (eg, "remote log out"). This is user interface work specific to the PDS implementation, and will not change or impact the existing OAuth specification.
• The second is to design an atproto-native scopes system which integrates with Lexicons, record collections, and other protocol features. This will allow granular app permissions in an extensible manner. This is expected to be orthogonal to the current OAuth specification, and it should be relatively easy for client apps to transition to more granular permissions, though it will likely require logout and re-authentication by users.

These are big priorities for user security, and the path to implementation and deployment is clear.

While that work is in progress, we are interested in feedback from SDK developers and early adopters. What pain points do you encounter? Are there requirements which could be relaxed without reducing user security?

The overall design of this OAuth profile is similar to that of other social web protocols, such as ActivityPub. There are some atproto-specific aspects, but we are open to collaboration and harmonization between profiles to simplify and improve security on the web generally.

Finally, a number of the specifications we adopt and build upon are still drafts undergoing active development. We are interested in feedback on our specification, and intend to work with standards bodies (including the IETF) and tweak our profile if necessary to ensure compliance with final versions of any relevant standards and best practices.

The motivation for these changes is the need to make the @atproto/api package compatible with OAuth session management. We don't have OAuth client support "launched" and documented quite yet, so you can keep using the current app password authentication system. When we do "launch" OAuth support and begin encouraging its usage in the near future (see the OAuth Roadmap), these changes will make it easier to migrate.

In addition, the redesigned session management system fixes a bug that could cause the session data to become invalid when Agent clones are created (e.g. using agent.withProxy()).

New Features​

We've restructured the XrpcClient HTTP fetch handler to be specified during the instantiation of the XRPC client, through the constructor, instead of using a default implementation (which was statically defined).

With this refactor, the XRPC client is now more modular and reusable. Session management, retries, cryptographic signing, and other request-specific logic can be implemented in the fetch handler itself rather than by the calling code.

A new abstract class named Agent, has been added to @atproto/api. This class will be the base class for all Bluesky agents classes in the @atproto ecosystem. It is meant to be extended by implementations that provide session management and fetch handling. Here is the class hierarchy:

As you adapt your code to these changes, make sure to use the Agent type wherever you expect to receive an agent, and use the AtpAgent type (class) only to instantiate your client. The reason for this is to be forward compatible with the OAuth agent implementation that will also extend Agent, and not AtpAgent.

import { Agent, AtpAgent } from '@atproto/api'

async function setupAgent(service: string, username: string, password: string): Promise<Agent> {
  const agent = new AtpAgent({
    service,
    persistSession: (evt, session) => {
      // handle session update
    },
  })

  await agent.login(username, password)

  return agent
}

import { Agent } from '@atproto/api'

async function doStuffWithAgent(agent: Agent, arg: string) {
  return agent.resolveHandle(arg)
}

import { Agent, AtpAgent } from '@atproto/api'

class MyClass {
  agent: Agent

  constructor () {
    this.agent = new AtpAgent()
  }
}

Breaking changes​

Most of the changes introduced in this version are backward-compatible. However, there are a couple of breaking changes you should be aware of:

• Customizing fetch: The ability to customize the fetch: FetchHandler property of @atproto/xrpc's Client and @atproto/api's AtpAgent classes has been removed. Previously, the fetch property could be set to a function that would be used as the fetch handler for that instance, and was initialized to a default fetch handler. That property is still accessible in a read-only fashion through the fetchHandler property and can only be set during the instance creation. Attempting to set/get the fetch property will now result in an error.
• The fetch() method, as well as WhatWG compliant Request and Headers constructors, must be globally available in your environment. Use a polyfill if necessary.
• The AtpBaseClient has been removed. The AtpServiceClient has been renamed AtpBaseClient. Any code using either of these classes will need to be updated.
• Instead of wrapping an XrpcClient in its xrpc property, the AtpBaseClient (formerly AtpServiceClient) class - created through lex-cli - now extends the XrpcClient class. This means that a client instance now passes the instanceof XrpcClient check. The xrpc property now returns the instance itself and has been deprecated.
• setSessionPersistHandler is no longer available on the AtpAgent or BskyAgent classes. The session handler can only be set though the persistSession options of the AtpAgent constructor.
• The new class hierarchy is as follows:
  • BskyAgent extends AtpAgent: but add no functionality (hence its deprecation).
  • AtpAgent extends Agent: adds password based session management.
  • Agent extends AtpBaseClient: this abstract class that adds syntactic sugar methods app.bsky lexicons. It also adds abstract session management methods and adds atproto specific utilities (labelers & proxy headers, cloning capability) - AtpBaseClient extends XrpcClient: automatically code that adds fully typed lexicon defined namespaces (instance.app.bsky.feed.getPosts()) to the XrpcClient.
  • XrpcClient is the base class.

Non-breaking changes​

• The com.* and app.* namespaces have been made directly available to every Agent instances.

Deprecations​

• The default export of the @atproto/xrpc package has been deprecated. Use named exports instead.
• The Client and ServiceClient classes are now deprecated. They are replaced by a single XrpcClient class.
• The default export of the @atproto/api package has been deprecated. Use named exports instead.
• The BskyAgent has been deprecated. Use the AtpAgent class instead.
• The xrpc property of the AtpClient instances has been deprecated. The instance itself should be used as the XRPC client.
• The api property of the AtpAgent and BskyAgent instances has been deprecated. Use the instance itself instead.

Migration​

The @atproto/api package​

If you were relying on the AtpBaseClient solely to perform validation, use this:

import { AtpBaseClient, ComAtprotoSyncSubscribeRepos } from '@atproto/api'

const baseClient = new AtpBaseClient()

baseClient.xrpc.lex.assertValidXrpcMessage('io.example.doStuff', {
  // ...
})

import { lexicons } from '@atproto/api'

lexicons.assertValidXrpcMessage('io.example.doStuff', {
  // ...
})

If you are extending the BskyAgent to perform custom session manipulation, define your own Agent subclass instead:

import { BskyAgent } from '@atproto/api'

class MyAgent extends BskyAgent {
  private accessToken?: string

  async createOrRefreshSession(identifier: string, password: string) {
    // custom logic here

    this.accessToken = 'my-access-jwt'
  }

  async doStuff() {
    return this.call('io.example.doStuff', {
      headers: {
        'Authorization': this.accessToken && `Bearer ${this.accessToken}`
      }
    })
  }
}

import { Agent } from '@atproto/api'

class MyAgent extends Agent {
  private accessToken?: string
  public did?: string

  constructor(private readonly service: string | URL) {
    super({
      service,
      headers: {
        Authorization: () =>
          this.accessToken ? `Bearer ${this.accessToken}` : null,
      }
    })
  }

  clone(): MyAgent {
    const agent = new MyAgent(this.service)
    agent.accessToken = this.accessToken
    agent.did = this.did
    return this.copyInto(agent)
  }

  async createOrRefreshSession(identifier: string, password: string) {
    // custom logic here

    this.did = 'did:example:123'
    this.accessToken = 'my-access-jwt'
  }
}

If you are monkey patching the xrpc service client to perform client-side rate limiting, you can now do this in the FetchHandler function:

import { BskyAgent } from '@atproto/api'
import { RateLimitThreshold } from "rate-limit-threshold"

const agent = new BskyAgent()
const limiter = new RateLimitThreshold(
  3000,
  300_000
)

const origCall = agent.api.xrpc.call
agent.api.xrpc.call = async function (...args) {
  await limiter.wait()
  return origCall.call(this, ...args)
}

import { AtpAgent } from '@atproto/api'
import { RateLimitThreshold } from "rate-limit-threshold"

class LimitedAtpAgent extends AtpAgent {
  constructor(options: AtpAgentOptions) {
    const fetch = options.fetch ?? globalThis.fetch
    const limiter = new RateLimitThreshold(
      3000,
      300_000
    )

    super({
      ...options,
      fetch: async (...args) => {
        await limiter.wait()
        return fetch(...args)
      }
    })
  }
}

If you configure a static fetch handler on the BskyAgent class - for example to modify the headers of every request - you can now do this by providing your own fetch function:

import { BskyAgent, defaultFetchHandler } from '@atproto/api'

BskyAgent.configure({
  fetch: async (httpUri, httpMethod, httpHeaders, httpReqBody) => {

    const ua = httpHeaders["User-Agent"]

    httpHeaders["User-Agent"] = ua ? `${ua} ${userAgent}` : userAgent

    return defaultFetchHandler(httpUri, httpMethod, httpHeaders, httpReqBody)
  }
})

import { AtpAgent } from '@atproto/api'

class MyAtpAgent extends AtpAgent {
  constructor(options: AtpAgentOptions) {
    const fetch = options.fetch ?? globalThis.fetch

    super({
      ...options,
      fetch: async (url, init) => {
        const headers = new Headers(init.headers)

        const ua = headersList.get("User-Agent")
        headersList.set("User-Agent", ua ? `${ua} ${userAgent}` : userAgent)

        return fetch(url, { ...init, headers })
      }
    })
  }
}

The @atproto/xrpc package​

The Client and ServiceClient classes are now deprecated. If you need a lexicon based client, you should update the code to use the XrpcClient class instead.

The deprecated ServiceClient class now extends the new XrpcClient class. Because of this, the fetch FetchHandler can no longer be configured on the Client instances (including the default export of the package). If you are not relying on the fetch FetchHandler, the new changes should have no impact on your code. Beware that the deprecated classes will eventually be removed in a future version.

Since its use has completely changed, the FetchHandler type has also completely changed. The new FetchHandler type is now a function that receives a url pathname and a RequestInit object and returns a Promise<Response>. This function is responsible for making the actual request to the server.

export type FetchHandler = (
  this: void,
  /**
   * The URL (pathname + query parameters) to make the request to, without the
   * origin. The origin (protocol, hostname, and port) must be added by this
   * {@link FetchHandler}, typically based on authentication or other factors.
   */
  url: string,
  init: RequestInit,
) => Promise<Response>

A noticeable change that has been introduced is that the uri field of the ServiceClient class has not been ported to the new XrpcClient class. It is now the responsibility of the FetchHandler to determine the full URL to make the request to. The same goes for the headers, which should now be set through the FetchHandler function.

If you do rely on the legacy Client.fetch property to perform custom logic upon request, you will need to migrate your code to use the new XrpcClient class. The XrpcClient class has a similar API to the old ServiceClient class, but with a few differences:

• The Client + ServiceClient duality was removed in favor of a single XrpcClient class. This means that:
  • There no longer exists a centralized lexicon registry. If you need a global lexicon registry, you can maintain one yourself using a new Lexicons (from @atproto/lexicon).
  • The FetchHandler is no longer a statically defined property of the Client class. Instead, it is passed as an argument to the XrpcClient constructor.
• The XrpcClient constructor now requires a FetchHandler function as the first argument, and an optional Lexicon instance as the second argument.
• The setHeader and unsetHeader methods were not ported to the new XrpcClient class. If you need to set or unset headers, you should do so in the FetchHandler function provided in the constructor arg.

import client, { defaultFetchHandler } from '@atproto/xrpc'

client.fetch = function (
  httpUri: string,
  httpMethod: string,
  httpHeaders: Headers,
  httpReqBody: unknown,
) {
  // Custom logic here
  return defaultFetchHandler(httpUri, httpMethod, httpHeaders, httpReqBody)
}

client.addLexicon({
  lexicon: 1,
  id: 'io.example.doStuff',
  defs: {},
})

const instance = client.service('http://my-service.com')

instance.setHeader('my-header', 'my-value')

await instance.call('io.example.doStuff')

import { XrpcClient } from '@atproto/xrpc'

const instance = new XrpcClient(
  async (url, init) => {
    const headers = new Headers(init.headers)

    headers.set('my-header', 'my-value')

    // Custom logic here

    const fullUrl = new URL(url, 'http://my-service.com')

    return fetch(fullUrl, { ...init, headers })
  },
  [
    {
      lexicon: 1,
      id: 'io.example.doStuff',
      defs: {},
    },
  ],
)

await instance.call('io.example.doStuff')

If your fetch handler does not require any "custom logic", and all you need is an XrpcClient that makes its HTTP requests towards a static service URL, the previous example can be simplified to:

import { XrpcClient } from '@atproto/xrpc'

const instance = new XrpcClient('http://my-service.com', [
  {
    lexicon: 1,
    id: 'io.example.doStuff',
    defs: {},
  },
])

If you need to add static headers to all requests, you can instead instantiate the XrpcClient as follows:

import { XrpcClient } from '@atproto/xrpc'

const instance = new XrpcClient(
  {
    service: 'http://my-service.com',
    headers: {
      'my-header': 'my-value',
    },
  },
  [
    {
      lexicon: 1,
      id: 'io.example.doStuff',
      defs: {},
    },
  ],
)

If you need the headers or service url to be dynamic, you can define them using functions:

import { XrpcClient } from '@atproto/xrpc'

const instance = new XrpcClient(
  {
    service: () => 'http://my-service.com',
    headers: {
      'my-header': () => 'my-value',
      'my-ignored-header': () => null, // ignored
    },
  },
  [
    {
      lexicon: 1,
      id: 'io.example.doStuff',
      defs: {},
    },
  ],
)

Moderation is the backbone of healthy social spaces online. Bluesky has our own moderation team dedicated to providing around-the-clock coverage to uphold our community guidelines, and additionally, we recognize that there is no one-size-fits-all approach to moderation. No single company can get online safety right for every country, culture, and community in the world. So we’ve also been building something bigger — an ecosystem of moderation and open-source safety tools that gives communities power to create their own spaces, with their own norms and preferences.

Labeling services on Bluesky allow users and communities to participate in a stackable ecosystem of services. Users can create and subscribe to filters from independent moderation services, which are layered on top of Bluesky’s own service. You can read more about how stackable moderation works on Bluesky here.

Update: As of 2025 we are not currently accepting further grant applications.

To support the first labelers in our ecosystem and encourage more, we are launching a microgrants program for labeling services.​

Program Details​

For this program, we have an allocation of $10,000. We will be distributing $500 per labeling service that is approved for a grant.

The application has a rolling deadline, and we will announce both the recipients of the grants and when all of the grants have been distributed. We pay out grants via public GitHub Sponsorships.

In addition, we’ve also partnered with Amazon Web Services (AWS) to offer $5,000 in AWS Activate¹ credits to labeling services as well. These credits are applied to your AWS bill to help cover costs from cloud services, including machine learning, compute, databases, storage, containers, dev tools, and more. Simply check a box in your grant application if you’re interested in receiving these credits as well.

If you’re an organization interested in running a labeler but do not currently have the technical capacity to implement one, please reach out to our team at partnerships@blueskyweb.xyz. We may be able to assist in matching you with a developer.

Initial Labeling Grant Recipients​

We're kicking off the program with grants to three initial recipients:

XBlock

@XBlock.aendra.dev is an attempt to help give users control over the types of content they see on Bluesky. Screenshots serve a variety of uses on social media, but quite often are intended to create discourse or drive dogpiles. By letting users toggle the visibility of screenshots from various platforms, XBlock aims to give users a "volume dial" for certain types of content.

Aegis

@aegis.blue is a volunteer-run labeling service providing community moderation predominantly to Bluesky's LGBTQIA+ and marginalized users. Featuring a diverse team of both industry and aspiring experts, Aegis lives by the motto, We Keep Us Safe. More info can be found on their website at https://aegis.blue/Home.

News Detective

News Detective fights misinformation by combining the experience of professional factcheckers with the wisdom of crowds. A crowd of volunteer factcheckers transparently investigates posts, and professional factcheckers make sure only the highest quality factchecks make it through the system. Users who use News Detective will be able to see factchecks (including explanations and sources) on posts they come across and request factchecks on posts they find questionable. They can also watch News Detectives discuss the posts and even participate in factchecking to create a more honest, democratic, and transparent internet. Incubated at MIT DesignX, MIT Sandbox, and HacksHackers.

Contact​

Please feel free to leave questions or comments on the GitHub discussion for this announcement here, or on the Bluesky post here.

This roadmap is an update on our progress and lays out our general goals and focus for the coming months. This document is written for developers working on atproto clients, implementations, and applications (including Bluesky-specific projects). This is not a product announcement: while some product features are hinted at, we aren't promising specific timelines here. As always, most Bluesky software is free and open source, and observant folks can follow along with our progress week by week in GitHub.

In the big picture, we made a lot of progress on the protocol in early 2024. We opened up federation on the production network, demonstrated account migration, specified and launched stackable moderation (labeling and Ozone), shared our plan for OAuth, specified a generic proxying mechanism, built a new API documentation website (docs.bsky.app), and more.

After this big push on the protocol, the Bluesky engineering team is spending a few months catching up on some long-requested features like GIFs, video, and DMs. At the same time, we do have a few "enabling" pieces of protocol work underway, and continue to make progress towards a milestone of protocol maturity and stability.

Summary-level notes:

• Federation is now open: you don't need to pre-register in Discord any more.
• It is increasingly possible to build independent apps and integrations on atproto. One early example is https://whtwnd.com/, a blogging web app built on atproto.
• The timeline for a formal standards body process is being pushed back until we have additional independent active projects building on the protocol.

Current Work​

Proxying of Independent Lexicons: earlier this year we added a generic HTTP proxying mechanism, which allows clients to specify which onward service (eg, AppView) instance they want to communicate with. To date this has been limited to known Lexicons, but we will soon relax this restriction and make arbitrary XRPC query and procedure requests. Combined with allowing records with independent Lexicon schemas (now allowed), this finally enables building new independent atproto applications. PR for this work

Open Federation: the Bluesky Relay service initially required pre-registration before new PDS instances were crawled. This was a very informal process (using Discord) to prevent automated abuse, but we have removed this requirement, making it even easier to set up PDS instances. We will also bump the per-PDS account limits, though we will still enforce some limits to minimize automated abuse; these limits can be bumped for rapidly growing communities and projects.

Email 2FA: while OAuth is our main focus for improving account security (OAuth flows will enable arbitrary MFA, including passkeys, hardware tokens, authenticators, etc), we are rapidly rolling out a basic form of 2FA, using an emailed code in addition to account password for sign-in. This will be an optional opt-in functionality. Announcement with details

OAuth: we continue to make progress implementing our plan for OAuth. Ultimately this will completely replace the current account sign-up, session, and app-password API endpoints, though we will maintain backwards compatibility for a long period. With OAuth, account lifecycle, sign-in, and permission flows will be implementation-specific web views. This means that PDS implementations can add any sign-up screening or MFA methods they see fit, without needing support in the com.atproto.* Lexicons. Detailed Proposal

Product Features​

These are not directly protocol-related, but are likely to impact many developers, so we wanted to give a heads up on these.

Harassment Mitigations: additional controls and mechanisms to reduce the prevalence, visibility, and impact of abusive mentions and replies, particularly coming from newly created single-purpose or throw-away accounts. May expand on the existing thread-gating and reply-gating functionality.

Post Embeds: the ability to embed Bluesky posts in external public websites. Including oEmbed support. This has already shipped! See embed.bsky.app

Basic "Off-Protocol" Direct Messages (DMs): having some mechanism to privately contact other Bluesky accounts is the most requested product feature. We looked closely at alternatives like linking to external services, re-using an existing protocol like Matrix, or rushing out on-protocol encrypted DMs, but ultimately decided to launch a basic centralized system to take the time pressure off our team and make our user community happy. We intend to iterate and fully support E2EE DMs as part of atproto itself, without a centralized service, and will take the time to get the user experience, security, and privacy polished. This will be a distinct part of the protocol from the repository abstraction, which is only used for public content.

Better GIF and Video support: the first step is improving embeds from external platforms (like Tenor for GIFs, and YouTube for video). Both the post-creation flow and embed-view experience will be improved.

Feed Interaction Metrics: feed services currently have no feedback on how users are interacting with the content that they curate. There is no way for users to tell specific feeds that they want to see more or less of certain kinds of content, or whether they have already seen content. We are adding a new endpoint for clients to submit behavior metrics to feed generators as a feedback mechanism. This feedback will be most useful for personalized feeds, and less useful for topic or community-oriented feeds. It also raises privacy and efficiency concerns, so sending of this metadata will both be controlled by clients (optional), and will require feed generator opt-in in the feed declaration record.

Topic/Community Feeds: one of the more common uses for feed generators is to categorize content by topic or community. These feeds are not personalized (they look the same to all users), are not particularly "algorithmic" (posts are either in the feed or not), and often have relatively clear inclusion criteria (though they may be additionally curated or filtered). We are exploring ways to make it easier to create, curate, and explore this type of feed.

User/Labeler Messaging: currently, independent moderators have no private mechanism to communicate with accounts which have reported content, or account which moderation actions have been taken against. All reports, including appeals, are uni-directional, and accounts have no record of the reports they have submitted. While Bluesky can send notification emails to accounts hosted on our own PDS instance, this does not work cross-provider with self-hosted PDS instances or independent labelers.

Protocol Stability Milestone​

A lot of progress has been made in recent months on the parts of the protocol relevant to large-scale public conversation. The core concepts of autonomous identity (DIDs and handles), self-certifying data (repositories), content curation (feed generators), and stackable moderation (labelers) have now all been demonstrated on the live network.

While we will continue to make progress on additional objectives (see below), we feel we are approaching a milestone in development and stability of these components of the protocol. There are a few smaller tasks to resolve towards this milestone.

Takedowns: we have a written proposal for how content and account takedowns will work across different pieces of infrastructure in the network. Takedowns are a stronger intervention that complement the labeling system. Bluesky already has mechanisms to enact takedowns on our own infrastructure when needed, but there are some details of how inter-provider takedown requests are communicated.

Remaining Written Specifications: a few parts of the protocol have not been written up in the specifications at atproto.com.

Guidance on Building Apps and Integrations: while we hope the protocol will be adopted and built upon in unexpected ways, it would be helpful to have some basic pointers and advice on creating new applications and integrations. These will probably be informal tutorials and example code to start.

Account and Identity Firehose Events: while account and identity state are authoritatively managed across the DID, DNS, and PDS systems, it is efficient and helpful for changes to this state to be broadcast over the repository event stream ("firehose"). The semantics and behavior of the existing #identity event type will be updated and clarified, and an additional #account event type will be added to communicate PDS account deletion and takedown state to downstream services (Relay, and on to AppView, feed generator, labelers, etc). Downstream services might still need to resolve state from an authoritative source after being notified on the firehose.

Private Account Data Iteration: the app.bsky Lexicons currently include a preferences API, as well as some additional private state like mutes. The design of the current API is somewhat error-prone, difficult for independent developers to extend, and has unclear expectations around providing access to service providers (like independent AppViews). We are planning to iterate on this API, though it might not end up part of the near-term protocol milestone.

Protocol Tech Debt: there are a few other small technical issues to resolve or clean up; these are tracked in this GitHub discussion

On the Horizon​

There are a few other pieces of protocol work which we are starting to plan out, but which are not currently scheduled to complete in 2024. It is very possible that priorities and schedules will be shuffled, but we mostly want to call these out as things we do want to complete, but will take a bit more time.

Protocol-Native DMs: as mentioned above, we want to have a "proper" DM solution as part of atproto, which is decentralized, E2EE, and follows modern security best practices.

Limited-Audience (Non-Public) Content: to start, we have prioritized the large-scale public conversation use cases in our protocol design, centered around the public data repository concept. While we support using the right tool for the job, and atproto is not trying to encompass every possible social modality, there are many situations and use-cases where having limited-audience content in the same overall application would be helpful. We intend to build a mechanism for group-private content sharing. It will likely be distinct from public data repositories and the Relay/firehose mechanism, but retain other parts of the protocol stack.

Firehose Bandwidth Efficiency: as the network grows, and the volume and rate of repository commits increases, the cost of subscribing to the entire Relay firehose increases. There are a number of ways to significantly improve bandwidth requirements: removing MST metadata for most use-cases; filtering by record types or subsets of accounts; batch compression; etc.

Record Versioning (Post Editing): atproto already supports updating records in repositories: one example is updating bsky profile records. And preparations were made early in the protocol design to support post editing while avoiding misleading edits. Ideally, it would also be possible to (optionally) keep old versions of records around in the repository, and allow referencing and accessing multiple versions of the same record.

PLC Transparency Log: we are exploring technical and organizational mechanisms to further de-centralize the DID PLC directory service. The most promising next step looks to be publishing a transparency log of all directory operations. This will make it easier for other organizations to audit the behavior of the directory and maintain verifiable replicas. The recent "tiling" transparency log design used for https://sunlight.dev/ (described here) is particularly promising. Compatibility with RFC 6962 (Certificate Transparency) could allow future integration with an existing ecosystem of witnesses and auditors.

Identity Key Self-Management UX: the DID PLC system has a concept of "rotation keys" to control the identity itself (in the form of the DID document). We would like to make it possible for users to optionally register additional keys on their personal devices, password managers, or hardware security keys. If done right, this should improve the resilience of the system and reduce some of the burden of responsibility on PDS operators. While this is technically possible today, it will require careful product design and security review to make this a safe and widely-adopted option.

Standards Body Timeline​

As described in our 2023 Protocol Roadmap, we hope to bring atproto to an existing standards body to solidify governance and interoperability of the lower levels of the protocol. We had planned to start the formal process this summer, but as we talked to more people experienced with this process, we realized that we should wait until the design of the protocol has been explored by more developers. It would be ideal to have a couple organizations with atproto experience collaborate on the standards process together. If you are interested in being part of the atproto standards process, leave a message in the discussion thread for this post, or email protocol@blueskyweb.xyz.

While there has been a flowering of many projects built around the app.bsky microblogging application, there have been very few additional Lexicons and applications built from scratch. Some of this stemmed from restrictions on data schemas and proxying behavior on the Bluesky-hosted PDS instances, only relaxed just recently. We hope that new apps and Lexicons will exercise the full capabilities and corner-cases of the protocol.

We will continue to participate in adjacent standards efforts to make connections and get experience. Bluesky staff will attend IETF 120 in July, and are always happy to discuss responsible DNS integrations, OAuth, and HTTP API best practices.