Onboarding new blockchains: Everstake’s DevOps insights

The launch of a new blockchain on the mainnet is always a momentous occasion eagerly awaited by the community and other stakeholders. More often than not, this comes along with airdrops, the emergence of new crypto markets, sought-after products, and new use cases. Ensuring a successful launch isn’t a walk in the park though; it calls for a teamwork effort from Genesis validators and chain reps.

In this short and to-the-point article, our DevOps team provides a unique peek into Everstake’s onboarding experiences with new chains. While our focus has largely been on Cosmos-based chains, these insights can serve as a helpful guide for onboarding other PoS blockchains as well.

Navigating Onboarding: Encountered Issues and Preventive Measures

1. Challenge: With multiple social media channels for communications with validators as well as for system updates, it’s often almost impossible to define a priority channel and confirm its authenticity. As a result, the research process becomes a tedious task of sifting through various messaging platforms, and comparing information to determine if the accounts are legit.

Solution: From the start, set up two official main channels for announcements and related discussions regarding updates, bugs, and features. These channels should be easy to find and join, ideally through Twitter, your website, associated platforms, and related publications.

2. Challenge: Sometimes, there is either no feedback from the team on important updates, or there is an excess of information from several chain team members, who may not even be designated liaisons, leading to chaos and confusion.

Solution: Officially appoint a group of staff to manage communication with validators, preferably on the announcement channel, so that the involved parties can swiftly connect with the right person.

3. Challenge: Inadequate onboarding tutorials are a common problem. Some teams assume that validators are experts in all aspects of blockchain software and languages and provide minimalistic guidelines. For example, to start a node, you may only be given a link to the repository and asked to send a gen transaction there.

Solution: To keep everyone on the same page, it’s essential to provide at least a concise guide on how the binary should be generated, clear requirements for gentx, which chain’s version is needed, and for which language (if matters). While these details may seem obvious, they’re not, and the lack of clarity can cause delays in launch, as validators may follow different, and sometimes incorrect, procedures.

4. Challenge: Lack of reminders can lead to oversights. When the genesis event is announced just once, especially if there’s a considerable time lapse, it’s easy for the information to be forgotten, even if it’s pinned in a relevant channel.

Solution: Send out numerous reminders close to the launch, and ideally, receive direct confirmation from all validators involved before the start time.

5. Challenge: The schedule can be unpredictable. Updates may be delayed, testnet scenarios might not be executed on time, and various factors can jeopardize the launch. These Challenges often arise just before the spawn time, leading to rushed updates and emergency measures, thereby increasing the risk of errors due to stress.

Solution: Ideally, a launch schedule should be provided at least a month before the Genesis event, along with a roadmap outlining releases, updates, and a detailed action plan for node operators. If milestones shift, operators should be informed via the announcement channel, providing patches, guidelines, and other necessary materials. If an emergency situation arises close to the spawn time, it might be better to reschedule the date, allowing validators to resolve Challenges in a calm setting

6. Challenge: Inconvenient slot for spawn time. Scheduling the Genesis event for late Friday nights or weekends might not be the most feasible option. Given the heavy workload during the week, there’s a likelihood of missing this event

Solution: Our DevOps suggest the launch slot to be around 12-15 am UTC from Monday to Wednesday. This time slot, they believe, would be more noticeable and suitable for the majority across different time zones.

7. Challenge: Outdated CLI commands. During testnet some commands and transactions can change and if docs don’t reflect such changes properly, it can lead to unwanted confusion and cause blockers for involved DevOps.

Solution: Keep the documentation concise and up-to-date in a single official source.

How can validators ensure the smooth onboarding of new blockchains?

1. This might seem obvious, but once the majority of the work is done – after generating a binary, setting up the node, and sending a gentx – don’t forget to start the node and verify its proper functionality. If the chain’s team doesn’t provide a step-by-step list, create one yourself.
2. Remember, once your node is operating, you’re responsible for contributing to consensus, which is the teamwork of all validators. It’s crucial to coordinate updates with other participants at the set time.
3. Treat your Node keys with the utmost care by storing them securely on a computer with limited access. Don’t copy the keys between different nodes until you’re sure that only one node is operational at a time.
4. A backup node is a must for emergency purposes. It’s highly recommended to distribute your primary and backup nodes across different data centers for optimal performance
5. Before using a server, carefully examine the policies of server providers, especially those related to crypto services. This can help avoid unwanted shutdowns. For instance, while Hetzner is a major server provider, its policy may raise concerns for crypto-related businesses.
6. It’s vital to pay close attention to slashing parameters. While homogeneous chains (e.g. cosmos based) might have a lot in common, alterations by even some minor aspects can lead to penalties or ‘jailing’. Responsible DevOps professionals must always be alert to these factors, keeping in mind the block window pertinent to a specific blockchain during which they should execute all fixes in case of emergency.

The Takeaway

At Everstake, we are confident that implementing the recommended strategies will result in smooth chain launches and stable ongoing operations, uniting chain teams and staking providers for the mutual benefit of everyone involved. We hope this article reaches its intended audience and enhances the collaboration between validators and blockchain teams, fostering increased understanding and harmony. Please feel free to email us with your valuable insights or feedback regarding the matters mentioned above or other queries. We warmly welcome opportunities for collaboration and forging new partnerships.