Introducing Trdelnik Explorer: Enhancing Testing and Analysis Capabilities in Trdelnik Framework

Introduction

A. Introduction to the Trdelnik framework and its capabilities

Much like the unexpected connection between the traditional Czech pastry and program testing, the Trdelnik framework offers a creative and efficient approach to structuring and executing testing procedures. Designed for Solana programs within the Anchor framework, Trdelnik is an open-source testing solution equipped with a range of tools and utilities to simplify and enhance program testing.  

While Anchor can generate client program code in JavaScript/TypeScript from the IDL, this approach poses challenges in terms of security, performance, and reliability. Trdelnik, the Rust Testing Framework for Solana, steps in to streamline this process. It comprises a CLI tool and a versatile library, complete with test helpers and APIs that can interact with various plugins, including a fuzzer.  

The framework's ambition extends beyond its current components. It envisions generating test scenarios and schemas directly from the Anchor IDL and program code, along with the potential development of a Rust/Wasm GUI layer for the client – a tool specialized for Anchor/Solana. By pursuing these ideas, Trdelnik eliminates the reliance on JavaScript and enables the creation of Rust-exclusive apps using the Anchor framework.  

Security and reliability are paramount in the blockchain realm, and Trdelnik, coded in Rust, responds aptly. By facilitating direct testing within Rust, the same language used for Solana programs, Trdelnik eliminates the need for JavaScript. Leveraging Rust's safety features, the framework aims to uncover and mitigate potential issues that might go unnoticed with alternative languages, ensuring a more robust testing process.  

B. Overview of the need for Trdelnik Explorer within the Rust Testing Framework for Solana.

Within the Rust Testing Framework for Solana, Trdelnik Explorer arises as a crucial component due to the necessity of efficiently observing and analyzing blockchain changes. While the existing Trdelnik handles program testing, there's a distinct requirement to understand blockchain modifications comprehensively, including account changes, deployment implications, risk identification, and transaction details. Trdelnik Explorer addresses this gap by introducing a dedicated library integrated with Trdelnik, offering seamless interaction through the CLI tool. This enhancement caters to developers and security researchers, aligning with the framework's goals of comprehensive Solana program analysis and validation.

II. Trdelnik Explorer: Enriching Solana Blockchain Analysis

A. The Core Concept of the Explorer

Testing Hurdles: An In-Depth Exploration

Within the framework of the Rust Testing Framework for Solana, the need for Trdelnik Explorer emerges from a core necessity to elevate the proficiency of testing and analytical capabilities for Solana programs. While the Trdelnik Client aptly handles the intricacies of testing Solana programs, there remains a notable gap concerning the proficient observation of alterations occurring within the blockchain – a pivotal concern for both developers and security evaluation entities.

In the current state, the testing process primarily revolves around confirming the functionality of programs. However, it falls short in providing an encompassing understanding of several critical dimensions:

Account Changes between Transactions: Existing testing methodologies often fail to adequately address the dynamic changes that transpire in accounts between different transactions. This deficiency obscures a comprehensive comprehension of how programs affect accounts over time.

Implications of Program Deployment: While testing captures the immediate functionality of deployed programs, it frequently neglects to illuminate the broader ramifications of program deployment. The intricacies of how deployed programs interact with the overall system and the potential implications often remain elusive.

Risk Identification in Transactions: The identification of potential risks in transactions, including vulnerabilities or unintended consequences, is paramount for robust blockchain development. Existing testing mechanisms often lack the depth required for robust risk assessment.

Granular Transaction Inspection: The granular examination of individual transactions provides valuable insights into their inner workings. Current testing paradigms may not afford developers the granularity necessary to thoroughly dissect transactions, leading to an incomplete understanding of their behavior.

Filling the Developer-Centric Void

While consumer-facing blockchain explorers exist, there is an unmet need for a developer-oriented tool that can systematically extract essential blockchain information, analyze it, and present it in a manner tailored to the needs of developers and security researchers. This need stems from the distinct requirements of consumers and developers, with existing tools primarily catering to the former.

Trdelnik Explorer addresses this gap by introducing a new function within the Rust Testing Framework for Solana. It provides a dedicated library integrated into Trdelnik, allowing developers to efficiently analyze ledger changes through a user-friendly API. This library also offers compatibility with the Trdelnik CLI tool, enabling seamless interaction for end-users.

This initiative proposes a transformative solution that empowers developers and security professionals to gain valuable insights from the blockchain. Trdelnik Explorer aligns perfectly with the overarching goals of the Rust Testing Framework for Solana, enhancing the toolset available to Solana developers and testers while advancing the broader landscape of Solana program analysis and validation.

B. Proposal for the Implementation of Functions in Trdelnik

Functional Requirements

Input Data Format for Users

The Trdelnik Explorer will offer users the capability to input a diverse range of valid addresses for accounts, including:

• Ed25519 public keys.

• Program-derived account addresses (32-byte values derived from the ed25519 curve).

• Hashes of ed25519 public keys accompanied by 32-character strings.

Additionally, the explorer will process program ids, which are Ed25519 Public Keys, when inspecting programs. Similarly, any valid transaction id (the first signature of a transaction) will be processed for transaction inspection. If inputs are invalid or fail to yield information about accounts, program ids, or transaction ids, users will receive immediate notification of these circumstances.

Output Presentation Options for Users

To accommodate user preferences, the Trdelnik Explorer will offer a selection of output formats, including:

• CLI format, serving as the standard human-readable display.

• JSON format, designed for both human and machine readability, facilitating further data processing.

• JSONPretty format, a visually enhanced variant of JSON, achieved by incorporating spaces and line breaks, ensuring an easily readable and aesthetically formatted JSON output.

Examination of Accounts     

The Trdelnik Explorer will enable a comprehensive account inspection mechanism. Users can effortlessly observe all pertinent information about any account based on its address. The explorer's functionality extends to analyzing changes occurring before and after transactions, providing valuable insights into the dynamics of account alterations.

Assessment of Programs 

With the Trdelnik Explorer, users can conduct thorough inspections of programs using their program ids. Furthermore, the explorer boasts the capacity to discern the deployment approach used for a program. This analysis is particularly crucial for differentiating between non-upgradeable and upgradeable programs, which present distinct risk factors owing to single points of failure and the necessity for trust in upgradeable deployments.

Inspection of Transactions

The most intricate and enlightening aspect of inspection involves transactions. The Trdelnik Explorer offers an exhaustive transaction inspection facility, allowing users to meticulously examine transactions based on their transaction ids. Due to the complexity of Solana transactions, the explorer offers two distinct perspectives for inspection:

• Raw View, which presents transactions in their unprocessed state, avoiding data manipulation or interpretation.

• Interpreted View, which provides a processed transaction view, offering comprehensive interpreted data encompassing the transaction, invoked programs, instructions, and logs.

Comparing Account Changes 

A core capability of the Trdelnik Explorer is the ability to highlight differences in accounts before and after transactions. This feature enables users to observe the ramifications of program instruction invocations and facilitates side-by-side comparisons of any two accounts, enhancing the understanding of account dynamics.

Parsing of Instructions

In the absence of standardized program APIs, the Trdelnik Explorer steps in during transaction inspections (as described in FR-05). It endeavors to deserialize and parse as many program instructions as feasible, leveraging heuristics and alternative techniques to provide insight into program instruction executions.

Personalized Program Instruction Parsing

The Trdelnik Explorer offers an essential feature by introducing the option to incorporate custom program instruction parsers. This facilitates the addition of programs to the instruction parser, particularly useful for auditing undisclosed or unreleased programs, ensuring that the explorer remains adaptable to diverse auditing needs.

Account Data Reconstruction

The explorer's capabilities extend to account data deserialization when the account structure and serialization format are known. This functionality empowers users to comprehend the account data in its original form, providing a deeper insight into the intricacies of account content.

Compatibility Across Multiple Clusters 

The Trdelnik Explorer seamlessly integrates with all accessible Solana clusters, affording users the flexibility to effortlessly switch between targeted clusters. Among the most frequently utilized clusters are:

• Beta Mainnet – the official Solana cluster characterized by its permanent status and non-reversible nature.

• Devnet – an official Solana cluster specifically designed to assist developers in assessing their programs within an environment resembling the mainnet, often incorporating innovative features yet to debut on the mainnet.

• Testnet – an official Solana cluster dedicated to the experimentation and testing of novel Solana versions and functionalities.

• Localnet – an adaptable option encompassing any locally deployed Solana cluster, typically employed for preliminary testing prior to transitioning to the Devnet environment.

Precise Data Display Control

The explorer introduces the concept of granular visibility, permitting users to tailor the level of detail presented in the output. By customizing the information to their immediate needs, users can focus exclusively on relevant aspects, enhancing their ability to extract valuable insights from the blockchain data.

Non-functional Requirements

Selection of Programming Language

The Trdelnik Explorer will be built using the Rust programming language. This choice is strategic, as it ensures smooth integration with the Trdelnik CLI and other Solana ecosystem tools. Given that the Solana ecosystem leans towards Rust, this decision aligns well with the ecosystem's direction.

Cross-Platform Suitability

The explorer's functionality is aimed at desktop computers that are compatible with the Rust compiler and LLVM backend. This ensures broad platform support while maintaining efficient performance.

User Interaction Scenarios

The explorer's intended use encompasses two scenarios: as an integral component of the Trdelnik CLI and as a standalone library. This dual usability approach offers flexibility to developers.

Operational Efficiency

A lightweight design approach is a hallmark of the explorer's architecture. It operates locally, eliminating the need for external databases. This design choice prioritizes efficient resource utilization.

Extensibility Potential

Flexibility for future growth is a key consideration. The explorer's design aims to accommodate the addition of new features with relative ease, ensuring its adaptability to evolving requirements.

Error Detection and Reporting

The explorer is equipped with a robust error reporting mechanism. It promptly informs users about various issues, such as input errors, missing accounts or transactions, communication glitches with nodes, and other potential problems, enhancing the user experience and troubleshooting process.

C. Use Cases

Explanation of specific use cases, including searching the blockchain, displaying differences, and security inspection.

Blockchain Exploration

This use case empowers users to query and display specific components within the Solana blockchain, including accounts, programs, and transactions. Trdelnik Explorer facilitates this exploration, enhancing user accessibility to blockchain information.

Visualizing Discrepancies

In this use case, Trdelnik Explorer allows users to assess variations in the Solana ledger caused by transactions. Users can compare and contrast the state before and after transactions are executed, providing a clear understanding of the impact of each transaction on the blockchain's state.

Security Assessment

Trdelnik Explorer plays a vital role in security assessment. This use case enables users to meticulously examine program instruction invocations and state alterations. Additionally, it provides the capability to incorporate unknown instructions, a valuable feature during the auditing of undisclosed programs. By facilitating risk identification, Trdelnik Explorer contributes to enhanced security measures within the Solana ecosystem.

The use scenarios are depicted in the form of a use case diagram.

III. Implementation of Trdelnik Explorer

A. Technology Used

The successful implementation of Trdelnik Explorer involves the utilization of various technologies and tools, each playing a pivotal role in shaping its functionality and effectiveness. The careful selection of these technologies is instrumental in achieving optimal performance, safety, and seamless integration with the Solana blockchain.

Rust Programming Language

At the core of the implementation lies the Rust programming language. This choice is driven by Rust's unique blend of safety and performance attributes. Leveraging Rust ensures that interactions with the Solana blockchain are not only secure but also highly efficient, contributing to the overall reliability of the Trdelnik Explorer.

Tokio Runtime

Central to the explorer's asynchronous operations is the Tokio runtime. Tokio is specifically designed for crafting network applications in Rust, making it an ideal choice for building a tool that interfaces with the Solana blockchain. The asynchronous nature of Tokio facilitates seamless communication with the blockchain network, enhancing the overall responsiveness of the Trdelnik Explorer.

Integration with Solana Blockchain

Seamlessly integrating with the Solana blockchain is a foundational element of the implementation. This entails establishing connections with RPC listening nodes and employing protocols such as JSON-RPC and WebSocket. In-depth knowledge of Solana's internal workings, programming model, and core concepts is paramount for this integration to ensure accurate and reliable interactions with the blockchain.

Solana Client

Simplifying communication with Solana nodes is the Solana Client, a vital abstraction layer for the JSON-RPC and WebSocket protocols. By leveraging the Solana Client, developers are relieved from the burden of creating their own communication clients using HTTP libraries. This integration streamlines the development process and amplifies the efficiency of the Trdelnik Explorer.

Solana SDK Dependency

Another critical dependency for the implementation is the Solana SDK. This SDK encompasses crucial components such as type definitions, constants, and utility functions. Its incorporation enriches the capabilities of the Solana Client and contributes to the overall resilience and versatility of the Trdelnik Explorer.

B. Explorer Crate

Introduction to the explorer crate and its modules

The explorer crate consists of various modules, each with a specific role in the functionality of the explorer. The crate prioritizes the use of Rust generics for code simplicity, readability, and extensibility.

Detailed description of Config, Error, Display, and Output modules

Config Module:

The Config module serves the purpose of creating a configuration that can be later utilized within the Explorer API. It offers two main configuration options: 

• Async RPC Client Setup: This involves establishing and configuring an asynchronous RPC client to communicate with a predefined Solana cluster via an RPC node. By default, the client employs the preferred cluster specified in the configuration file of the official Solana Tool Suite (https://docs.solana.com/cli/install-solana-cli-tools).

• Logger Settings: The module also handles logger settings, with a default setting of the ERROR level.

Error Module:

The Error module functions as a repository for error definitions and the common return type definition of functions that can fail. It transforms errors originating from libraries into a standardized error type used across the entire explorer crate. These errors can be caught by either the crate's user or displayed to the user of the Trdelnik CLI in case of a failure.

Display Module:

Using generics to enable elegant printing, the Display module features the DisplayFormat enum, which offers three variants. These variants allow obtaining string outputs of any explorer item T that implements the fmt::Display and Serialize traits. This module contributes to enhancing the presentability of data.

Output Module:

The Output module constitutes the primary public API of the explorer library. It encompasses numerous public functions that can be employed as necessary for distinct use cases, some of which are directly integrated into the Trdelnik CLI tool.

For instance, the function print_transaction, exemplified in the snippet displayed in the picture below, showcases the public API of this module. It includes a complete function signature and definition for print_transaction. This function is designed to take a unique signature identifying a transaction, define what to display, specify the display format, and utilize the explorer configuration to communicate with the Solana cluster.

Account, Program, and Transaction modules for handling respective data

These modules contain essential data structures for representing Accounts, Programs, and Transactions. They enable precise data interpretation and granular visibility. Notably, the Transaction module is particularly intricate due to the complexity of Solana's transaction structure. This module relies on the parser module to handle tasks like instruction deserialization, vital for accurate data interpretation.

Parser module for program instruction parsing

The Parser module, including its submodules, focuses on parsing program instructions within transactions. This is vital for transaction deserialization and interpretation. Since a standardized approach for publishing program APIs in Solana is absent, interpreting data can be challenging.

Table for looking up programs that can be parsed

The chosen solution involves maintaining a lookup table of common Solana programs. This table, constructed during compilation, aids in identifying program instructions during transaction processing. Recognized instructions are decoded and interpreted based on known attributes. Unrecognized programs display associated account information and raw instruction data.

The logic for parsing instructions

Deserialization and interpretation logic for recognized programs reside in submodules within the Parser module. Initiatives, led by the Anchor Framework, aim to upload a program's Interface Description Language (IDL) to deterministic blockchain addresses. This automation could be integrated into the explorer's future versions.

Integrating Trdelnik

While the explorer can function independently as a library, there's also a desire to incorporate certain features within the Trdelnik CLI tool. To accomplish this, enhancements have been made to the trdelnik-cli crate of the Trdelnik CLI tool, ensuring support for common use cases directly from the command line.

IV. Testing

A. Testing in Rust

In Rust, both unit and integration testing are inherently supported. Tests in Rust are essentially specialized user-defined functions that validate the accuracy of the non-test code. The sole distinction is that, to ensure they are omitted during regular compilation, they need to be appropriately designated.

B. Unit Testing

The parser is a vital component within the explorer framework, responsible for the deserialization and interpretation of program instructions that can be parsed, as elaborated earlier. To validate the parser's accuracy, every submodule of the parser module has undergone thorough unit testing. This comprehensive testing approach ensures the proper recognition of various instruction types.  

The only exception is the absence of a unit test for a new marginal instruction of the stake program, as it lacks proper documentation. This test will be incorporated once adequate information is accessible to construct a precise unit test.

C. Integration Testing

Given the dynamic nature of public blockchains and the constant changes in tested items, automating testing in Trdelnik Explorer is intricate. While simulating a local blockchain with test transactions is an option, this approach carries significant coding overhead and abstract value. As a result, beyond Parser unit testing, the validation process leans towards manual and semi-manual methods for overall output accuracy.

D. Functional and User Testing

The comprehensive testing regimen of the Trdelnik Explorer encompasses multiple facets. It has been rigorously tested against the Solana beta mainnet, rigorously evaluating its performance across various blockchain components: accounts, programs, and transactions. This exhaustive testing entails diverse output scenarios, ensuring correctness across the spectrum. For instance, account testing spans a range of scenarios including data accounts with different sizes, program-owned PDA accounts, program accounts managed by different loaders, and native accounts associated with both native programs and specialized runtime accounts.  

In the realm of program deployment, the testing scope encapsulates programs deployed using the BPF Loader deprecated, BPF Loader, and BPF Upgradeable Loader. Transaction testing takes a dual-pronged approach, analyzing transactions with program instructions that align with recognized parsable programs as well as a randomized sampling of transactions from the network. This thorough evaluation extends to the explorer's internal user testing, which has been instrumental in refining the framework based on feedback from Ackee Blockchain, a guiding presence throughout this thesis.

V. Conclusion 

Trdelnik Explorer has significantly bolstered Solana blockchain analysis, streamlining the process and offering valuable insights. The robustness of Rust's safety features has underscored its importance in testing, ensuring reliability and security. The journey of Trdelnik Explorer doesn't end here. It envisions further enhancements and advancements, including generating test scenarios from Anchor IDL, a Rust/Wasm GUI layer, and more. Acknowledgments go to Ackee Blockchain for their input, shaping this innovative tool for Solana developers.