Phase 2 Starts Today July 16th (Shinkai <> Coinlist Testnet)

Artificial Intelligence (AI) is rapidly advancing and is likely here to stay, transforming industries and enhancing our personal capabilities. An AI looking out for us would be extremely helpful in organizing our lives, making better financial decisions, keeping up and better communicating with friends and close ones, and much more.

However, for an AI to reach that degree of integration, it needs access to our lives, conversations, apps, financial services, etc. There is always the option to give all that information to one single company like OpenAI, and many people might be okay with that.

But for those who value privacy and ownership of their data, Shinkai aims to cover that need. Additionally, Shinkai is completely free and open-source. It also works towards agentic capabilities, allowing users to teach and command their AI to perform tasks autonomously.

Shinkai is a powerful platform that uses open-source AI models to perform a wide range of tasks locally, from answering questions to analyzing files and managing finances. It operates entirely on your device, ensuring no data is sent externally. The platform is easy to install and accessible on Mac, Windows, and Linux.

Shinkai also connects to a decentralized network for updating AI models, extending their knowledge, adding more capabilities through the cooperation of local and decentralized agents, and verifying information authenticity through cryptographic proofs. 

Shinkai launched its Phase 1 Testnet in collaboration with Coinlist (http://coinlist.co/shinkai-testnet) around three weeks ago and we were very happy with the results. Some of the achievements of Phase 1:

• Over 12,000 on-chain identities.
• Over 12,000 downloads of Shinkai Desktop and Chrome.
• Twitter followers up from 8,000 to 32,500 users.
• Discord up to ~9,900 members.
• A lot of useful suggestions and bug reports.

And today we are launching the new Phase 2 Testnet!

Link to Coinlist Campaign: http://coinlist.co/shinkai-testnet?ref=blog.shinkai.com

Link to Galxe Campaign (beware of the Coinlist campaign restrictions): https://app.galxe.com/quest/shinkai/GCfpWtkAN1

Phase 2 Testnet

Phase 2 is about testing the network capabilities of the Shinkai network. One of the key components of Shinkai is the ability to connect nodes (that look and feel like a simple app) trustlessly. The diagram below shows a very simplistic representation of how the smart contract works:

ou can read more about this in our documentation or in our technical whitepaper here or in the section below titled: "Technical Overview of Phase 2".

Phase 2 Schedule

The second testnet will be divided into multiple smaller phases, each with specific goals, each lasting one week.

Phase 2: First Week

The first week is focused on testing the Shinkai app, the network, and the privacy-focused proxy service (also known as the relayer). The goal is to test how the network operates under heavy conditions by users subscribing to folders, receiving updates, and using AI to interact with those folders and files.

Phase 2: Second Week

The second week will introduce new features based on our new domain-specific language (DSL) to utilize workflows and tools on top of Shinkai. During this period, we will offer a small set of workflows that you can use with subscription files (or even your own!). Some of these workflows are implementations of AI papers in just a few lines of code, thanks to the simplicity of the DSL language.

At the same time, we will open applications to be a node to offer content using the network. The applications will be open for 72 hours, and we will provide results 24 hours after. For applicants, we recommend thinking about what type of content you would like to share and having a good grasp of your system using Shinkai.

Phase 2: Third Week

In the last part of Phase 2, we will expand the app to also show subscriptions from the selected nodes. The nodes providing extra content will be evaluated on performance, responsiveness, and the type of content provided. Users will play a critical role in helping us find bugs, improve UI/UX, and battle-test the network. Also users will be able to delegate for access or directly pay for subscriptions using KAI testnet tokens.

As a reminder, Shinkai is not just about a fully local AI app or even a network of content, but the creation of a network where every app running on your computer is capable of performing tasks, gaining extra capabilities through the Shinkai network, or providing work or resources to the network. With the finalization of Phase 2, we hope to have accomplished the following:

• A two-click install local AI app that any user can easily install on Windows, Mac, or Linux.
• No extra technical knowledge required to participate in the Shinkai Network and extend some of the capabilities of AI.
• Any user could eventually leverage their Shinkai installation to provide services through the network.
• Integration of basic agentic capabilities to local AI with extended knowledge from the network (Phase 3 is all about more advanced agents).

Phase 2 Key Links

Link to Coinlist Campaign: http://coinlist.co/shinkai-testnet?ref=blog.shinkai.com

Link to Galxe Campaign (beware of the Coinlist campaign restrictions): https://app.galxe.com/quest/shinkai/GCfpWtkAN1

See Shinkai In Action

Shinkai Protocol

Video’s delen met vrienden, familie en de rest van de wereld

YouTube

Technical Overview of Phase 2

Main Technical Points Of Shinkai Phase 2

• Shinkai integrates blockchain to ensure decentralized, censorship-resistant identities.
• Identities are registered as NFTs by staking Shinkai tokens.
• Identities can be transferred, with ownership held in users' wallets.
• Sub-identities allow intricate device and agent management.
• Router nodes enhance privacy and message delivery assurance.
• The Identity Registry supports staking, delegation, and node information.
• ShinkaiMessage utilizes strong encryption for secure communication.
• Node-to-node communication uses TCP and REST APIs with AES-GCM encryption.
• The system supports batch operations for efficient identity management.
• Shinkai's architecture ensures upgradability and cross-chain compatibility.

For the ones that are interested in understanding more of the network architecture the rest of this blogpost was written for you.

Decentralized Shinkai Identities and Their Smart Contract

The Shinkai Network integrates directly into blockchain to implement a decentralized and censorship-resistant system for Shinkai identities. Storing public node data on-chain ensures that Shinkai nodes within the ecosystem can effortlessly discover and communicate with each other. The decentralized Identity Registry smart contracts are central to Shinkai, enabling secure and autonomous interactions for managing identities with no middleman required.

Shinkai Identities

As mentioned above, at the core of the Shinkai Network is the concept of Shinkai Identities. These identities serve as unique "names" for users/nodes on the network, akin to a domain name, but with broader functionalities. By staking Shinkai tokens, with the amount required inversely proportional to the length of the identity name, users can register an identity on-chain (e.g., @@alice.shinkai). This inversely proportional relationship ensures that shorter (desirable) names are harder to acquire, thereby preventing name squatting while encouraging the efficient use of the whole namespace.

Key Features of Shinkai Identities

• Decentralized Identity Registration: Identities (as NFTs) are registered/created by staking Shinkai tokens, with the amount required inversely proportional to the name's length, prioritizing the efficient use of the namespace.
• Ownership and Transferability: Registered identities are owned and held in a user's wallet as an NFT, with the capability to transfer ownership of the identity (and thus associated stake) to other addresses.
• Connects With The Shinkai Node: Your identity connects directly with your Shinkai node, allowing all messages, files, or anything else anyone (or any AI Agent) sends to you to seamlessly arrive at your node.
• Dynamic Management: Owners can adjust their stake, withdraw excess tokens without forfeiting the identity, and fully unstake to relinquish ownership (thereby burning the NFT).
• Infinite Sub-identities: Underneath a single Shinkai identity, an infinite number of sub-identities can be created for all devices/computers/AI agents that the user owns.
• Router/Proxy Nodes: By specifying one or more router/proxy nodes, the network supports further scalability, enhanced message delivery assurance, and privacy.
• Batch Operations: Facilitates the management of multiple identities through batch registration, transfer, and unstaking processes.

Routing Nodes (Relayer Nodes) and Network Resilience

Routing nodes offer a strategic solution to the challenges often faced with direct P2P connections. These nodes allow users looking for enhanced privacy (hiding the public IP of their node) or ease of use via tunneling through restrictive firewalls while also ensuring consistent message delivery across the network (backed by end-to-end encryption).

Furthermore, nodes with extremely high demand may utilize multiple routing nodes as balance loaders to ensure high connectivity even under serious traffic. All together, routing nodes offer an on-network native solution to overcome classical P2P issues.

Shinkai's On-Chain Identity Registry and Delegation Mechanism

Shinkai's on-chain Identity Registry supports all core capabilities for managing and operating Shinkai Identities in a fully decentralized manner. Each identity lives on-chain and contains information about:

• Staked Tokens: Reflects the Shinkai tokens staked underneath the identity. Each identity's staked tokens are distinct, ensuring isolated management and facilitating specific actions like partial withdrawals or stake increases without affecting other identities.
• Node Addresses/Proxy Nodes: Critical for the network's decentralized infrastructure, allowing identities to specify direct node addresses or utilize proxy nodes for enhanced connectivity and privacy.
• Encryption and Signature Keys: Underpinning secure communications across the network, these keys ensure nodes can decrypt and validate who they are sending/receiving messages from.
• Tokens Delegated: The amount of Shinkai tokens the identity has delegated to others.
• Incoming Delegations: The amount of Shinkai tokens other identities on the network are delegating to the identity.
• Records HashMap: Provides a flexible datastore for adding additional information related to an identity, enabling data attestation and extensibility for other protocols.

The delegation of tokens is a cornerstone of Shinkai's economy, enabling identities to support other nodes/service providers on the network with no added personal cost.

Networking and The Shinkai Message

For many projects, networking and messaging details may just be a footnote. However, for Shinkai, with multiple systems interacting with the Node, it's crucial to explain the core mechanics at play that the Node interacts with and supports.

• Node-to-Node Communication: Utilizes TCP for the transmission of ShinkaiMessage as the core primitive. Also supports sharing an AES-GCM symmetric key as a session key for direct file transfers.
• Node API Communication: Implements REST APIs that leverage ShinkaiMessage, with strong security through encryption and message signing which guarantees Shinkai identity verifiability. Nodes and clients must synchronize their clocks to prevent the rejection of old messages. To avoid message replay, the Node records the hash of each message, ensuring they cannot be reused. Additionally, clients can share an AES-GCM symmetric key within an encrypted ShinkaiMessage for easier file uploads. WebSocket (WS) connections, secured with AES-GCM, facilitate real-time updates between Nodes and Clients.

ShinkaiMessage Structure

A ShinkaiMessage is a complex structure designed to facilitate secure and versatile communication and data transfer within the Shinkai ecosystem. It consists of several components:

• Body (MessageBody): Carries the actual content of the message, supporting both encrypted (using ChaCha20Poly1305) and unencrypted data. The encrypted body ensures secure transmission, while the unencrypted option is available for specific high-throughput, non-sensitive use cases.
• External Metadata (ExternalMetadata): Encapsulates essential delivery and verification information, including sender and recipient identities, scheduled time, signatures (using Ed25519), and additional fields for custom use cases.
• Encryption (EncryptionMethod): Details the encryption used for the message, currently supporting x25519_dalek for key exchange and ChaCha20Poly1305 for encrypting message content.
• Version (ShinkaiVersion): Indicates the ShinkaiMessage format version, ensuring compatibility and facilitating a clean upgrade path for the future.

Each component of the ShinkaiMessage plays a crucial role in securely transferring data within the Shinkai ecosystem by leveraging state-of-the-art cryptography for both signing and encryption.

Lifecycle and Example of ShinkaiMessage Communication

Let's take a look at the process of sending a message between two users, Alice and Bob, on the Shinkai network. Alice wishes to send the message "Welcome to Shinkai!" to Bob. This involves several steps and interactions including with Alice's device, Alice's Shinkai Node, the Blockchain, Bob's device, and Bob's Shinkai Node.

1. Identity and Key Discovery: Alice retrieves Bob's Node information (IP, port, or router/proxy) and public encryption keys from the Identity Registry on the blockchain (using Bob's identity @@bob.shinkai), and queries Bob's Node for his device's keys as well.
2. Message Preparation: Alice crafts a ShinkaiMessage with an inner layer targeting Bob's device and an outer layer for Alice's Node. The inner layer can be decrypted with Bob's device encryption key, while the outer layer via Alice's Node encryption key, following a Diffie-Hellman key exchange.
3. Signing and Sending: Alice's device signs both the inner and outer layers of the message to confirm its authenticity and sends it to her Shinkai Node.
4. Node Processing (Alice's Side): Alice's Node decrypts the outer layer and verifies Alice's signature. It then uses Bob's Node information from the blockchain to re-encrypt the outer part of the message, but now targeting Bob's Node. This re-encryption ensures a high level of privacy and limited data leakage.
5. Transmission to Bob's Node: The message is sent over the network from Alice's Shinkai Node to Bob's Shinkai Node.
6. Node Processing (Bob's Side): Bob's Node decrypts the outer message, verifies the signature against Alice's Node information on the blockchain, and securely stores the message for Bob to read from one of his devices.
7. Delivery to Bob: When Bob connects to his Node, the message is delivered to his device, and the inner message is finally fully decrypted and read. This provides end-to-end encryption, and allows for private communication to be possible between users even when using 3rd party hosting providers for the Shinkai Node.

Thank you everyone! We'll come back with more exciting news very soon.