Announcement: Shinkai’s First Testnet Goes Live

TLDR: Shinkai is a groundbreaking AI-powered operating system designed to drastically enhance the capabilities of Large Language Models (LLMs) by securely integrating with your personal data through a revolutionary AI-specific file system called VectorFS. This is all within a decentralized peer-to-peer network that prioritizes user privacy and sovereignty. The first testnet introduces features such as the Shinkai Node, Chrome Extension, Zapier Integration, and more. Shinkai also offers libraries and APIs for developers to create custom integrations. The project aims to open-source the majority of its stack in the near future.

What’s Shinkai?

Shinkai is an AI-Powered Operating System crafted to augment the functionality of Large Language Models (LLMs). It is characterized by three primary features:

• Privacy Comes First: Shinkai allows you to securely engage an LLM with your personal emails, files and data, with the assurance of privacy. It prioritizes secure interactions, allowing you to communicate without compromising your sensitive information.
• Continuous Flow of Fresh Info For Your AI: Shinkai keeps your AI informed with the most recent data from various sources including news, sports, and cryptocurrency. This ensures that your AI has the latest information to assist you effectively.
• AI Tools for Everyday Tasks: Shinkai equips your AI with the necessary tools and internet access to perform a wide array of tasks. Whether it's finding travel deals or providing tips, Shinkai enables your AI to assist you with your daily activities.

Central to the Shinkai ecosystem is the Shinkai Node, which integrates a revolutionary AI-specific file system called VectorFS within a decentralized peer-to-peer network. VectorFS represents a step-function improvement over existing file systems by being specifically optimized for efficient storage and retrieval of AI data embeddings at scale. This powerful combination allows for a robust and private infrastructure that radically enhances the capabilities of your AI.

Shinkai Aims To Be Much More Than That

AI is advancing rapidly, and with this progress come many questions across a broad range of topics. However, one important issue we aim to address is the exponential centralization of personal information and the inference bias by a few AIs that are owned and controlled by a mere handful of companies worldwide.

We are creating software that grants you sovereignty over your AI, ensuring you are in complete control. This allows you to use an AI system as the new foundation for your "intelligent digital life," accessible through an AI. In many ways, Shinkai acts like an operating system with many core complex features, but one that can run as a simple app. It supports your privacy across emails, files, personal conversations, and the analysis of your medical records and financials. It offers unbiased education and reporting, truthful data analysis, and the freedom to explore without feeling watched or fearing repercussions for seeking understanding.

Shinkai is committed to open-source, self-sovereignty and freedom of unbiased computation.

The Power of the Shinkai Node

The Shinkai Node is a very complex piece of engineering that extends the capabilities of LLMs by implementing several core primitives required to achieve our vision of a foundational system for a sovereign intelligent digital life.

Features enabled by the Shinkai node:

• Full Privacy: When running locally with a local LLM (e.g., Ollama), you get full privacy for processing local files and inferencing. You can even use it in an airplane without an internet connection.
• A Web Assistant: Navigate the web with an AI assistant next to you. By using the Shinkai Visor Chrome Extension, you get a nice user-friendly UI in the browser that enables you to send information from any webpage to your Shinkai Node for processing. You can even take screenshots of a webpage for processing.
• Top Notch Talk to PDFs and Other Files. You are able to upload almost any type of document and have your AI help you process them (pdf, word docs, excel docs, images, etc).
• Integration with Centralized and Open-source LLMs: Shinkai works with open-source solutions like Mixtral, Llama2, among others, while also supporting ChatGPT and other providers.
• On-Chain Identities Using a Decentralized Peer-to-Peer Network: Shinkai is integrated with an on-chain decentralized identity system that allows for intra-node communication. It’s a decentralized Whatsapp / Signal that’s censorship-resistant. This system also includes a permission system that leverages delegation. This work is a stepping stone for extending the knowledge of LLMs and decentralized Agent capabilities.
• Shinkai SaaS Solution: For those users that prefer a ready-to-go solution in just a few clicks.
• Creation of New Services Through Our Libraries and Integrations: Shinkai comes with multiple libraries in Rust, Typescript (Native), WASM (Typescript Wrapper), and Python.

Upcoming Features:

• (Testnet 2) Shinkai Desktop App with an embedded Shinkai Node 
• (Testnet 2) Extended Knowledge for LLMs using the Decentralized Shinkai AI Data Network
  • This will open up the creation of decentralized “GPTs”
• (Testnet 3) Mobile Apps
• (Testnet 3) Tooling System for LLMs
  • Very likely, this will replace Zapier for most use cases and it will open to door for the creation of zk proofs of the computation from these tooling which can be grouped with zk proofs of inferencing to generate proofs for trustless agents.
• (Testnet 3) Distributed Agent Capabilities
  • This will open up the creation of decentralized agents.

Technical Foundations Included in the Node:

• The creation from scratch of an AI vector database on top of RocksDB that includes a refined permission system, making it more akin to a file system specialized in AI embeddings.
• Implementation of an advanced RAG system from scratch that’s on par with the top current systems out there.
• A decentralized network built on top of smart contracts that allow nodes to find and connect to other nodes in a completely decentralized manner.
• Implementation of privacy-preserving crypto primitives for signing and encryption in two different layers for a Shinkai Message, which allows for the creation of a robust API without the need for TLS and is fully compatible with a privacy routing protocol.
• Implementation of a decentralized messaging system that works in conjunction with the job system so AI agents are connected to a network.
• Multiple compatible libraries in Rust, WASM, Native Typescript, and Python which have enabled the integration and creation of applications using Replit, Zapier, Node.js, Cloudflare, among others.

We have come a long way and feel we are ahead of most self-hosted solutions for AI. However, our goals are much bigger, so this is only part of our first testnet, and we expect to have two more testnets that considerably extend the capabilities of the Node.

One key point that may be overlooked is that the Shinkai Node is entirely written in Rust on top of a RocksDB database. This is not only extremely high-performance but will play a critical role in delivering the best possible UX that an app can offer. Current self-hosting solutions for AI require technical knowledge from users to install them because they rely on Python (which requires the system to have certain libraries and dependencies) or other databases like PostgreSQL. We aim to make the Shinkai Node as easy to install as the Spotify app, and we are very close to achieving this — you can see that in a video demonstration later in this post.

Shinkai’s Whitepaper Goes Live

For a more detailed exploration of Shinkai's technical concepts and architecture, you can read the whitepaper or read our documentation here: https://docs.shinkai.com.

Shinkai’s Overall Architecture

For people that are more visual, this representation of the different internal and external components of Shinkai may be easier to understand.

What’s the First Testnet About?

The first testnet of Shinkai introduces a suite of features and capabilities that showcase the potential of this AI-powered operating system. Users can engage with the following functionalities:

• Shinkai Node: A core component that enables private, decentralized operations with AI, supporting both local and third-party Large Language Models (LLMs).
• Chrome Extension: Enhances web browsing by allowing users to interact with web content through their Shinkai Node, including sending webpage information for processing or taking screenshots for AI analysis.
• Zapier Integration: Connects Shinkai with hundreds of apps through Zapier, enabling automated workflows and AI-enhanced task execution.
• Smart Contract and Decentralized Messaging: Utilizes blockchain technology for secure, on-chain identities and censorship-resistant communication between nodes.
• SaaS Compatibility: Offers a Software as a Service (SaaS) solution for users to experience Shinkai's capabilities without setting up a local node.
• Decentralized AI Data Network: A glimpse into the future of decentralized AI, where nodes are able to find and communicate with each other in a censorship resistant way.
• Building Shinkai Integrations: Developers can use Shinkai libraries to create custom integrations, such as:
  • A Slack bot that processes messages using AI to manage tasks and update a Trello board automatically.
  • A Discord bot that answers questions about your project by leveraging Shinkai's integrations and the RAG system, providing real-time, accurate responses.
  • An IoT application that uses Shinkai's AI capabilities to analyze sensor data and make predictions or decisions based on environmental changes.
  • A customer support automation tool that integrates with CRM platforms, using Shinkai's AI to provide personalized responses and support ticket routing.
  • A home automation system that’s fully private using a raspberry pi, a camera and a shinkai node with a local LLM.

This testnet is an opportunity for users to experience the privacy, flexibility, and power of Shinkai, providing feedback that will shape future developments.

Shinkai Libs and API

Shinkai is designed to be developer-friendly, offering libraries and APIs for deep integration into various projects. The available libraries include:

• Rust: For high-performance, system-level integrations.
• Typescript: Facilitates development in web and Node.js environments.
• WASM (WebAssembly): Enables high-speed execution in web browsers, allowing for client-side AI processing. For compatibility reasons we also have the previous Typescript library.
• Python: Offers ease of use for data science and AI research projects.

These libraries ensure that developers can integrate Shinkai's capabilities into a wide range of applications, from web services to IoT devices.

Additionally, Shinkai is preparing to open-source integrations for popular communication platforms such as Slack, Discord, and Telegram, as well as for Zapier. This will allow non-developers to automate workflows and leverage AI in their daily tasks without writing a single line of code.

By providing these tools, Shinkai aims to foster a community of developers and users who can contribute to the ecosystem, creating a diverse range of AI-powered applications and services.

Galxe Quests

Sometimes the best way to learn is by doing. We created some campaigns for the folks that would like to be an early pioneer by testing the Shinkai Node and network capabilities. You can find the campaigns here:

https://galxe.com/shinkai

Shinkai Open-sourcing

We are fully committed to open-sourcing the vast majority of the Shinkai stack, including the Shinkai Node and various apps and integrations. This is a key priority for us in the near future.

What exactly will be open-sourced?  

- The complete Shinkai Node codebase, written entirely in Rust. This includes the AI vector database built on RocksDB, the advanced RAG system, decentralized networking and messaging components, and crypto primitives for signing and encryption. Open-sourcing the node will allow developers to inspect, modify and extend its capabilities.

- Libraries and SDKs in multiple languages including Rust, WASM, TypeScript, and Python. These enable developers to easily integrate with the Shinkai ecosystem and build their own applications on top of it. We want to empower a vibrant developer community.

- Key application integrations for platforms like Slack, Discord, Telegram and Zapier. This will showcase real-world examples of how Shinkai can be used to bring AI capabilities into popular tools and automate workflows. It also allows non-developers to leverage Shinkai without having to write code.

- Documentation, guides and examples to help developers get started quickly. We know having great docs is essential for adoption and building a strong open source community.

When will this happen?

We're targeting early April, less than a month from now, to have the bulk of the Shinkai stack open-sourced. This is an ambitious timeline but we believe it's achievable.

Before that, our main focus is on launching the second Shinkai testnet by the end of March. This is a major milestone that will allow LLMs to tap into extended knowledge from across the decentralized Shinkai Network. Once this is live, we'll turn our full attention to preparing the codebase for open-sourcing.

Why is open source important for Shinkai?

There are several key reasons we're committed to open source:

1. Transparency and Trust - Open sourcing the core Shinkai components allows anyone to inspect the code, understand how it works, identify potential issues, and verify that it matches our claims around privacy, security etc. This builds trust with users and the broader ecosystem. 

2. Extensibility - We want developers to be able to freely modify and extend Shinkai to unlock new use cases we haven't envisioned. Open source is essential for this. We're laying the foundation but it's ultimately the community that will take it to the next level.

3. Decentralization - While Shinkai can be self-hosted today, open sourcing is an important step towards further decentralization over time. It reduces reliance on us and allows anyone to run their own compatible node software.

4. Adoption and Ecosystem Growth - History shows that successful developer platforms tend to be open source. It reduces friction, encourages experimentation, and helps accelerate adoption and ecosystem growth. We want to enable a Cambrian explosion of apps and services in the AI and Web3 space.

In summary, open-sourcing is a natural progression for Shinkai and aligns tightly with our principles around transparency, extensibility, decentralization and empowering developers. Stay tuned for more updates as we approach this key milestone in the coming weeks. In the meantime, we encourage you to get involved in our testnets, provide feedback, and start thinking about the amazing things you'll be able to build with Shinkai. The future of decentralized AI is almost here and we couldn't be more excited!

Shinkai Node Binary Release

You can find the link here: https://docs.shinkai.com/getting-started

Shinkai's P2P Network Smart-Contracts

You can find it here: https://shinkai-contracts.pages.dev/

Shinkai SaaS

You can find it here: https://www.shinkai.com/get-shinkai#hosting

We are also sharing a demo here:

Shinkai Chrome Extension

You can find it here: https://chromewebstore.google.com/detail/shinkai-visor-supercharge/emahbbinjjmancgckmoodjficjdkjkif

We are also sharing a demo here:

Shinkai Desktop Application (Soon)

The Shinkai desktop application is a native application that can run in Linux, Windows and Mac natively and it embeds the Shinkai Node. It’s as easy to install as the Spotify App. We will share it in a couple more weeks. We are sharing an early sneak peak for you here:

We Thank Our Investors For Supporting Us

Archetype, Circle Ventures, Arrington Capital, Coinbase Ventures, Hypersphere, Borderless Capital, D1 Ventures, Seax Ventures, Assembly Capital, Sora Ventures among others. Also to our angel investors Balaji, Naval, Scott Belsky, Michelle Lai, Steven Kokinos, David Post, Serge Ugarte, Kenny Hertz, Andrea Kalmans, Nick Plante, Galen Law-Kun, Matt Batsinelas, among others.