Skip to content

Commit 91756bb

Browse files
committed
feat: add documentation and update readme to reflect llm v2 changes
1 parent 972c787 commit 91756bb

10 files changed

Lines changed: 623 additions & 77 deletions

File tree

Readme.md

Lines changed: 27 additions & 76 deletions
Original file line numberDiff line numberDiff line change
@@ -1,104 +1,55 @@
1-
# BitNet.js
1+
<p align="center">
2+
<img src="docs/assets/bitnet-stack.png" alt="BitNet-Stack" />
3+
</p>
24

3-
[![License: MIT](https://img.shields.io/badge/license-MIT-purple.svg)](https://opensource.org/licenses/MIT)
4-
![version](https://img.shields.io/badge/version-1.0.0-purple)
5+
<h1 align="center">BitNet-Stack</h1>
56

6-
BitNet.js is the Node.js implementation of Microsoft's [bitnet.cpp](https://github.com/microsoft/BitNet) inference framework. This repository facilitates real-time interaction between a Node.js frontend and the bitnet 1-bit LLM model using **Socket.IO**. The app allows users to send queries to the BitNet LLM (Large Language Model) and receive responses line by line via a web interface.
7+
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](https://opensource.org/licenses/MIT)
78

8-
## Working Example
9+
Run a small BitNet model on local machine with one Docker command, and chat in browser.
910

10-
Web app will display the results in real-time.
11+
## What you get
1112

12-
### Docker App Run
13+
- **One Docker Compose command** — Build and start the LLM server with a single command. The image pulls the **official small BitNet GGUF** from Hugging Face (`BitNet-b1.58-2B-4T-gguf`, `ggml-model-i2_s.gguf`) so you do not manage weights by hand.
14+
- **Web interface** — Open the app in your browser, start a chat, and talk to the model from a simple page.
15+
- **Chat history in the browser** — Conversations are stored in **local storage** on your device. They stay until you clear them.
16+
- **Clear all chats** — Use the control in the UI to remove every saved thread and stop model sessions in one go.
17+
- **Session context** — Each chat keeps a **conversation session** with the model so follow-up messages stay in context until you start a new chat or clear data.
18+
- **Responsive replies** — Streaming output and a tuned server path so answers show up quickly as they are generated.
19+
- **Runs anywhere Docker runs** — On a laptop, a desktop, or a small server: clone the repo, run `docker compose`, open the URL. No extra install steps for Node or Python on the host.
1320

14-
![bitnet js-docker-run](https://github.com/user-attachments/assets/8c439904-fec7-4465-ac7d-5ebfc29d2442)
21+
## Working example
1522

16-
### Web Interface Interaction
17-
18-
![bitnet js-llm](https://github.com/user-attachments/assets/83882cf5-0fb4-4fe4-9068-6447e26c4283)
19-
20-
21-
## Features
22-
23-
- **Web-based interface**: A simple frontend built with HTML and JavaScript to interact with the BitNet model.
24-
- **Real-time communication**: Uses **Socket.IO** for bi-directional communication between the Node.js app and the Python-based BitNet model.
25-
- **Dockerized environment**: Both the Node.js app and BitNet model run in separate Docker containers managed by **docker compose**.
26-
27-
## Directory Structure
28-
29-
```bash
30-
bitnet.js/
31-
├── apps/
32-
│ ├── llm/
33-
│ │ ├── Dockerfile # Dockerfile for the BitNet model
34-
│ │ ├── requirements-local.txt # List of packages that are used in Local Server
35-
│ │ ├── run_model.py # Local Python Server to start Socket
36-
│ ├── web/
37-
│ │ ├── Dockerfile # Dockerfile for the Node.js application
38-
│ │ ├── app.js # Node.js app (Socket.IO client)
39-
│ │ ├── index.html # Frontend (textarea for input, display for output)
40-
├── docker-compose.yml # Compose file to run both containers together
41-
├── README.md # Documentation
42-
```
23+
![bitnet-stack](docs/assets/conversational-chat.gif)
4324

4425
## Prerequisites
4526

46-
- **Docker**: Ensure Docker is installed. [Install Docker](https://docs.docker.com/get-docker/)
27+
- [Docker](https://docs.docker.com/get-docker/) (with Docker Compose)
4728

48-
## Setup Instructions
29+
## Quick start
4930

50-
### Step 1: Clone the Repository
31+
### 1. Clone the repository
5132

5233
```bash
53-
git clone https://github.com/stackblogger/bitnet.js.git
54-
cd bitnet.js
34+
git clone https://github.com/stackblogger/BitNet-Stack.git
35+
cd BitNet-Stack
5536
```
5637

57-
### Step 2: Build and Run Containers
58-
59-
To build and start both the Node.js app and the BitNet model, use:
38+
### 2. Start the server
6039

6140
```bash
6241
docker compose up --build -d
6342
```
6443

65-
This command will build the Docker images for both the **web** (Node.js) and **llm** (Python) containers, and run the services defined in the `docker-compose.yml` file.
66-
67-
### Step 3: Access the Web Application
68-
69-
After starting the containers, access the web interface by navigating to:
70-
71-
`http://localhost:3000`
72-
73-
You'll see a textarea where you can input your queries and send them to the BitNet model.
74-
75-
### Step 4: Interact with the BitNet Model
76-
77-
1. **Enter a query** in the provided textarea (e.g., "Why is the sky blue?").
78-
2. **Click Send**. Your query is transmitted to the BitNet model via Socket.IO.
79-
3. The response will be returned **line by line** in real-time.
80-
81-
## How it Works
82-
83-
- The **Node.js application** (in `apps/web`) serves a simple web interface where users can input their queries. The application connects to the **Python backend** using Socket.IO.
84-
- The **Python app** (in `apps/llm`) runs the BitNet model, processes the queries, and streams the response back to the Node.js client, word by word.
85-
- Both applications are containerized and communicate through **docker compose**.
86-
87-
### Node.js Frontend (app.js)
88-
89-
The Node.js frontend sends user queries to the backend via a Socket.IO connection. Here's the simplified workflow:
44+
This builds the **LLM** image and starts one container. The model is downloaded during the image build (first time can take a while).
9045

91-
1. **User Input**: The user enters a query and clicks "Send."
92-
2. **Socket Emission**: The query is sent to the backend using Socket.IO.
93-
3. **Receive Data**: The response is received word by word and displayed on the page.
46+
### 3. Open the chat UI
9447

95-
### Python Backend (BitNet Integration)
48+
In your browser go to:
9649

97-
The Python backend runs the BitNet model and returns the response:
50+
**http://localhost:5001**
9851

99-
1. **Receive Query**: The Python app receives a query from the Node.js frontend.
100-
2. **Model Inference**: The query is processed by the BitNet model.
101-
3. **Stream Results**: The model's response is streamed word by word back to the Node.js client.
52+
(Port **5001** is mapped to the app inside the container on **5000**; change it in `docker-compose.yml` if you need another port.)
10253

10354
## License
10455

apps/llm/static/index.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -450,7 +450,7 @@
450450
<body>
451451
<aside>
452452
<div class="side-head">
453-
<div class="logo">BitNet <span>local</span></div>
453+
<div class="logo">BitNet <span>stack</span></div>
454454
</div>
455455
<button type="button" class="btn btn-new" id="btnNew">New chat</button>
456456
<div class="history">

assets/bitnet-stack.png

82.4 KB
Loading

docs/assets/bitnet-stack.png

256 KB
Loading

docs/assets/clear-all-chats.gif

349 KB
Loading

docs/assets/context-memory.gif

557 KB
Loading
310 KB
Loading

docs/assets/generate-article.gif

328 KB
Loading

docs/assets/generate-code.gif

670 KB
Loading

0 commit comments

Comments
 (0)